Interactive demo · synthetic dataEvery record on this page is fabricated.
Order Intake Trial
A hiring trial built as an engineering artefact, where seeded defects are graded by response and you are the candidate.
The assessment here is my own work — assessment design & direction, answer-key design, candidate evaluation. The dataset generator behind it was engineered by Prada Dipa — LinkedIn profile, opens in a new tab and Luthfi Aditya — LinkedIn profile, opens in a new tab under my direction.
THE BATCH, AS IT ARRIVES
38 images in two drops. The filenames are the ones a sales rep’s phone produces, so no order number, no page marker and no duplicate is discoverable without opening the picture. Nine are here.
ONE FORM, AND WHAT THE KEY SAYS TO DO WITH IT
Boxes are the generator’s own coordinates, pushed through the same rotation and crop as the photograph.
FINDINGS ON THIS FORM
Pick a finding, or click a highlighted region on the form. You are asked what the pipeline should do with it — not what it says.
THE TWO TRAPS
Every defect above announces itself. These two do not. A candidate hands in a spreadsheet and states two numbers; the numbers are graded against the generator’s ground truth, and a wrong one names which trap produced it.
One image is not one order. Two of these 38 images are halves of the same order, and three are photographs of orders that were already delivered yesterday.
THE FOURTEEN CLASSES
Counts are from this run of the generator: 39 seeded findings across 38 images. The right-hand column is the assessment — a candidate who extracts every character correctly and answers this column wrongly has failed, because a review queue that rejects the safe cases and accepts the dangerous ones is worse than no queue at all.
D1_SIZE_TOTAL_MISMATCH
Size breakdown does not sum to the stated total
Both numbers are legible and they disagree. Silently picking either one invents a quantity nobody wrote down.
FLAG FOR REVIEW
D2_UNKNOWN_STYLE_CODE
Style code is not in the product master
There is nothing to reconcile it against. It goes to the review queue with the reason attached, not into the order.
REJECT TO QUEUE
D3_CODE_NAME_MISMATCH
Code is valid, but the description belongs to a different code
Either field could be the mistake. Which one is authoritative is a business call, not an extraction call.
FLAG FOR REVIEW
D4_COLOR_NOT_AVAILABLE
Colour is not offered in that style
Not a spelling problem — the combination does not exist. Substituting the nearest colour ships the wrong garment.
REJECT TO QUEUE
D5_COLOR_NEAR_MISS
Colour is a near-miss spelling of a real colour
“Navy” is “Ink Navy”. This one normalises and goes straight through. Rejecting it is the wrong call — a pipeline that queues every near-miss buries the office team and gets switched off.
ACCEPT
D6_PRICE_MISMATCH
Unit price on the form differs from the master
The rep may have agreed a price. Overwriting it with the master price destroys the only record that it was agreed.
FLAG FOR REVIEW
D7_MISSING_SHIP_DATE
A required order-level field is blank
There is nothing to extract. An empty cell in the sheet reads as though somebody answered.
FLAG FOR REVIEW
D8_FREETEXT_INSTRUCTION
A handwritten instruction in the margin that is not an order line
“CANCEL the 2104 Ember Clay from PO MK-4460.” An extractor that only reads the table drops this completely, and nobody finds out until the cancelled goods ship.
ESCALATE TO A HUMAN
D9_IMAGE_ROTATED
Photo taken at an angle
Still legible. Extract it, or record low confidence — but queueing a form for a human because it is crooked is a false positive.
ACCEPT
D9_IMAGE_LOW_LIGHT
Dark, low-contrast phone photo
Same call as the rotated one. Bad photography is not bad data until it is actually unreadable.
ACCEPT
D9_IMAGE_CROPPED
The frame cuts off part of the table
The fields are genuinely not in the image. Guessing them is the exact failure this assessment exists to detect.
FLAG FOR REVIEW
T1_CROSS_BATCH_DUPLICATE
A batch-2 image is a re-send of an order already in batch 1
Dedupe on order number at the store, not per file. One image per order creates orders that were never placed.
DEDUPE ON ORDER NO.
T2_MULTI_PAGE_ORDER
One order photographed as two pages
“Page 1 of 2” and “Page 2 of 2” merge into one order of six lines. Treating each image as an order both loses three lines and invents an order.
MERGE THE PAGES
T3_NEW_FORM_TEMPLATE
The rep switched to a new form template mid-batch
Fields reordered, QTY relabelled UNITS, ACCOUNT CODE gone. Schema-driven extraction survives it; a position-hardcoded prompt drops fields and reports success.
ACCEPT
What is real, and what is simulated
Runs in your browser
- The dataset itself — generate_dataset.py re-pointed at MERIDIAN and re-run, so these forms are the generator's output, not pictures of it
- The fourteen-class defect taxonomy and all 39 seeded findings of this run, with their counts
- The answer key: every response you choose is checked against the key's own column for that finding
- The trap scorecard, graded against the regenerated ground truth of 34 unique orders and 98 line items
Provenance: the generator in this directory is the trial script with its brand, style master, account master and seed replaced — one substantive change beyond that, a stable checksum in place of Python's per-process salted hash, so the images are now reproducible as well as the key. The original seed, answer key, filename cross-reference, evaluator config and candidate brief are not in this repository and will not be: the instrument is still in use.
Simulated
- Almost nothing, which is worth saying plainly. There is no extraction model on this page — the forms are not read, they are known, because the generator wrote them
- Nine of the 38 images are shipped, chosen to cover twelve of the fourteen classes. The duplicate re-sends and the two-page order are graded in the scorecard instead of shown
- The shipped copies are downscaled to 900px wide; the delivered originals are 1240×1754 JPEGs carrying the same degradation