Reading scanned freight PDFs with an LLM, validation first

Most shipper load lists can be parsed deterministically (how). One of SmartLoads' first agency's shippers sends a daily PDF that's a scan: an image of a page with no text layer. There's nothing for a text parser to read. That's where an LLM earns its place, and it's also where it can do the most damage, because its answer goes onto a live load board that posts to Truckstop automatically.

The approach in one sentence: treat the model's answer as untrusted input, exactly like a file from a stranger.

Where SmartLoads uses an LLM, and where it doesn't

For scanned load lists the order matters: a plain-text parse runs first, and Claude is called only if that finds zero loads. A shipper who starts sending a PDF with real text gets the deterministic parser automatically, and a switch can force either path.

The request

The PDF is sent to Claude as a base64 document block, so the model sees the page itself rather than OCR output. The prompt describes the document and ends with a strict contract:

Return ONLY valid JSON, no other text, in exactly this shape:
{ "sections": [ { "facility": "...", "loads": [ { ...fields... } ] } ] }

The output is grouped by facility, the way the page is. A later section explains why.

For photos of weight tickets we use a forced tool call instead of a JSON instruction. The model must answer through a function with typed fields, including legible and confidence, so "I can't read this" is a first-class answer rather than a guess. Those calls retry up to three times on rate limits and server errors, with a 60-second timeout. Paperwork extraction has a 90-second timeout, and a response cut off at the token limit is treated as an error, never as a shorter answer.

Validation

Nothing the model returns goes straight to the board.

1. All or nothing at the document level. If any section of the JSON is malformed, the whole response is rejected and the import fails loudly. An earlier version dropped the bad section and imported the rest. That looked like success while silently losing loads.
2. Required fields per row. A load without its rate is dropped and logged as a parse error, not posted with a blank.
3. Batch sanity checks. They look for patterns that suggest a column shift: numbers in the equipment field, most rows missing a rate, street addresses in the city field, one value repeated down a whole column. These raise an import alert for a person.
4. Holds before posting. Loads whose rate leaves too thin a margin go on the board but aren't posted or exported until an admin approves. The same goes for any load whose city can't be resolved.

Five lessons from real bugs

Each of these was a production incident, and each changed the design.

1. Reject, don't salvage. When one section of the JSON was malformed, the old code skipped it and imported the rest. The board looked fine and was missing loads. Now a malformed answer fails the whole document, and a person looks.

2. The prompt must match the page's structure. An early prompt treated the page as one facility's list. The model put all 13 loads under the first facility when there were really four. Asking for sections by facility fixed it, because the output shape now mirrors the document.

3. When a prompt fix doesn't hold, write code. Two cities printed under one rate header were sometimes merged into one. A prompt change fixed it on some runs and not others, which is expected: the model isn't deterministic. We replaced the prompt fix with deterministic post-processing that splits them after extraction. Use the model for reading and code for rules.

4. Never derive identity from the model's output order. Generated load numbers once included the load's position in the model's answer. The model doesn't list loads in the same order every time. So re-importing the same document produced different numbers, collided with a unique key after the old rows were removed, and left the board 18 loads short. Identity now comes only from the load's own content (how fingerprints work).

5. Scope replacements by date. A new document replaces the previous one's loads. When the replacement wasn't scoped to the ship date, tomorrow's list deleted today's loads. The replacement is now limited to the dates the new document covers.

What this costs

The scanned list arrives once a day, so the fallback is one model call a day. Paperwork and ticket reads are one call per document. The cost is small next to the time it replaces. The bigger cost is engineering attention, and that's what the validation and the lessons above are for.

Frequently asked questions

Why not use traditional OCR?

OCR gives you text, but a load list still needs structure: which city belongs to which facility, and which rate applies to which rows. A multimodal model reads the page's layout along with its text, so it can return structured loads directly. We still validate everything it returns.

Which model does SmartLoads use?

Anthropic's Claude (Sonnet models) for documents and images. For most jobs the model is a setting, so it can change without code changes.

How do you know the model didn't invent a load?

Every row must carry its required fields, and the batch is checked for patterns that suggest misreading. Loads still post from the same board as every other source, with the same holds.

Would you use an LLM for every format?

No. Wherever a deterministic parser can do the job, it's exact, fast, testable and free, and the same sheet produces the same loads every time.