Engineering a regulated document-AI platform · Chapter 1
Engineering document AI for 10,000-page records
How we built a pipeline that reads every page of a 10,000-page medical record, proves it did, and scales with the work rather than the CPU.
Facing a similar engineering problem?
Discuss your projectThe engineering problem
Our client's records are not tidy 20-page PDFs: a single case can be a bundle of hundreds of files adding up to more than 10,000 pages, plus imaging studies with thousands of DICOM images. A reviewer expects answers the same day, and a lawyer or clinician will act on them, so the platform must read every page and be able to prove it.
Across production the pipeline has now read 175,000+ pages from 28,000+ documents and 21,000+ DICOM images.
Design principle 1: the page is the unit of work
Documents are split into single pages, and each page becomes an independent, retryable task that a Google Gemini model on Vertex AI reads as an image. One upload fans out into thousands of parallel tasks across a worker fleet.
One document fans out into page tasks; workers scale on queue depth because the work waits on the model, not the CPU.
Read this diagram as text
- One document is split into single-page tasks.
- The tasks wait in a queue.
- Workers pull tasks; each page is read independently and can be retried on its own.
- A monitor counts completed pages; the gate opens when every page is done.
- Note: CPU stays low; queue depth grows. Workers therefore scale on queue depth, not CPU.
Why it matters:
- Failure stays small. A transient model error costs one page retry, not an hours-long job.
- Resumable by design. Pages are content-hashed, so a restart, deployment or scaling event skips completed pages and does only the missing work.
- Parallelism follows the work. A 10,000-page record gets thousands of concurrent units, not one long thread.
Design principle 2: completeness is counted, not assumed
"Finished" is a claim the pipeline has to prove:
- The true page count of every file is saved before processing starts.
- Progress is reconciled against that count, not against pages produced so far.
- A document that yields fewer pages than it contains is marked incomplete, and the case is held with a clear reason rather than analysed on partial evidence.
- Recovery tools check for an incomplete split before they can release a case.
- Infrastructure errors such as disk exhaustion are classified as retryable, not as a fault in the customer's file.
Completeness is counted, not assumed: the release gate opens only when processed pages equal expected pages.
Read this diagram as text
- Expected pages: the true page count of every file is saved before processing starts.
- Processed pages: progress is reconciled against that count, not against pages produced so far.
- Release gate: held until processed equals expected; a document that yields fewer pages than it contains holds the case with a clear reason.
- A result is released only when every page is accounted for.
Design principle 3: keep every unit small
A naive PDF split copies a document's shared resources (fonts, images, colour profiles) into every single-page file, so image-heavy pages can balloon to tens of megabytes. Our splitter subsets each page so it carries only the resources it uses. The text layer is preserved, and image-heavy page files come out roughly 1,000x smaller. Each page file is deleted as soon as it has been uploaded, so intermediate files don't accumulate.
Subsetting each page keeps image-heavy page files roughly 1,000x smaller than a naive split.
Read this diagram as text
- Naive split: shared resources (fonts, images, colour profiles) are copied into every single-page file, so image-heavy pages can balloon to tens of megabytes.
- Subsetting: each page carries only the resources it uses and the text layer is preserved; image-heavy page files come out roughly 1,000x smaller, and each is deleted once uploaded.
Design principle 4: scale on the signal that matches the bottleneck
Model-based OCR is IO-bound: each task spends its time waiting on the model, so CPU stays low while the queue grows. CPU-based autoscaling would never react. Our worker fleets scale on real queue depth, per region, from a metric we verify is actually being published. In a recorded large-record run, queue-driven scaling more than doubled OCR throughput (~25 to ~56 pages a minute).
Design principle 5: time budgets that scale with the work
A watchdog that suits the median case would cut off the largest legitimate one, so:
- long steps emit a heartbeat, letting liveness checks tell "busy" from "stuck";
- completion budgets scale with page count, between a floor and a ceiling;
- total dispatches per case are bounded;
- a disk-fill alarm watches the processing fleet.
The outcome
- Records of 10,000+ pages processed end to end, with every page accounted for.
- 175,000+ pages and 21,000+ DICOM images read in production.
- Incomplete splits hold the case instead of releasing it.
Questions to ask of your own pipeline
- Is your unit of work small, retryable and resumable, or one long job per file?
- Does "done" mean the system has counted every expected unit?
- Can any recovery or admin action release a result that is missing input?
- Do your workers scale on queue depth, and have you verified the metric exists?
- Have you measured intermediate file sizes on your largest real input?
Next: Chapter 2, LLM operations at billion-token scale.