Scanned pages turn RAG into an evidence problem
A local assistant can answer polishedly from a scanned PDF while relying on damaged, incomplete, or misordered text. Image-only documents lack a usable text layer; multilingual scans add language detection, script, layout, and query-language matching problems.
An Ollama RAG OCR PDF workflow must prove a chain from page image to readable OCR, page-preserving chunks, correct retrieval, and supporting page citations. Generic RAG starts at chunking; scanned-document RAG starts with diagnostics.
The deliverable is a proof artifact, not a chat window: a repeatable test pack containing image-only pages, mixed layouts, and every expected language. It exposes failures before contract archives, engineering drawings, or compliance review.
The evidence path from pixels to an answer
Ollama belongs at generation and, where suitable, embedding; it does not reliably read rasterized pages in a retrieval pipeline. PDF inspection, rendering, OCR, layout reconstruction, indexing, and citation assembly remain separate.
A defensible local flow has seven links:
- Inspect the PDF: classify each page as native text, image-only, or mixed; one document can contain all three.
- Render image pages consistently: retain page number, rotation, dimensions, source fingerprint, and rendering settings in the manifest.
- Run OCR with language and layout settings: store raw output, confidence signals, and bounding boxes where available.
- Apply quality gates: route weak coverage, suspect language detection, broken reading order, or missing named entities.
- Build citation-aware chunks: retain document ID, page number, excerpt, and location data. A chunk without lineage is text, not evidence.
- Retrieve with dense and lexical signals: semantic search supports paraphrases; lexical search protects rare names, identifiers, clause numbers, and OCR-distorted values.
- Generate only from retrieved evidence: Ollama receives chunks and citation labels, then cites pages or states that evidence is insufficient.
This separation shows whether a wrong date vanished in OCR, chunking, ranking, or generation; a chat interface hides these defects.
OCR gates must run before indexing
OCR quality is not cosmetic: a misplaced decimal, dropped negation, or misread surname changes meaning. High engine confidence does not prove table rows or two-column reading order survived.
Create page-level diagnostics for text density, detected language or script, rotation, confidence distribution, missing-character patterns, and suspiciously short output. Compare OCR with embedded text on mixed PDFs, but not as ground truth: embedded layers may be stale, truncated, or unrelated to the image.
For the proof artifact, create verified reference transcriptions for material headings, dates, names, reference numbers, amounts, table headers, and footnotes. Calculate Character Error Rate:
CER = character edits needed to match reference text / characters in reference text
CER finds OCR regressions but does not certify retrieval readiness. Also check whether hyphenated identifiers survive, table values stay attached to labels, and footnotes follow the claims they qualify. Whole-page scores can hide these failures.
Detect language by page or block, not only file. A bilingual page may contain English body text, a French legal notice, and a Cyrillic stamp. OCR packs must cover expected scripts, and embeddings must map query and source languages into a useful shared space. Visually review right-to-left text, diacritics, vertical stamps, and mixed Latin product-code characters during the pilot.
Do not use a universal pass score without document-risk review: a brochure and payment schedule need different tolerance. Review low-confidence pages, exclude them from answerable evidence, or label dependent answers unverified. Indexing flawed text makes a known input problem untraceable.
Page lineage makes citations inspectable
A citation must let readers reach the source page and see support without guessing the paragraph. A file name fails, especially after revised copies accumulate.
Attach to every chunk:
document_idand original-file content fingerprint- title, revision label, and ingestion timestamp
- physical PDF page number and rendered-image reference
- chunk ID, extracted text, and adjacent context when needed
- OCR settings, language label, and quality status
- bounding boxes or line references when available
Display a citation such as Policy Manual, page 14 with a short supporting excerpt. Page previews or highlighted regions help with dense tables and image-heavy pages, where extracted text cannot prove correct reading.
Formatting alone does not ensure faithfulness. Tests must confirm that a cited page supports the claim, not merely matching words: a model can treat an exception as the rule or cite a neighboring chunk sharing a heading. For high-stakes sets, require claim-level citations for dates, amounts, obligations, and named parties rather than one citation for a long paragraph.
Retrieval tests expose plausible failures
Score evidence selection before answer fluency. Define queries with known expected pages and preserve them as corpus, OCR engine, index, prompt, or model changes. Include:
- direct lookup questions with one exact-answer page
- paraphrased semantic-match questions
- cross-page questions requiring two citations
- unanswerable questions requiring abstention rather than invention
Add queries in every source language and, if supported, cross-language queries. A German query about an English scan tests a different path from an English query about an English scan. Add adversarial pairs: repeated headings, near-identical names, conflicting revisions, and a value appearing in both a table and footnote.
| Measure | Calculation | Failure revealed |
|---|---|---|
| Evidence Recall@k | expected evidence pages in top k / all expected evidence pages | OCR loss, weak chunking, poor ranking |
| Citation precision | citations supporting stated claim / citations shown | decorative or misplaced citations |
| Abstention correctness | unsupported queries correctly declined / unsupported queries tested | confident answers without evidence |
Track answer accuracy separately. A model may answer a direct lookup from prior knowledge or a misleading partial chunk; that does not show RAG worked. Review retrieved page, cited excerpt, and answer wording together.
Use deterministic expected-page checks before an LLM evaluator. An evaluator can flag phrasing defects but cannot be the only judge that page 37 is correct. Human review remains necessary for tables, handwriting, stamps, and layout-dependent questions.
A local proof artifact teams can rerun
Keep the pilot narrow: choose a bounded set and freeze a test pack before changing prompts or models. Store source PDFs, rendered pages, OCR output, chunk manifests, index version, test queries, expected citations, and results in version control or an immutable artifact store.
A practical runbook:
- Ingest PDFs; record hashes, page counts, detected text layers, and access classification.
- Render selected pages; verify orientation, resolution, and cropped margins.
- OCR the corpus; send gate failures to review.
- Create page-aware chunks; keep tables intact where possible, because row splitting breaks label-value relationships.
- Index with dense retrieval and lexical fallback; retain revision, language, and permission filters.
- Send top evidence to Ollama with strict instructions to cite retrieved pages, quote only supplied evidence, and abstain without support.
- Run the frozen suite and review every failed evidence query before changing more than one variable.
Show successes and failures. A correct-answer screenshot proves little; retain the page, OCR text, selected chunks, citation, expected evidence, and failure reason. This provides product, legal, and security teams a shared review object.
Component choices follow document failure modes
Choose OCR, retrieval, and generation by failing documents, not leaderboards. Clean single-language reports may need basic extraction and semantic retrieval; archival scans, forms, engineering pages, and multilingual agreements need stronger layout, language, and citation handling.
| Decision area | Prefer this when | Trade-off to accept |
|---|---|---|
| OCR with coordinates | citations need highlights or table verification | larger artifacts and more processing time |
| Page-level chunks | pages are coherent or legal citation matters | less precision on long pages |
| Section-level chunks with page links | headings and paragraphs reconstruct reliably | damaged layouts can split evidence wrongly |
| Hybrid retrieval | names, IDs, and OCR-distorted terms matter | extra ranking logic and test maintenance |
| Cross-lingual embeddings | query language differs from source | test language-pair quality; do not assume it |
No generation model repairs wrong transcription. More context can combine conflicting chunks and invite blending; deeper retrieval may improve recall while reducing answer-model signal. Tune k, chunk size, overlap, and reranking against the frozen pack, not intuition.
Local inference still requires role-based access, encrypted storage, audit logs, retention rules, and copied-export controls. The index, OCR cache, and answer logs can expose content; treat them as part of the document system, not harmless residue.
Pilot failures often start in mixed documents
Mixed PDFs are harder than image-only scans because they look healthy at file level. Selectable text can coexist with image-only annexes, signatures, charts, or amendments. Skipping OCR after finding any text layer creates searchable documents with invisible pages.
Revision drift also misleads. If a revised PDF changes pagination, old page-only citations become wrong. Bind citations to a content fingerprint and revision label, then invalidate or re-ingest affected chunks. Duplicate files require the same discipline: title matching is not identity matching.
OCR artifacts create retrieval traps. A number read as a letter may still match semantically, while an identifier with one wrong character can disappear from lexical search. Keep rendered pages available for disputed facts. For exact-value questions, answer policy should favor visual verification over fluent synthesis.
Make the evidence chain releasable
Start with scans the system handles least well: image-only pages, multilingual sections, tables, stamps, repeated revisions, and deliberately unanswerable questions. Build quality gates before the vector index, page lineage before the chat prompt, and retrieval evaluation before accuracy claims.
Ollama is the answer layer in a traceable pipeline, not a confident narrator on PDFs. When source page, OCR output, retrieved chunk, and citation agree, trust is earned; when they do not, failure is visible early enough to fix.