Table of Contents
- Agenda: data and systems for historical OCR
- A historical page is not a clean line of text
- Digitization still leaves an access gap
- Scholarly transcriptions as supervision
- Full-page VLM versus OCR pipelines
- HDML as a training representation
- Unifying CHURRO-DS
- Model comparison boundaries
- From transcription to HistoryGenie
- Layered evaluation
- A ten-page prototype
- Model scores are not the endpoint
- A concrete exercise
- Material gaps
- References
🌏 中文版
This guide reconstructs the lecture from the official Fall 2025 deck; system descriptions and reported results below are attributed to that historical course material unless a paper is linked at the claim.
The schedule labels Lecture 12 “NLP Building Blocks,” while its deck is specifically “Vision-Language Models to Make Historical Documents Accessible.” It is not a general NLP-components survey. It presents CHURRO from representation and data through model and historical-research application.
Agenda: data and systems for historical OCR
The lecture surveys digitization projects and document/OCR benchmarks, then explains gaps in layout, handwriting, languages, and annotation. It covers full-page extraction, HDML, CHURRO-DS, the CHURRO VLM, cross-model evaluation, and integration with WikiChat and HistoryGenie. (lecture source)
A historical page is not a clean line of text
Pages contain columns, footnotes, marginalia, captions, damage, and mixed writing systems. Plain OCR drops reading order and metadata; isolated crops lose page structure. Page-level VLMs retain the full image but need a consistent target representation. (lecture source)
HDML aims to represent the whole document and nothing but the document, combining text, layout structure, and metadata in a unique form. That supports training and dataset unification. CHURRO-DS combines existing and newly collected material across print, handwriting, languages, and centuries. This data engineering enables long-tail coverage.
Digitization still leaves an access gap
Scanned collections may be searchable only by collection metadata. Without reliable transcription, names, dates, and low-resource language words remain invisible. Modern document benchmarks cover layout and OCR, but historical paper, scripts, handwriting, and language distributions differ. (lecture source)
Flagship VLM errors on CHURRO motivate data acquisition rather than assuming scale alone fixes the domain.
Scholarly transcriptions as supervision
Historical publications often pair page images with expert transcriptions. Turning them into training data requires alignment because editions can normalize spelling, add notes, or omit unreadable text. (lecture source)
Author extension: The following alignment/provenance recipe and train/research/no-redistribution classification are data-governance recommendations from this article, not rules stated by the deck.
Preserve publication, page, transformation, license, and provenance rather than treating every printed transcription as pixel-level truth. Track collection terms separately and classify material as usable for training, research-only, or not redistributable.
Full-page VLM versus OCR pipelines
Traditional OCR separates layout, segmentation, and recognition; component errors cascade. Full-page models use cross-region context but can omit blocks or normalize language. Historical spelling and damaged text must be transcribed rather than made fluent. (lecture source)
Dynamic resolution helps with page shapes, while tiny text and large tables can still require tiling. Evaluate whole-page completeness as well as local recognition.
HDML as a training representation
HDML represents reading order, paragraphs, headings, tables, marginalia, image regions, missing text, and metadata in a consistent serialization. A simple schema loses layout; a complex one spends capacity on tags. Syntax validity and image-region alignment require separate checks. (lecture source)
Unifying CHURRO-DS
According to the official Lecture 12 deck, CHURRO-DS spans twenty-two centuries and multiple language clusters. Source datasets differ in annotation, normalization, crops, and metadata. They must be converted, aligned, deduplicated, and split without leaking neighboring pages of one document. Report documents, pages, languages, centuries, print/handwriting, and source—not only total size.
Per-language results need sample counts so large language clusters do not hide unstable long-tail performance.
Model comparison boundaries
The official deck identifies the compact base as a Qwen VL 2.5 3B-class model. CHURRO fine-tunes that compact open-weight VLM and compares commercial models, open VLMs, and OCR systems across print, handwriting, and languages. Fair comparison needs consistent resolution, prompt, normalization, and metrics. Closed APIs change, so the lecture table is a snapshot.
Character error alone misses omitted columns, reading-order errors, metadata, and critical names or dates. Preserve page, layout, text, and targeted entity evaluation.
From transcription to HistoryGenie
Indexing OCR output reintroduces retrieval and grounding failures. Answers should link to transcription and original page image. Normalized spelling can improve search but must remain separate from diplomatic text. (lecture source)
Classroom and historian use demonstrates practical exploration, not automatically OCR accuracy or learning outcomes. These evidence layers should remain distinct.
Layered evaluation
Author extension: Audit source, license, alignment, and split leakage; then page/layout/text/metadata; then retrieval; then claim-to-page grounding. Analyze by writing mode, language, century, scan quality, and layout, with targeted names, places, dates, and numbers.
A ten-page prototype
Author extension: The following is an implementation or review method derived from the lecture, not a result reported by the deck.
Define a small HDML subset and transcription guideline for ten varied pages. Have two reviewers align elements to coordinates. Evaluate model output for syntax, completeness, reading order, and text.
Index the result and write five page-grounded queries. Keep normalization separate and trace three failures through OCR, retrieval, and generation.
Model scores are not the endpoint
The CHURRO VLM fine-tunes an existing vision-language model. The lecture compares commercial VLMs, OCR systems, and open models across print, handwriting, and languages. Dataset composition remains critical: overall averages conceal sparse-language and script failures, so per-language and document-type results matter. (lecture source)
HistoryGenie connects page extraction to search and conversation. Provenance must remain visible: answers should link back to page images and extracted text so historians can identify OCR errors rather than treating a model transcript as the primary source.
A concrete exercise
Author extension: The following is an implementation or review method derived from the lecture, not a result reported by the deck.
Select ten pages from one collection and define representation rules for reading order, marginalia, and missing characters. Preserve image coordinates, HDML/text output, and corrections. Evaluate layout, text, and metadata errors separately instead of hiding them in one score.
Material gaps
Author extension: The following is an implementation or review method derived from the lecture, not a result reported by the deck.
The course site provides slides but no recording or complete reproducibility bundle. The deck summarizes the EMNLP 2025 work; this article does not infer undisclosed training details or costs from its charts.
References
Loading...