DeepAI AI Chat
Log In Sign Up

Efficient OCR for Building a Diverse Digital History

by   Jacob Carlson, et al.
Harvard University

Thousands of users consult digital archives daily, but the information they can access is unrepresentative of the diversity of documentary history. The sequence-to-sequence architecture typically used for optical character recognition (OCR) - which jointly learns a vision and language model - is poorly extensible to low-resource document collections, as learning a language-vision model requires extensive labeled sequences and compute. This study models OCR as a character level image retrieval problem, using a contrastively trained vision encoder. Because the model only learns characters' visual features, it is more sample efficient and extensible than existing architectures, enabling accurate OCR in settings where existing solutions fail. Crucially, the model opens new avenues for community engagement in making digital history more representative of documentary history.


page 11

page 12


Listen, Attend and Spell

We present Listen, Attend and Spell (LAS), a neural network that learns ...

From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network

In this paper, we abandon the dominant complex language model and rethin...

Learning Character-level Compositionality with Visual Features

Previous work has modeled the compositionality of words by creating char...

Character-Centric Storytelling

Sequential vision-to-language or visual storytelling has recently been o...

There Is a Digital Art History

In this paper, we revisit Johanna Drucker's question, "Is there a digita...

Quantifying Character Similarity with Vision Transformers

Record linkage is a bedrock of quantitative social science, as analyses ...

Musketeer (All for One, and One for All): A Generalist Vision-Language Model with Task Explanation Prompts

We present a sequence-to-sequence vision-language model whose parameters...