Combining Visual and Textual Features for Semantic Segmentation of Historical Newspapers

02/14/2020
by   Raphaël Barman, et al.
14

The massive amounts of digitized historical documents acquired over the last decades naturally lend themselves to automatic processing and exploration. Research work seeking to automatically process facsimiles and extract information thereby are multiplying with, as a first essential step, document layout analysis. If the identification and categorization of segments of interest in document images have seen significant progress over the last years thanks to deep learning techniques, many challenges remain with, among others, the use of finer-grained segmentation typologies and the consideration of complex, heterogeneous documents such as historical newspapers. Besides, most approaches consider visual features only, ignoring textual signal. In this context, we introduce a multimodal approach for the semantic segmentation of historical newspapers that combines visual and textual features. Based on a series of experiments on diachronic Swiss and Luxembourgish newspapers, we investigate, among others, the predictive power of visual and textual features and their capacity to generalize across time and sources. Results show consistent improvement of multimodal models in comparison to a strong visual baseline, as well as better robustness to high material variance.

READ FULL TEXT

page 6

page 9

page 12

page 17

research
09/17/2021

Including Keyword Position in Image-based Models for Act Segmentation of Historical Registers

The segmentation of complex images into semantic regions has seen a grow...
research
12/12/2022

Page Layout Analysis of Text-heavy Historical Documents: a Comparison of Textual and Visual Approaches

Page layout analysis is a fundamental step in document processing which ...
research
04/25/2017

Automatic Compositor Attribution in the First Folio of Shakespeare

Compositor attribution, the clustering of pages in a historical printed ...
research
12/23/2021

Digital Editions as Distant Supervision for Layout Analysis of Printed Books

Archivists, textual scholars, and historians often produce digital editi...
research
03/18/2021

Learning Multimodal Affinities for Textual Editing in Images

Nowadays, as cameras are rapidly adopted in our daily routine, images of...
research
03/23/2016

CONDITOR1: Topic Maps and DITA labelling tool for textual documents with historical information

Conditor is a software tool which works with textual documents containin...
research
04/07/2019

Measuring Human Perception to Improve Handwritten Document Transcription

The subtleties of human perception, as measured by vision scientists thr...

Please sign up or login with your details

Forgot password? Click here to reset