DeepAI AI Chat
Log In Sign Up

Subjective Assessments of Legibility in Ancient Manuscript Images – The SALAMI Dataset

by   Simon Brenner, et al.
TU Wien

The research field concerned with the digital restoration of degraded written heritage lacks a quantitative metric for evaluating its results, which prevents the comparison of relevant methods on large datasets. Thus, we introduce a novel dataset of Subjective Assessments of Legibility in Ancient Manuscript Images (SALAMI) to serve as a ground truth for the development of quantitative evaluation metrics in the field of digital text restoration. This dataset consists of 250 images of 50 manuscript regions with corresponding spatial maps of mean legibility and uncertainty, which are based on a study conducted with 20 experts of philology and paleography. As this study is the first of its kind, the validity and reliability of its design and the results obtained are motivated statistically: we report a high intra- and inter-rater agreement and show that the bulk of variation in the scores is introduced by the images regions observed and not by controlled or uncontrolled properties of participants and test environments, thus concluding that the legibility scores measured are valid attributes of the underlying images.


page 3

page 4

page 5

page 7


Novel Evaluation Metrics for Seam Carving based Image Retargeting

Image retargeting effectively resizes images by preserving the recogniza...

A Locally Weighted Fixation Density-Based Metric for Assessing the Quality of Visual Saliency Predictions

With the increased focus on visual attention (VA) in the last decade, a ...

Approaching Peak Ground Truth

Machine learning models are typically evaluated by computing similarity ...

Modeling Topical Coherence in Discourse without Supervision

Coherence of text is an important attribute to be measured for both manu...

An Analysis of Variations in the Effectiveness of Query Performance Prediction

A query performance predictor estimates the retrieval effectiveness of a...

MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aids

Improving the user's hearing ability to understand speech in noisy envir...

Describing Subjective Experiment Consistency by p-Value P-P Plot

There are phenomena that cannot be measured without subjective testing. ...