CSSL-MHTR: Continual Self-Supervised Learning for Scalable Multi-script Handwritten Text Recognition

03/16/2023
by   Marwa Dhiaf, et al.
0

Self-supervised learning has recently emerged as a strong alternative in document analysis. These approaches are now capable of learning high-quality image representations and overcoming the limitations of supervised methods, which require a large amount of labeled data. However, these methods are unable to capture new knowledge in an incremental fashion, where data is presented to the model sequentially, which is closer to the realistic scenario. In this paper, we explore the potential of continual self-supervised learning to alleviate the catastrophic forgetting problem in handwritten text recognition, as an example of sequence recognition. Our method consists in adding intermediate layers called adapters for each task, and efficiently distilling knowledge from the previous model while learning the current task. Our proposed framework is efficient in both computation and memory complexity. To demonstrate its effectiveness, we evaluate our method by transferring the learned model to diverse text recognition downstream tasks, including Latin and non-Latin scripts. As far as we know, this is the first application of continual self-supervised learning for handwritten text recognition. We attain state-of-the-art performance on English, Italian and Russian scripts, whilst adding only a few parameters per task. The code and trained models will be publicly available.

READ FULL TEXT

page 4

page 8

page 13

research
12/08/2021

Self-Supervised Models are Continual Learners

Self-supervised models have been shown to produce comparable or better v...
research
03/09/2022

Text-DIAE: Degradation Invariant Autoencoders for Text Recognition and Document Enhancement

In this work, we propose Text-Degradation Invariant Auto Encoder (Text-D...
research
05/15/2022

Learning Representations for New Sound Classes With Continual Self-Supervised Learning

In this paper, we present a self-supervised learning framework for conti...
research
06/10/2020

Self-Supervised Learning Aided Class-Incremental Lifelong Learning

Lifelong or continual learning remains to be a challenge for artificial ...
research
12/30/2021

Continually Learning Self-Supervised Representations with Projected Functional Regularization

Recent self-supervised learning methods are able to learn high-quality i...
research
10/27/2022

Efficient few-shot learning for pixel-precise handwritten document layout analysis

Layout analysis is a task of uttermost importance in ancient handwritten...
research
03/31/2020

Towards Lifelong Self-Supervision For Unpaired Image-to-Image Translation

Unpaired Image-to-Image Translation (I2IT) tasks often suffer from lack ...

Please sign up or login with your details

Forgot password? Click here to reset