I2C2W: Image-to-Character-to-Word Transformers for Accurate Scene Text Recognition

by   Chuhui Xue, et al.

Leveraging the advances of natural language processing, most recent scene text recognizers adopt an encoder-decoder architecture where text images are first converted to representative features and then a sequence of characters via `direct decoding'. However, scene text images suffer from rich noises of different sources such as complex background and geometric distortions which often confuse the decoder and lead to incorrect alignment of visual features at noisy decoding time steps. This paper presents I2C2W, a novel scene text recognizer that is accurate and tolerant to various noises in scenes. I2C2W consists of an image-to-character module (I2C) and a character-to-word module (C2W) which are complementary and can be trained end-to-end. I2C detects characters and predicts their relative positions in a word. It strives to detect all possible characters including incorrect and redundant ones based on different alignments of visual features without the restriction of time steps. Taking the detected characters as input, C2W learns from character semantics and their positions to filter out incorrect and redundant detection and produce the final word recognition. Extensive experiments over seven public datasets show that I2C2W achieves superior recognition performances and outperforms the state-of-the-art by large margins on challenging irregular scene text datasets.


page 3

page 6

page 7

page 8


Decoupled Attention Network for Text Recognition

Text recognition has attracted considerable research interests because o...

AE TextSpotter: Learning Visual and Linguistic Representation for Ambiguous Text Spotting

Scene text spotting aims to detect and recognize the entire word or sent...

SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition

Scene text recognition is a hot research topic in computer vision. Recen...

STEFANN: Scene Text Editor using Font Adaptive Neural Network

Textual information in a captured scene play important role in scene int...

Soft-PHOC Descriptor for End-to-End Word Spotting in Egocentric Scene Images

Word spotting in natural scene images has many applications in scene und...

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization

State-of-the-art models in natural language processing rely on separate ...

Contextual Text Block Detection towards Scene Text Understanding

Most existing scene text detectors focus on detecting characters or word...