Log In Sign Up

Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer

by   Wenqi Zhao, et al.

Encoder-decoder models have made great progress on handwritten mathematical expression recognition recently. However, it is still a challenge for existing methods to assign attention to image features accurately. Moreover, those encoder-decoder models usually adopt RNN-based models in their decoder part, which makes them inefficient in processing long sequences. In this paper, a transformer-based decoder is employed to replace RNN-based ones, which makes the whole model architecture very concise. Furthermore, a novel training strategy is introduced to fully exploit the potential of the transformer in bidirectional language modeling. Compared to several methods that do not use data augmentation, experiments demonstrate that our model improves the ExpRate of current state-of-the-art methods on CROHME 2014 by 2.23 CROHME 2016 and CROHME 2019, we improve the ExpRate by 1.92 respectively.


CoMER: Modeling Coverage for Transformer-based Handwritten Mathematical Expression Recognition

The Transformer-based encoder-decoder architecture has recently made sig...

A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition

In this study, we present a novel end-to-end approach based on the encod...

Offline Handwritten Mathematical Recognition using Adversarial Learning and Transformers

Offline Handwritten Mathematical Expression Recognition (HMER) is a majo...

Improving Attention-Based Handwritten Mathematical Expression Recognition with Scale Augmentation and Drop Attention

Handwritten mathematical expression recognition (HMER) is an important r...

Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition

Handwritten mathematical expression recognition is a challenging problem...

Handwritten Mathematical Expression Recognition via Attention Aggregation based Bi-directional Mutual Learning

Handwritten mathematical expression recognition aims to automatically ge...

Rethinking Text Line Recognition Models

In this paper, we study the problem of text line recognition. Unlike mos...