Log In Sign Up

ConvMath: A Convolutional Sequence Network for Mathematical Expression Recognition

by   Zuoyu Yan, et al.

Despite the recent advances in optical character recognition (OCR), mathematical expressions still face a great challenge to recognize due to their two-dimensional graphical layout. In this paper, we propose a convolutional sequence modeling network, ConvMath, which converts the mathematical expression description in an image into a LaTeX sequence in an end-to-end way. The network combines an image encoder for feature extraction and a convolutional decoder for sequence generation. Compared with other Long Short Term Memory(LSTM) based encoder-decoder models, ConvMath is entirely based on convolution, thus it is easy to perform parallel computation. Besides, the network adopts multi-layer attention mechanism in the decoder, which allows the model to align output symbols with source feature vectors automatically, and alleviates the problem of lacking coverage while training the model. The performance of ConvMath is evaluated on an open dataset named IM2LATEX-100K, including 103556 samples. The experimental results demonstrate that the proposed network achieves state-of-the-art accuracy and much better efficiency than previous methods.


A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition

In this study, we present a novel end-to-end approach based on the encod...

A sequential guiding network with attention for image captioning

The recent advances of deep learning in both computer vision (CV)and nat...

Translating Mathematical Formula Images to LaTeX Sequences Using Deep Neural Networks with Sequence-level Training

In this paper we propose a deep neural network model with an encoder-dec...

Expression Recognition in the Wild Using Sequence Modeling

As we exceed upon the procedures for modelling the different aspects of ...

AttentionHTR: Handwritten Text Recognition Based on Attention Encoder-Decoder Networks

This work proposes an attention-based sequence-to-sequence model for han...

Multi-layer Attention Mechanism for Speech Keyword Recognition

As an important part of speech recognition technology, automatic speech ...

EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition

Printed Mathematical expression recognition (PMER) aims to transcribe a ...