Towards End-to-End In-Image Neural Machine Translation

by   Elman Mansimov, et al.

In this paper, we offer a preliminary investigation into the task of in-image machine translation: transforming an image containing text in one language into an image containing the same text in another language. We propose an end-to-end neural model for this task inspired by recent approaches to neural machine translation, and demonstrate promising initial results based purely on pixel-level supervision. We then offer a quantitative and qualitative evaluation of our system outputs and discuss some common failure modes. Finally, we conclude with directions for future work.


page 1

page 2

page 3

page 4


A Multifaceted Evaluation of Neural versus Phrase-Based Machine Translation for 9 Language Directions

We aim to shed light on the strengths and weaknesses of the newly introd...

PharmMT: A Neural Machine Translation Approach to Simplify Prescription Directions

The language used by physicians and health professionals in prescription...

Exploring the Use of Attention within an Neural Machine Translation Decoder States to Translate Idioms

Idioms pose problems to almost all Machine Translation systems. This typ...

Generative Imagination Elevates Machine Translation

There are thousands of languages on earth, but visual perception is shar...

A Benchmark Dataset for Understandable Medical Language Translation

In this paper, we introduce MedLane – a new human-annotated Medical Lang...

A Survey of Multilingual Neural Machine Translation

We present a survey on multilingual neural machine translation (MNMT), w...

Explaining and Generalizing Back-Translation through Wake-Sleep

Back-translation has become a commonly employed heuristic for semi-super...