DeepAI AI Chat
Log In Sign Up

Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation

07/04/2017
by   Jean-Benoit Delbrouck, et al.
University of Mons
0

In Multimodal Neural Machine Translation (MNMT), a neural model generates a translated sentence that describes an image, given the image itself and one source descriptions in English. This is considered as the multimodal image caption translation task. The images are processed with Convolutional Neural Network (CNN) to extract visual features exploitable by the translation model. So far, the CNNs used are pre-trained on object detection and localization task. We hypothesize that richer architecture, such as dense captioning models, may be more suitable for MNMT and could lead to improved translations. We extend this intuition to the word-embeddings, where we compute both linguistic and visual representation for our corpus vocabulary. We combine and compare different confi

READ FULL TEXT

page 3

page 4

10/15/2018

UMONS Submission for WMT18 Multimodal Translation Task

This paper describes the UMONS solution for the Multimodal Machine Trans...
07/14/2017

LIUM-CVC Submissions for WMT17 Multimodal Translation Task

This paper describes the monomodal and multimodal Neural Machine Transla...
02/15/2019

Deeply Supervised Multimodal Attentional Translation Embeddings for Visual Relationship Detection

Detecting visual relationships, i.e. <Subject, Predicate, Object> triple...
01/10/2019

Equalizing Gender Biases in Neural Machine Translation with Word Embeddings Techniques

Neural machine translation has significantly pushed forward the quality ...
08/31/2018

The MeMAD Submission to the WMT18 Multimodal Translation Task

This paper describes the MeMAD project entry to the WMT Multimodal Machi...
01/15/2016

Multimodal Pivots for Image Caption Translation

We present an approach to improve statistical machine translation of ima...
09/15/2020

Simultaneous Machine Translation with Visual Context

Simultaneous machine translation (SiMT) aims to translate a continuous i...