Does Multimodality Help Human and Machine for Translation and Image Captioning?

05/30/2016
by   Ozan Caglayan, et al.
0

This paper presents the systems developed by LIUM and CVC for the WMT16 Multimodal Machine Translation challenge. We explored various comparative methods, namely phrase-based systems and attentional recurrent neural networks models trained using monomodal or multimodal data. We also performed a human evaluation in order to estimate the usefulness of multimodal data for human machine translation and image description generation. Our systems obtained the best results for both tasks according to the automatic evaluation metrics BLEU and METEOR.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/14/2017

LIUM-CVC Submissions for WMT17 Multimodal Translation Task

This paper describes the monomodal and multimodal Neural Machine Transla...
research
06/23/2016

CUNI System for WMT16 Automatic Post-Editing and Multimodal Translation Tasks

Neural sequence to sequence learning recently became a very promising pa...
research
01/15/2016

Multimodal Pivots for Image Caption Translation

We present an approach to improve statistical machine translation of ima...
research
11/28/2019

Multimodal Machine Translation through Visuals and Speech

Multimodal machine translation involves drawing information from more th...
research
10/26/2020

Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale

Automatic evaluation of language generation systems is a well-studied pr...
research
02/15/2018

Teaching Machines to Code: Neural Markup Generation with Visual Attention

We present a deep recurrent neural network model with soft visual attent...
research
06/07/2021

BERTGEN: Multi-task Generation through BERT

We present BERTGEN, a novel generative, decoder-only model which extends...

Please sign up or login with your details

Forgot password? Click here to reset