DeepAI AI Chat
Log In Sign Up

Machine Translation Verbosity Control for Automatic Dubbing

10/08/2021
by   Surafel M. Lakew, et al.
Amazon
0

Automatic dubbing aims at seamlessly replacing the speech in a video document with synthetic speech in a different language. The task implies many challenges, one of which is generating translations that not only convey the original content, but also match the duration of the corresponding utterances. In this paper, we focus on the problem of controlling the verbosity of machine translation output, so that subsequent steps of our automatic dubbing pipeline can generate dubs of better quality. We propose new methods to control the verbosity of MT output and compare them against the state of the art with both intrinsic and extrinsic evaluations. For our experiments we use a public data set to dub English speeches into French, Italian, German and Spanish. Finally, we report extensive subjective tests that measure the impact of MT verbosity control on the final quality of dubbed video clips.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/16/2021

Isometric MT: Neural Machine Translation for Automatic Dubbing

Automatic dubbing (AD) is among the use cases where translations should ...
01/19/2020

From Speech-to-Speech Translation to Automatic Dubbing

We present enhancements to a speech-to-speech translation pipeline in or...
12/16/2021

Prosody-Aware Neural Machine Translation for Dubbing

We introduce the task of prosody-aware machine translation which aims at...
10/24/2021

Understanding the Impact of UGC Specificities on Translation Quality

This work takes a critical look at the evaluation of user-generated cont...
02/25/2023

Jointly Optimizing Translations and Speech Timing to Improve Isochrony in Automatic Dubbing

Automatic dubbing (AD) is the task of translating the original speech in...
02/16/2018

Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT

Although measuring intrinsic quality has been a key factor in the advanc...
09/19/2022

A Snapshot into the Possibility of Video Game Machine Translation

We present in this article what we believe to be one of the first attemp...