Towards speech-to-text translation without speech recognition

02/13/2017
by   Sameer Bansal, et al.
0

We explore the problem of translating speech to text in low-resource scenarios where neither automatic speech recognition (ASR) nor machine translation (MT) are available, but we have training data in the form of audio paired with text translations. We present the first system for this problem applied to a realistic multi-speaker dataset, the CALLHOME Spanish-English speech translation corpus. Our approach uses unsupervised term discovery (UTD) to cluster repeated patterns in the audio, creating a pseudotext, which we pair with translations to create a parallel text and train a simple bag-of-words MT model. We identify the challenges faced by the system, finding that the difficulty of cross-speaker UTD results in low recall, but that our system is still able to correctly translate some content words in test data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/13/2021

The IWSLT 2021 BUT Speech Translation Systems

The paper describes BUT's English to German offline speech translation(S...
research
05/21/2023

VAKTA-SETU: A Speech-to-Speech Machine Translation Service in Select Indic Languages

In this work, we present our deployment-ready Speech-to-Speech Machine T...
research
10/26/2022

Improving Speech-to-Speech Translation Through Unlabeled Text

Direct speech-to-speech translation (S2ST) is among the most challenging...
research
02/24/2019

The ARIEL-CMU Systems for LoReHLT18

This paper describes the ARIEL-CMU submissions to the Low Resource Human...
research
11/07/2018

Towards Fluent Translations from Disfluent Speech

When translating from speech, special consideration for conversational s...
research
12/02/2019

Language Model Bootstrapping Using Neural Machine Translation For Conversational Speech Recognition

Building conversational speech recognition systems for new languages is ...
research
09/02/2018

MTNT: A Testbed for Machine Translation of Noisy Text

Noisy or non-standard input text can cause disastrous mistranslations in...

Please sign up or login with your details

Forgot password? Click here to reset