b'Juan Pino'

research

∙ 08/22/2023

SeamlessM4T-Massively Multilingual Multimodal Machine Translation

What does it take to create the Babel Fish, a tool that can help individ...

0 Seamless Communication, et al. ∙

research

∙ 07/17/2023

Multilingual Speech-to-Speech Translation into Multiple Target Languages

Speech-to-speech translation (S2ST) enables spoken communication between...

0 Hongyu Gong, et al. ∙

research

∙ 06/01/2023

Exploration on HuBERT with Multiple Resolutions

Hidden-unit BERT (HuBERT) is a widely-used self-supervised learning (SSL...

0 Jiatong Shi, et al. ∙

research

∙ 05/04/2023

Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks

Transducer and Attention based Encoder-Decoder (AED) are two widely used...

0 Yun Tang, et al. ∙

research

∙ 04/10/2023

Enhancing Speech-to-Speech Translation with Multiple TTS Targets

It has been known that direct speech-to-speech translation (S2ST) models...

0 Jiatong Shi, et al. ∙

research

∙ 04/10/2023

ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit

ESPnet-ST-v2 is a revamp of the open-source ESPnet-ST toolkit necessitat...

0 Brian Yan, et al. ∙

research

∙ 03/01/2023

MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translation

We introduce MuAViC, a multilingual audio-visual corpus for robust speec...

0 Mohamed Anwar, et al. ∙

research

∙ 01/27/2023

Pre-training for Speech Translation: CTC Meets Optimal Transport

The gap between speech and text modalities is a major challenge in speec...

0 Phuong-Hang Le, et al. ∙

research

∙ 12/15/2022

UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units

Direct speech-to-speech translation (S2ST), in which all components can ...

2 Hirofumi Inaguma, et al. ∙

research

∙ 11/11/2022

Speech-to-Speech Translation For A Real-world Unwritten Language

We study speech-to-speech translation (S2ST) that translates speech from...

0 Peng-Jen Chen, et al. ∙

research

∙ 11/08/2022

SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations

We present SpeechMatrix, a large-scale multilingual corpus of speech-to-...

0 Paul-Ambroise Duquenne, et al. ∙

research

∙ 10/18/2022

Simple and Effective Unsupervised Speech Translation

The amount of labeled data to train models for speech tasks is limited f...

0 Changhan Wang, et al. ∙

research

∙ 04/11/2022

Unified Speech-Text Pre-training for Speech Translation and Recognition

We describe a method to jointly pre-train speech and text in an encoder-...

1 Yun Tang, et al. ∙

research

∙ 04/06/2022

Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation

Direct speech-to-speech translation (S2ST) models suffer from data scarc...

0 Sravya Popuri, et al. ∙

research

∙ 12/15/2021

Textless Speech-to-Speech Translation on Real Data

We present a textless speech-to-speech translation (S2ST) system that ca...

0 Ann Lee, et al. ∙

research

∙ 11/17/2021

XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale

This paper presents XLS-R, a large-scale model for cross-lingual speech ...

0 Arun Babu, et al. ∙

research

∙ 10/15/2021

Direct simultaneous speech to speech translation

We present the first direct simultaneous speech-to-speech translation (S...

0 Xutai Ma, et al. ∙

research

∙ 10/15/2021

Incremental Speech Synthesis For Speech-To-Speech Translation

In a speech-to-speech translation (S2ST) pipeline, the text-to-speech (T...

0 Danni Liu, et al. ∙

research

∙ 09/14/2021

fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit

This paper presents fairseq S^2, a fairseq extension for speech synthesi...

0 Changhan Wang, et al. ∙

research

∙ 07/14/2021

FST: the FAIR Speech Translation System for the IWSLT21 Multilingual Shared Task

In this paper, we describe our end-to-end multilingual speech translatio...

7 Yun Tang, et al. ∙

research

∙ 07/12/2021

Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task

Pretraining and multitask learning are widely used to improve the speech...

12 Yun Tang, et al. ∙

research

∙ 07/12/2021

Direct speech-to-speech translation with discrete units

We present a direct speech-to-speech translation (S2ST) model that trans...

0 Ann Lee, et al. ∙

research

∙ 06/21/2021

Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modeling

Multi-head attention has each of the attention heads collect salient inf...

4 Hongyu Gong, et al. ∙

research

∙ 06/02/2021

Lightweight Adapter Tuning for Multilingual Speech Translation

Adapter modules were recently introduced as an efficient alternative to ...

0 Hang Le, et al. ∙

research

∙ 04/14/2021

Large-Scale Self- and Semi-Supervised Learning for Speech Translation

In this paper, we improve speech translation (ST) through effectively le...

0 Changhan Wang, et al. ∙

research

∙ 01/02/2021

VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation

We introduce VoxPopuli, a large-scale multilingual corpus providing 100K...

0 Changhan Wang, et al. ∙

research

∙ 11/03/2020

SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation

Simultaneous text translation and end-to-end speech translation have rec...

0 Xutai Ma, et al. ∙

research

∙ 11/02/2020

Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation

We introduce dual-decoder Transformer, a new model architecture that joi...

0 Hang Le, et al. ∙

research

∙ 10/30/2020

Streaming Simultaneous Speech Translation with Augmented Memory Transformer

Transformer-based models have achieved state-of-the-art performance on s...

0 Xutai Ma, et al. ∙

research

∙ 10/24/2020

Cross-Modal Transfer Learning for Multilingual Speech-to-Text Translation

We propose an effective approach to utilize pretrained speech and text m...

8 Chau Tran, et al. ∙

research

∙ 10/21/2020

A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks

Attention-based sequence-to-sequence modeling provides a powerful and el...

0 Yun Tang, et al. ∙

research

∙ 10/11/2020

fairseq S2T: Fast Speech-to-Text Modeling with fairseq

We introduce fairseq S2T, a fairseq extension for speech-to-text (S2T) m...

0 Changhan Wang, et al. ∙

research

∙ 07/31/2020

SimulEval: An Evaluation Toolkit for Simultaneous Translation

Simultaneous translation on both text and speech focuses on a real-time ...

0 Xutai Ma, et al. ∙

research

∙ 07/20/2020

CoVoST 2 and Massively Multilingual Speech-to-Text Translation

Speech translation has recently become an increasingly popular topic of ...

0 Changhan Wang, et al. ∙

research

∙ 06/22/2020

Self-Supervised Representations Improve End-to-End Speech Translation

End-to-end speech-to-text translation can provide a simpler and smaller ...

0 Anne Wu, et al. ∙

research

∙ 06/09/2020

Improving Cross-Lingual Transfer Learning for End-to-End Speech Recognition with Speech Translation

Transfer learning from high-resource languages is known to be an efficie...

0 Changhan Wang, et al. ∙

research

∙ 06/03/2020

Self-Training for End-to-End Speech Translation

One of the main challenges for end-to-end speech translation is data sca...

0 Juan Pino, et al. ∙

research

∙ 02/27/2020

SkinAugment: Auto-Encoding Speaker Conversions for Automatic Speech Translation

We propose autoencoding speaker conversion for training data augmentatio...

0 Arya D. McCarthy, et al. ∙

research

∙ 02/04/2020

CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus

Spoken language translation has recently witnessed a resurgence in popul...

0 Changhan Wang, et al. ∙

research

∙ 09/26/2019

Monotonic Multihead Attention

Simultaneous machine translation models start generating a target sequen...

0 Xutai Ma, et al. ∙

research

∙ 09/14/2019

Harnessing Indirect Training Data for End-to-End Automatic Speech Translation: Tricks of the Trade

For automatic speech translation (AST), end-to-end approaches are outper...

0 Juan Pino, et al. ∙

research

∙ 09/14/2019

Leveraging Out-of-Task Data for End-to-End Automatic Speech Translation

For automatic speech translation (AST), end-to-end approaches are outper...

0 Juan Pino, et al. ∙

research

∙ 06/27/2019

Findings of the First Shared Task on Machine Translation Robustness

We share the findings of the first shared task on improving robustness o...

0 Xian Li, et al. ∙

research

∙ 02/04/2019

Two New Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English

The vast majority of language pairs in the world are low-resource becaus...

0 Francisco Guzmán, et al. ∙

Juan Pino

Featured Co-authors