Abdelrahman Mohamed

research

∙ 09/19/2023

AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models

Audio-visual representation learning aims to develop systems with human-...

0 Yuan Tseng, et al. ∙

research

∙ 05/19/2023

Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Mode

In this paper, we show that representations capturing syllabic units eme...

0 Puyuan Peng, et al. ∙

research

∙ 05/18/2023

ML-SUPERB: Multilingual Speech Universal PERformance Benchmark

Speech processing Universal PERformance Benchmark (SUPERB) is a leaderbo...

0 Jiatong Shi, et al. ∙

research

∙ 12/02/2022

Continual Learning for On-Device Speech Recognition using Disentangled Conformers

Automatic speech recognition research focuses on training and evaluating...

0 Anuj Diwan, et al. ∙

research

∙ 11/10/2022

Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilities

End-to-end multilingual ASR has become more appealing because of several...

0 Andros Tjandra, et al. ∙

research

∙ 11/04/2022

Biased Self-supervised learning for ASR

Self-supervised learning via masked prediction pre-training (MPPT) has s...

0 Florian L. Kreyssig, et al. ∙

research

∙ 10/21/2022

Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speech

Recent self-supervised learning (SSL) models have proven to learn rich r...

0 Cheol Jun Cho, et al. ∙

research

∙ 10/16/2022

SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning

We present the SUPERB challenge at SLT 2022, which aims at learning self...

0 Tzu-hsun Feng, et al. ∙

research

∙ 06/22/2022

DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexicon

Finding word boundaries in continuous speech is challenging as there is ...

9 Robin Algayres, et al. ∙

research

∙ 06/07/2022

LegoNN: Building Modular Encoder-Decoder Models

State-of-the-art encoder-decoder models (e.g. for machine translation (M...

0 Siddharth Dalmia, et al. ∙

research

∙ 05/21/2022

Self-Supervised Speech Representation Learning: A Review

Although supervised deep learning has revolutionized speech and audio pr...

0 Abdelrahman Mohamed, et al. ∙

research

∙ 05/15/2022

Learning Lip-Based Audio-Visual Speaker Embeddings with AV-HuBERT

This paper investigates self-supervised pre-training for audio-visual sp...

0 Bowen Shi, et al. ∙

research

∙ 04/11/2022

Unified Speech-Text Pre-training for Speech Translation and Recognition

We describe a method to jointly pre-train speech and text in an encoder-...

1 Yun Tang, et al. ∙

research

∙ 04/08/2022

Federated Learning with Partial Model Personalization

We consider two federated learning algorithms for training partially per...

0 Krishna Pillutla, et al. ∙

research

∙ 03/30/2022

Generative Spoken Dialogue Language Modeling

We introduce dGSLM, the first "textless" model able to generate audio sa...

5 Tu Anh Nguyen, et al. ∙

research

∙ 03/14/2022

SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities

Transfer learning has proven to be crucial in advancing the state of spe...

0 Hsiang-Sheng Tsai, et al. ∙

research

∙ 03/09/2022

DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answering

Spoken Question Answering (SQA) is to find the answer from a spoken docu...

0 Guan-Ting Lin, et al. ∙

research

∙ 02/15/2022

textless-lib: a Library for Textless Spoken Language Processing

Textless spoken language processing research aims to extend the applicab...

11 Eugene Kharitonov, et al. ∙

research

∙ 01/21/2022

Object Detection in Aerial Images: What Improves the Accuracy?

Object detection is a challenging and popular computer vision problem. T...

0 Hashmat Shadab Malik, et al. ∙

research

∙ 01/05/2022

Robust Self-Supervised Audio-Visual Speech Recognition

Audio-based automatic speech recognition (ASR) degrades significantly in...

0 Bowen Shi, et al. ∙

research

∙ 01/05/2022

Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction

Video recordings of speech contain correlated audio and visual informati...

0 Bowen Shi, et al. ∙

research

∙ 11/14/2021

Textless Speech Emotion Conversion using Decomposed and Discrete Representations

Speech emotion conversion is the task of modifying the perceived emotion...

6 Felix Kreuk, et al. ∙

research

∙ 11/10/2021

Scaling ASR Improves Zero and Few Shot Learning

With 4.5 million hours of English speech from 10 different sources acros...

0 Alex Xiao, et al. ∙

research

∙ 09/07/2021

Text-Free Prosody-Aware Generative Spoken Language Modeling

Speech pre-training has primarily demonstrated efficacy on classificatio...

14 Eugene Kharitonov, et al. ∙

research

∙ 06/14/2021

Kaizen: Continuously improving teacher using Exponential Moving Average for semi-supervised speech recognition

In this paper, we introduce the Kaizen framework that uses a continuousl...

0 Vimal Manohar, et al. ∙

research

∙ 06/14/2021

HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

Self-supervised approaches for speech representation learning are challe...

0 Wei-Ning Hsu, et al. ∙

research

∙ 05/03/2021

SUPERB: Speech processing Universal PERformance Benchmark

Self-supervised learning (SSL) has proven vital for advancing research i...

0 Shu-wen Yang, et al. ∙

research

∙ 04/01/2021

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations

We propose using self-supervised discrete representations for the task o...

10 Adam Polyak, et al. ∙

research

∙ 03/09/2021

Contrastive Semi-supervised Learning for ASR

Pseudo-labeling is the most adopted method for pre-training automatic sp...

0 Alex Xiao, et al. ∙

research

∙ 06/24/2020

Unsupervised Cross-lingual Representation Learning for Speech Recognition

This paper presents XLSR which learns cross-lingual speech representatio...

0 Alexis Conneau, et al. ∙

research

∙ 06/20/2020

wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

We show for the first time that learning powerful representations from s...

0 Alexei Baevski, et al. ∙

research

∙ 05/16/2020

Large scale weakly and semi-supervised learning for low-resource video ASR

Many semi- and weakly-supervised approaches have been investigated for o...

0 Kritika Singh, et al. ∙

research

∙ 12/17/2019

Libri-Light: A Benchmark for ASR with Limited or No Supervision

We introduce a new collection of spoken English audio suitable for train...

0 Jacob Kahn, et al. ∙

research

∙ 11/10/2019

Effectiveness of self-supervised pre-training for speech recognition

We present pre-training approaches for self-supervised representation le...

0 Alexei Baevski, et al. ∙

research

∙ 11/09/2019

Enforcing Encoder-Decoder Modularity in Sequence-to-Sequence Models

Inspired by modular software design principles of independence, intercha...

0 Siddharth Dalmia, et al. ∙

research

∙ 10/29/2019

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

We present BART, a denoising autoencoder for pretraining sequence-to-seq...

35 Mike Lewis, et al. ∙

research

∙ 10/27/2019

Training ASR models by Generation of Contextual Information

Supervised ASR models have reached unprecedented levels of accuracy, tha...

0 Kritika Singh, et al. ∙

research

∙ 10/22/2019

Transformer-based Acoustic Modeling for Hybrid Speech Recognition

We propose and evaluate transformer-based acoustic models (AMs) for hybr...

0 Yongqiang Wang, et al. ∙

research

∙ 04/26/2019

Transformers with convolutional context for ASR

The recent success of transformer networks for neural machine translatio...

0 Abdelrahman Mohamed, et al. ∙

research

∙ 02/24/2017

Sequence Modeling via Segmentations

Segmental structure is a common pattern in many types of sequences such ...

0 Chong Wang, et al. ∙

research

∙ 03/17/2016

Do Deep Convolutional Nets Really Need to be Deep and Convolutional?

Yes, they do. This paper provides the first empirical demonstration that...

0 Gregor Urban, et al. ∙

Abdelrahman Mohamed

Featured Co-authors