b'Florian Metze'

research

∙ 07/13/2022

Masked Autoencoders that Listen

This paper studies a simple extension of image-based Masked Autoencoders...

Po-Yao Huang, et al. ∙

research

∙ 06/07/2022

LegoNN: Building Modular Encoder-Decoder Models

State-of-the-art encoder-decoder models (e.g. for machine translation (M...

Siddharth Dalmia, et al. ∙

research

∙ 05/24/2022

On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization

Integrating vision and language has gained notable attention following t...

Shruti Palaskar, et al. ∙

research

∙ 05/06/2022

Robustness of Neural Architectures for Audio Event Detection

Traditionally, in Audio Recognition pipeline, noise is suppressed by the...

Juncheng B. Li, et al. ∙

research

∙ 03/25/2022

AudioTagging Done Right: 2nd comparison of deep learning methods for environmental sound classification

After its sweeping success in vision and language tasks, pure attention-...

Juncheng B. Li, et al. ∙

research

∙ 03/23/2022

On Adversarial Robustness of Large-scale Audio Visual Learning

As audio-visual systems are being deployed for safety-critical tasks suc...

Juncheng B. Li, et al. ∙

research

∙ 10/12/2021

Speech Summarization using Restricted Self-Attention

Speech summarization is typically performed by using a cascade of speech...

Roshan Sharma, et al. ∙

research

∙ 09/28/2021

VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding

We present VideoCLIP, a contrastive approach to pre-train a unified mode...

Hu Xu, et al. ∙

research

∙ 07/24/2021

Differentiable Allophone Graphs for Language-Universal Speech Recognition

Building language-universal speech recognition systems entails producing...

Brian Yan, et al. ∙

research

∙ 06/29/2021

Rethinking End-to-End Evaluation of Decomposable Tasks: A Case Study on Spoken Language Understanding

Decomposable tasks are complex and comprise of a hierarchy of sub-tasks....

Siddhant Arora, et al. ∙

research

∙ 05/20/2021

VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding

We present a simplified, task-agnostic multi-modal pre-training approach...

Hu Xu, et al. ∙

research

∙ 05/02/2021

Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks

End-to-end approaches for sequence tasks are becoming increasingly popul...

Siddharth Dalmia, et al. ∙

research

∙ 04/13/2021

Self-supervised object detection from audio-visual correspondence

We tackle the problem of learning object detectors without supervision. ...

Triantafyllos Afouras, et al. ∙

research

∙ 03/18/2021

Space-Time Crop Attend: Improving Cross-modal Video Representation Learning

The quality of the image representations obtained from self-supervised l...

Mandela Patrick, et al. ∙

research

∙ 03/16/2021

Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models

This paper studies zero-shot cross-lingual transfer of vision-language m...

Po-Yao Huang, et al. ∙

research

∙ 11/15/2020

Audio-Visual Event Recognition through the lens of Adversary

As audio/visual classification models are widely deployed for sensitive ...

Juncheng B. Li, et al. ∙

research

∙ 10/16/2020

Multimodal Speech Recognition with Unstructured Audio Masking

Visual context has been shown to be useful for automatic speech recognit...

Tejas Srinivasan, et al. ∙

research

∙ 10/06/2020

Support-set bottlenecks for video-text representation learning

The dominant paradigm for learning video-text representations – noise co...

Mandela Patrick, et al. ∙

research

∙ 10/05/2020

Fine-Grained Grounding for Multimodal Speech Recognition

Multimodal automatic speech recognition systems integrate information fr...

Tejas Srinivasan, et al. ∙

research

∙ 09/12/2020

Revisiting Factorizing Aggregated Posterior in Learning Disentangled Representations

In the problem of learning disentangled representations, one of the prom...

Ze Cheng, et al. ∙

research

∙ 08/18/2020

How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language

Sign Language is the primary means of communication for the majority of ...

Amanda Duarte, et al. ∙

research

∙ 04/17/2020

AlloVera: A Multilingual Allophone Database

We introduce a new resource, AlloVera, which provides mappings from 218 ...

David R. Mortensen, et al. ∙

research

∙ 03/13/2020

ASR Error Correction and Domain Adaptation Using Machine Translation

Off-the-shelf pre-trained Automatic Speech Recognition (ASR) systems are...

Anirudh Mani, et al. ∙

research

∙ 02/26/2020

Universal Phone Recognition with a Multilingual Allophone System

Multilingual models can improve language processing, particularly for lo...

Xinjian Li, et al. ∙

research

∙ 02/26/2020

Towards Zero-shot Learning for Automatic Phonemic Transcription

Automatic phonemic transcription tools are useful for low-resource langu...

Xinjian Li, et al. ∙

research

∙ 02/13/2020

Looking Enhances Listening: Recovering Missing Speech Using Images

Speech is understood better by using visual context; for this reason, th...

Tejas Srinivasan, et al. ∙

research

∙ 01/29/2020

Gun Source and Muzzle Head Detection

There is a surging need across the world for protection against gun viol...

Zhong Zhou, et al. ∙

research

∙ 11/09/2019

Enforcing Encoder-Decoder Modularity in Sequence-to-Sequence Models

Inspired by modular software design principles of independence, intercha...

Siddharth Dalmia, et al. ∙

research

∙ 11/04/2019

On Compositionality in Neural Machine Translation

We investigate two specific manifestations of compositionality in Neural...

Vikas Raunak, et al. ∙

research

∙ 10/31/2019

Adversarial Music: Real World Audio Adversary Against Wake-word Detection System

Voice Assistants (VAs) such as Amazon Alexa or Google Assistant rely on ...

Juncheng B. Li, et al. ∙

research

∙ 10/27/2019

Multitask Learning For Different Subword Segmentations In Neural Machine Translation

In Neural Machine Translation (NMT) the usage of subwords and characters...

Tejas Srinivasan, et al. ∙

research

∙ 10/07/2019

On Leveraging the Visual Modality for Neural Machine Translation

Leveraging the visual modality effectively for Neural Machine Translatio...

Vikas Raunak, et al. ∙

research

∙ 10/05/2019

On Dimensional Linguistic Properties of the Word Embedding Space

Word embeddings have become a staple of several natural language process...

Vikas Raunak, et al. ∙

research

∙ 08/02/2019

SANTLR: Speech Annotation Toolkit for Low Resource Languages

While low resource speech recognition has attracted a lot of attention f...

Xinjian Li, et al. ∙

research

∙ 08/02/2019

Multilingual Speech Recognition with Corpus Relatedness Sampling

Multilingual acoustic models have been successfully applied to low-resou...

Xinjian Li, et al. ∙

research

∙ 07/24/2019

Cross-Attention End-to-End ASR for Two-Party Conversations

We present an end-to-end speech recognition model that learns interactio...

Suyoun Kim, et al. ∙

research

∙ 06/30/2019

Analyzing Utility of Visual Context in Multimodal Speech Recognition Under Noisy Conditions

Multimodal learning allows us to leverage information from multiple sour...

Tejas Srinivasan, et al. ∙

research

∙ 06/27/2019

Gated Embeddings in End-to-End Speech Recognition for Conversational-Context Fusion

We present a novel conversational-context aware end-to-end speech recogn...

Suyoun Kim, et al. ∙

research

∙ 06/19/2019

Multimodal Abstractive Summarization for How2 Videos

In this paper, we study abstractive summarization for open-domain videos...

Shruti Palaskar, et al. ∙

research

∙ 06/13/2019

Grounding Object Detections With Transcriptions

A vast amount of audio-visual data is available on the Internet thanks t...

Yasufumi Moriya, et al. ∙

research

∙ 05/21/2019

Acoustic-to-Word Models with Conversational Context Information

Conversational context information, higher-level knowledge that spans ac...

Suyoun Kim, et al. ∙

research

∙ 02/24/2019

The ARIEL-CMU Systems for LoReHLT18

This paper describes the ARIEL-CMU submissions to the Low Resource Human...

Aditi Chaudhary, et al. ∙

research

∙ 02/20/2019

Phoneme Level Language Models for Sequence Based Low Resource ASR

Building multilingual and crosslingual models help bring different langu...

Siddharth Dalmia, et al. ∙

research

∙ 02/18/2019

Learned In Speech Recognition: Contextual Acoustic Word Embeddings

End-to-end acoustic-to-word speech recognition models have recently gain...

Shruti Palaskar, et al. ∙

research

∙ 11/21/2018

Learning from Multiview Correlations in Open-Domain Videos

An increasing number of datasets contain multiple views, such as video, ...

Nils Holzenberger, et al. ∙

research

∙ 11/09/2018

Multimodal Grounding for Sequence-to-Sequence Speech Recognition

Humans are capable of processing speech by making use of multiple sensor...

Ozan Caglayan, et al. ∙

research

∙ 11/01/2018

How2: A Large-scale Dataset for Multimodal Language Understanding

In this paper, we introduce How2, a multimodal collection of instruction...

Ramon Sanabria, et al. ∙

research

∙ 10/22/2018

Connectionist Temporal Localization for Sound Event Detection with Sequential Labeling

Research on sound event detection (SED) with weak labeling has mostly fo...

Yun Wang, et al. ∙

research

∙ 10/22/2018

A comparison of five multiple instance learning pooling functions for sound event detection with weak labeling

Sound event detection (SED) entails two subtasks: recognizing what types...

Yun Wang, et al. ∙

research

∙ 09/01/2018

Activity Recognition on a Large Scale in Short Videos - Moments in Time Dataset

Moments capture a huge part of our lives. Accurate recognition of these ...

Ankit Shah, et al. ∙

Florian Metze

Featured Co-authors