research
∙
12/11/2022
Error-aware Quantization through Noise Tempering
Quantization has become a predominant approach for model compression, en...
share
research
∙
10/27/2022
Token-level Sequence Labeling for Spoken Language Understanding using Compositional End-to-End Models
End-to-end spoken language understanding (SLU) systems are gaining popul...
research
∙
10/13/2022
SQuAT: Sharpness- and Quantization-Aware Training for BERT
Quantization is an effective technique to reduce memory footprint, infer...
research
∙
10/11/2022
CTC Alignments Improve Autoregressive Translation
Connectionist Temporal Classification (CTC) is a widely used approach fo...
research
∙
09/06/2022
ASR2K: Speech Recognition for Around 2000 Languages without Audio
Most recent speech recognition models rely on large supervised datasets,...
research
∙
07/13/2022
Masked Autoencoders that Listen
This paper studies a simple extension of image-based Masked Autoencoders...
research
∙
06/07/2022
LegoNN: Building Modular Encoder-Decoder Models
State-of-the-art encoder-decoder models (e.g. for machine translation (M...
research
∙
05/24/2022
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization
Integrating vision and language has gained notable attention following t...
research
∙
05/06/2022
Robustness of Neural Architectures for Audio Event Detection
Traditionally, in Audio Recognition pipeline, noise is suppressed by the...
research
∙
03/25/2022
AudioTagging Done Right: 2nd comparison of deep learning methods for environmental sound classification
After its sweeping success in vision and language tasks, pure attention-...
research
∙
03/23/2022
On Adversarial Robustness of Large-scale Audio Visual Learning
As audio-visual systems are being deployed for safety-critical tasks suc...
research
∙
10/12/2021
Speech Summarization using Restricted Self-Attention
Speech summarization is typically performed by using a cascade of speech...
research
∙
09/28/2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding
We present VideoCLIP, a contrastive approach to pre-train a unified mode...
research
∙
07/24/2021
Differentiable Allophone Graphs for Language-Universal Speech Recognition
Building language-universal speech recognition systems entails producing...
research
∙
06/29/2021
Rethinking End-to-End Evaluation of Decomposable Tasks: A Case Study on Spoken Language Understanding
Decomposable tasks are complex and comprise of a hierarchy of sub-tasks....
research
∙
05/20/2021
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding
We present a simplified, task-agnostic multi-modal pre-training approach...
research
∙
05/02/2021
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks
End-to-end approaches for sequence tasks are becoming increasingly popul...
research
∙
04/13/2021
Self-supervised object detection from audio-visual correspondence
We tackle the problem of learning object detectors without supervision. ...
research
∙
03/18/2021
Space-Time Crop Attend: Improving Cross-modal Video Representation Learning
The quality of the image representations obtained from self-supervised l...
research
∙
03/16/2021
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models
This paper studies zero-shot cross-lingual transfer of vision-language m...
research
∙
11/15/2020
Audio-Visual Event Recognition through the lens of Adversary
As audio/visual classification models are widely deployed for sensitive ...
research
∙
10/16/2020
Multimodal Speech Recognition with Unstructured Audio Masking
Visual context has been shown to be useful for automatic speech recognit...
research
∙
10/06/2020
Support-set bottlenecks for video-text representation learning
The dominant paradigm for learning video-text representations – noise co...
research
∙
10/05/2020
Fine-Grained Grounding for Multimodal Speech Recognition
Multimodal automatic speech recognition systems integrate information fr...
research
∙
09/12/2020
Revisiting Factorizing Aggregated Posterior in Learning Disentangled Representations
In the problem of learning disentangled representations, one of the prom...
research
∙
08/18/2020
How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language
Sign Language is the primary means of communication for the majority of ...
research
∙
04/17/2020
AlloVera: A Multilingual Allophone Database
We introduce a new resource, AlloVera, which provides mappings from 218 ...
research
∙
03/13/2020
ASR Error Correction and Domain Adaptation Using Machine Translation
Off-the-shelf pre-trained Automatic Speech Recognition (ASR) systems are...
research
∙
02/26/2020
Universal Phone Recognition with a Multilingual Allophone System
Multilingual models can improve language processing, particularly for lo...
research
∙
02/26/2020
Towards Zero-shot Learning for Automatic Phonemic Transcription
Automatic phonemic transcription tools are useful for low-resource langu...
research
∙
02/13/2020
Looking Enhances Listening: Recovering Missing Speech Using Images
Speech is understood better by using visual context; for this reason, th...
research
∙
01/29/2020
Gun Source and Muzzle Head Detection
There is a surging need across the world for protection against gun viol...
research
∙
11/09/2019
Enforcing Encoder-Decoder Modularity in Sequence-to-Sequence Models
Inspired by modular software design principles of independence, intercha...
research
∙
11/04/2019
On Compositionality in Neural Machine Translation
We investigate two specific manifestations of compositionality in Neural...
research
∙
10/31/2019
Adversarial Music: Real World Audio Adversary Against Wake-word Detection System
Voice Assistants (VAs) such as Amazon Alexa or Google Assistant rely on ...
research
∙
10/27/2019
Multitask Learning For Different Subword Segmentations In Neural Machine Translation
In Neural Machine Translation (NMT) the usage of subwords and characters...
research
∙
10/07/2019
On Leveraging the Visual Modality for Neural Machine Translation
Leveraging the visual modality effectively for Neural Machine Translatio...
research
∙
10/05/2019
On Dimensional Linguistic Properties of the Word Embedding Space
Word embeddings have become a staple of several natural language process...
research
∙
08/02/2019
SANTLR: Speech Annotation Toolkit for Low Resource Languages
While low resource speech recognition has attracted a lot of attention f...
research
∙
08/02/2019
Multilingual Speech Recognition with Corpus Relatedness Sampling
Multilingual acoustic models have been successfully applied to low-resou...
research
∙
07/24/2019
Cross-Attention End-to-End ASR for Two-Party Conversations
We present an end-to-end speech recognition model that learns interactio...
research
∙
06/30/2019
Analyzing Utility of Visual Context in Multimodal Speech Recognition Under Noisy Conditions
Multimodal learning allows us to leverage information from multiple sour...
research
∙
06/27/2019
Gated Embeddings in End-to-End Speech Recognition for Conversational-Context Fusion
We present a novel conversational-context aware end-to-end speech recogn...
research
∙
06/19/2019
Multimodal Abstractive Summarization for How2 Videos
In this paper, we study abstractive summarization for open-domain videos...
research
∙
06/13/2019
Grounding Object Detections With Transcriptions
A vast amount of audio-visual data is available on the Internet thanks t...
research
∙
05/21/2019
Acoustic-to-Word Models with Conversational Context Information
Conversational context information, higher-level knowledge that spans ac...
research
∙
02/24/2019
The ARIEL-CMU Systems for LoReHLT18
This paper describes the ARIEL-CMU submissions to the Low Resource Human...
research
∙
02/20/2019
Phoneme Level Language Models for Sequence Based Low Resource ASR
Building multilingual and crosslingual models help bring different langu...
research
∙
02/18/2019
Learned In Speech Recognition: Contextual Acoustic Word Embeddings
End-to-end acoustic-to-word speech recognition models have recently gain...
research
∙
11/21/2018