AVATAR submission to the Ego4D AV Transcription Challenge

11/18/2022

∙

In this report, we describe our submission to the Ego4D AudioVisual (AV) Speech Transcription Challenge 2022. Our pipeline is based on AVATAR, a state of the art encoder-decoder model for AV-ASR that performs early fusion of spectrograms and RGB images. We describe the datasets, experimental settings and ablations. Our final method achieves a WER of 68.40 on the challenge test set, outperforming the baseline by 43.7

READ FULL TEXT

AVATAR submission to the Ego4D AV Transcription Challenge

Channel-Aware Pretraining of Joint Encoder-Decoder Self-Supervised Model for Telephonic-Speech ASR

Building state-of-the-art distant speech recognition using the CHiME-4 challenge with a setup of speech enhancement baseline

RGB-D Salient Object Detection via 3D Convolutional Neural Networks

Adaptive Feature Selection for End-to-End Speech Translation

UniCon+: ICTCAS-UCAS Submission to the AVA-ActiveSpeaker Task at ActivityNet Challenge 2022

Bootstrapping Techniques for Polysynthetic Morphological Analysis

Multi-modal Feature Fusion with Feature Attention for VATEX Captioning Challenge 2020

AVATAR submission to the Ego4D AV Transcription Challenge

Related Research

Channel-Aware Pretraining of Joint Encoder-Decoder Self-Supervised Model for Telephonic-Speech ASR

Building state-of-the-art distant speech recognition using the CHiME-4 challenge with a setup of speech enhancement baseline

RGB-D Salient Object Detection via 3D Convolutional Neural Networks

Adaptive Feature Selection for End-to-End Speech Translation

UniCon+: ICTCAS-UCAS Submission to the AVA-ActiveSpeaker Task at ActivityNet Challenge 2022

Bootstrapping Techniques for Polysynthetic Morphological Analysis

Multi-modal Feature Fusion with Feature Attention for VATEX Captioning Challenge 2020