Self-supervision and Learnable STRFs for Age, Emotion, and Country Prediction

06/25/2022
by   Roshan Sharma, et al.
0

This work presents a multitask approach to the simultaneous estimation of age, country of origin, and emotion given vocal burst audio for the 2022 ICML Expressive Vocalizations Challenge ExVo-MultiTask track. The method of choice utilized a combination of spectro-temporal modulation and self-supervised features, followed by an encoder-decoder network organized in a multitask paradigm. We evaluate the complementarity between the tasks posed by examining independent task-specific and joint models, and explore the relative strengths of different feature sets. We also introduce a simple score fusion mechanism to leverage the complementarity of different feature sets for this task. We find that robust data preprocessing in conjunction with score fusion over spectro-temporal receptive field and HuBERT models achieved our best ExVo-MultiTask test score of 0.412.

READ FULL TEXT
research
06/24/2022

Multitask vocal burst modeling with ResNets and pre-trained paralinguistic Conformers

This technical report presents the modeling approaches used in our submi...
research
10/19/2019

Label-efficient audio classification through multitask learning and self-supervision

While deep learning has been incredibly successful in modeling tasks wit...
research
06/24/2022

Burst2Vec: An Adversarial Multi-Task Approach for Predicting Emotion, Age, and Origin from Vocal Bursts

We present Burst2Vec, our multi-task learning approach to predict emotio...
research
06/23/2022

Comparing supervised and self-supervised embedding for ExVo Multi-Task learning track

The ICML Expressive Vocalizations (ExVo) Multi-task challenge 2022, focu...
research
12/02/2022

ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning

Pretraining has been shown to scale well with compute, data size and dat...
research
07/13/2020

Multitask Non-Autoregressive Model for Human Motion Prediction

Human motion prediction, which aims at predicting future human skeletons...

Please sign up or login with your details

Forgot password? Click here to reset