Comparing supervised and self-supervised embedding for ExVo Multi-Task learning track

06/23/2022

∙

The ICML Expressive Vocalizations (ExVo) Multi-task challenge 2022, focuses on understanding the emotional facets of the non-linguistic vocalizations (vocal bursts (VB)). The objective of this challenge is to predict emotional intensities for VB, being a multi-task challenge it also requires to predict speakers' age and native-country. For this challenge we study and compare two distinct embedding spaces namely, self-supervised learning (SSL) based embeddings and task-specific supervised learning based embeddings. Towards that, we investigate feature representations obtained from several pre-trained SSL neural networks and task-specific supervised classification neural networks. Our studies show that the best performance is obtained with a hybrid approach, where predictions derived via both SSL and task-specific supervised learning are used. Our best system on test-set surpasses the ComPARE baseline (harmonic mean of all sub-task scores i.e., S_MTL) by a relative 13% margin.

READ FULL TEXT

Comparing supervised and self-supervised embedding for ExVo Multi-Task learning track

End-to-End and Self-Supervised Learning for ComParE 2022 Stuttering Sub-Challenge

Improving out-of-distribution generalization via multi-task self-supervised pretraining

Burst2Vec: An Adversarial Multi-Task Approach for Predicting Emotion, Age, and Origin from Vocal Bursts

UmBERTo-MTSA @ AcCompl-It: Improving Complexity and Acceptability Prediction with Multi-task Learning on Self-Supervised Annotations

Continual Robot Learning using Self-Supervised Task Inference

Aalto's End-to-End DNN systems for the INTERSPEECH 2020 Computational Paralinguistics Challenge

Self-supervision and Learnable STRFs for Age, Emotion, and Country Prediction

Comparing supervised and self-supervised embedding for ExVo Multi-Task learning track

Related Research

End-to-End and Self-Supervised Learning for ComParE 2022 Stuttering Sub-Challenge

Improving out-of-distribution generalization via multi-task self-supervised pretraining

Burst2Vec: An Adversarial Multi-Task Approach for Predicting Emotion, Age, and Origin from Vocal Bursts

UmBERTo-MTSA @ AcCompl-It: Improving Complexity and Acceptability Prediction with Multi-task Learning on Self-Supervised Annotations

Continual Robot Learning using Self-Supervised Task Inference

Aalto's End-to-End DNN systems for the INTERSPEECH 2020 Computational Paralinguistics Challenge

Self-supervision and Learnable STRFs for Age, Emotion, and Country Prediction