An empirical study of weakly supervised audio tagging embeddings for general audio representations

09/30/2022
by   Heinrich Dinkel, et al.
0

We study the usability of pre-trained weakly supervised audio tagging (AT) models as feature extractors for general audio representations. We mainly analyze the feasibility of transferring those embeddings to other tasks within the speech and sound domains. Specifically, we benchmark weakly supervised pre-trained models (MobileNetV2 and EfficientNet-B0) against modern self-supervised learning methods (BYOL-A) as feature extractors. Fourteen downstream tasks are used for evaluation ranging from music instrument classification to language classification. Our results indicate that AT pre-trained models are an excellent transfer learning choice for music, event, and emotion recognition tasks. Further, finetuning AT models can also benefit speech-related tasks such as keyword spotting and intent classification.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/07/2023

Improving Self-Supervised Learning for Audio Representations by Feature Diversity and Decorrelation

Self-supervised learning (SSL) has recently shown remarkable results in ...
research
07/12/2021

Codified audio language modeling learns useful representations for music information retrieval

We demonstrate that language models pre-trained on codified (discretely-...
research
07/19/2023

From West to East: Who can understand the music of the others better?

Recent developments in MIR have led to several benchmark deep learning m...
research
04/15/2022

BYOL for Audio: Exploring Pre-trained General-purpose Audio Representations

Pre-trained models are essential as feature extractors in modern machine...
research
04/10/2022

Deep Embeddings for Robust User-Based Amateur Vocal Percussion Classification

Vocal Percussion Transcription (VPT) is concerned with the automatic det...
research
03/03/2023

Low-Complexity Audio Embedding Extractors

Solving tasks such as speaker recognition, music classification, or sema...
research
05/17/2022

Composing General Audio Representation by Fusing Multilayer Features of a Pre-trained Model

Many application studies rely on audio DNN models pre-trained on a large...

Please sign up or login with your details

Forgot password? Click here to reset