Log In Sign Up

Improving weakly supervised sound event detection with self-supervised auxiliary tasks

by   Soham Deshmukh, et al.

While multitask and transfer learning has shown to improve the performance of neural networks in limited data settings, they require pretraining of the model on large datasets beforehand. In this paper, we focus on improving the performance of weakly supervised sound event detection in low data and noisy settings simultaneously without requiring any pretraining task. To that extent, we propose a shared encoder architecture with sound event detection as a primary task and an additional secondary decoder for a self-supervised auxiliary task. We empirically evaluate the proposed framework for weakly supervised sound event detection on a remix dataset of the DCASE 2019 task 1 acoustic scene data with DCASE 2018 Task 2 sounds event data under 0, 10 and 20 dB SNR. To ensure we retain the localisation information of multiple sound events, we propose a two-step attention pooling mechanism that provides a time-frequency localisation of multiple audio events in the clip. The proposed framework with two-step attention outperforms existing benchmark models by 22.3 ablation study to determine the contribution of the auxiliary task and two-step attention pooling to the SED performance improvement.


Multi-Task Learning for Interpretable Weakly Labelled Sound Event Detection

Weakly Labelled learning has garnered lot of attention in recent years d...

Joint Analysis of Acoustic Scenes and Sound Events with Weakly labeled Data

Considering that acoustic scenes and sound events are closely related to...

Hierarchical Pooling Structure for Weakly Labeled Sound Event Detection

Sound event detection with weakly labeled data is considered as a proble...

Affinity Mixup for Weakly Supervised Sound Event Detection

The weakly supervised sound event detection problem is the task of predi...

Disentangled Feature for Weakly Supervised Multi-class Sound Event Detection

We propose a disentangled feature for weakly supervised multiclass sound...

Self-paced Learning for Weakly Supervised Evidence Discovery in Multimedia Event Search

Multimedia event detection has been receiving increasing attention in re...