Learning audio representations via phase prediction

by   Félix de Chaumont Quitry, et al.

We learn audio representations by solving a novel self-supervised learning task, which consists of predicting the phase of the short-time Fourier transform from its magnitude. A convolutional encoder is used to map the magnitude spectrum of the input waveform to a lower dimensional embedding. A convolutional decoder is then used to predict the instantaneous frequency (i.e., the temporal rate of change of the phase) from such embedding. To evaluate the quality of the learned representations, we evaluate how they transfer to a wide variety of downstream audio tasks. Our experiments reveal that the phase prediction task leads to representations that generalize across different tasks, partially bridging the gap with fully-supervised models. In addition, we show that the predicted phase can be used as initialization of the Griffin-Lim algorithm, thus reducing the number of iterations needed to reconstruct the waveform in the time domain.



There are no comments yet.


page 3


Learning Speech Representations from Raw Audio by Joint Audiovisual Self-Supervision

The intuitive interaction between the audio and visual modalities is val...

Audio Classification of Bit-Representation Waveform

This paper investigates waveform representation for audio signal classif...

A Deep Generative Model of Speech Complex Spectrograms

This paper proposes an approach to the joint modeling of the short-time ...

Using growth transform dynamical systems for spatio-temporal data sonification

Sonification, or encoding information in meaningful audio signatures, ha...

Temporal envelope and fine structure cues for dysarthric speech detection using CNNs

Deep learning-based techniques for automatic dysarthric speech detection...

Self-supervised audio representation learning for mobile devices

We explore self-supervised models that can be potentially deployed on mo...

A Multi-Head Relevance Weighting Framework For Learning Raw Waveform Audio Representations

In this work, we propose a multi-head relevance weighting framework to l...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.