DeepAI AI Chat
Log In Sign Up

Learning audio representations via phase prediction

We learn audio representations by solving a novel self-supervised learning task, which consists of predicting the phase of the short-time Fourier transform from its magnitude. A convolutional encoder is used to map the magnitude spectrum of the input waveform to a lower dimensional embedding. A convolutional decoder is then used to predict the instantaneous frequency (i.e., the temporal rate of change of the phase) from such embedding. To evaluate the quality of the learned representations, we evaluate how they transfer to a wide variety of downstream audio tasks. Our experiments reveal that the phase prediction task leads to representations that generalize across different tasks, partially bridging the gap with fully-supervised models. In addition, we show that the predicted phase can be used as initialization of the Griffin-Lim algorithm, thus reducing the number of iterations needed to reconstruct the waveform in the time domain.


Masked Spectrogram Prediction For Self-Supervised Audio Pre-Training

Transformer-based models attain excellent results and generalize well wh...

A Deep Generative Model of Speech Complex Spectrograms

This paper proposes an approach to the joint modeling of the short-time ...

Self-supervised audio representation learning for mobile devices

We explore self-supervised models that can be potentially deployed on mo...

Using growth transform dynamical systems for spatio-temporal data sonification

Sonification, or encoding information in meaningful audio signatures, ha...

Temporal envelope and fine structure cues for dysarthric speech detection using CNNs

Deep learning-based techniques for automatic dysarthric speech detection...

iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transform

In recent text-to-speech synthesis and voice conversion systems, a mel-s...

Point Cloud Audio Processing

Most audio processing pipelines involve transformations that act on fixe...