AutoClip: Adaptive Gradient Clipping for Source Separation Networks

07/25/2020
by   Prem Seetharaman, et al.
0

Clipping the gradient is a known approach to improving gradient descent, but requires hand selection of a clipping threshold hyperparameter. We present AutoClip, a simple method for automatically and adaptively choosing a gradient clipping threshold, based on the history of gradient norms observed during training. Experimental results show that applying AutoClip results in improved generalization performance for audio source separation networks. Observation of the training dynamics of a separation network trained with and without AutoClip show that AutoClip guides optimization into smoother parts of the loss landscape. AutoClip is very simple to implement and can be integrated readily into a variety of applications across multiple domains.

READ FULL TEXT
research
10/07/2020

Adversarial attacks on audio source separation

Despite the excellent performance of neural-network-based audio source s...
research
10/20/2020

Phase recovery with Bregman divergences for audio source separation

Time-frequency audio source separation is usually achieved by estimating...
research
10/25/2020

Unified Gradient Reweighting for Model Biasing with Applications to Source Separation

Recent deep learning approaches have shown great improvement in audio so...
research
05/08/2020

Asteroid: the PyTorch-based audio source separation toolkit for researchers

This paper describes Asteroid, the PyTorch-based audio source separation...
research
12/28/2011

A general framework for online audio source separation

We consider the problem of online audio source separation. Existing algo...
research
09/28/2021

FastMVAE2: On improving and accelerating the fast variational autoencoder-based source separation algorithm for determined mixtures

This paper proposes a new source model and training scheme to improve th...
research
04/06/2018

Does k Matter? k-NN Hubness Analysis for Kernel Additive Modelling Vocal Separation

Kernel Additive Modelling (KAM) is a framework for source separation aim...

Please sign up or login with your details

Forgot password? Click here to reset