GACELA – A generative adversarial context encoder for long audio inpainting

05/11/2020
by   Andrés Marafioti, et al.
5

We introduce GACELA, a generative adversarial network (GAN) designed to restore missing musical audio data with a duration ranging between hundreds of milliseconds to a few seconds, i.e., to perform long-gap audio inpainting. While previous work either addressed shorter gaps or relied on exemplars by copying available information from other signal parts, GACELA addresses the inpainting of long gaps in two aspects. First, it considers various time scales of audio information by relying on five parallel discriminators with increasing resolution of receptive fields. Second, it is conditioned not only on the available information surrounding the gap, i.e., the context, but also on the latent variable of the conditional GAN. This addresses the inherent multi-modality of audio inpainting at such long gaps and provides the option of user-defined inpainting. GACELA was tested in listening tests on music signals of varying complexity and gap durations ranging from 375 ms to 1500 ms. While our subjects were often able to detect the inpaintings, the severity of the artifacts decreased from unacceptable to mildly disturbing. GACELA represents a framework capable to integrate future improvements such as processing of more auditory-related features or more explicit musical features.

READ FULL TEXT

page 4

page 5

page 8

page 9

research
10/09/2020

Audio-Visual Speech Inpainting with Deep Learning

In this paper, we present a deep-learning-based framework for audio-visu...
research
10/29/2018

A context encoder for audio inpainting

We studied the ability of deep neural networks (DNNs) to restore missing...
research
05/24/2023

Diffusion-Based Audio Inpainting

Audio inpainting aims to reconstruct missing segments in corrupted recor...
research
03/13/2020

Audio inpainting with generative adversarial network

We study the ability of Wasserstein Generative Adversarial Network (WGAN...
research
01/08/2020

Audio Inpainting: Revisited and Reweighted

We deal with the problem of sparsity-based audio inpainting. A consequen...
research
07/04/2022

Stochastic Restoration of Heavily Compressed Musical Audio using Generative Adversarial Networks

Lossy audio codecs compress (and decompress) digital audio streams by re...
research
07/22/2016

Similarity graphs for the concealment of long duration data loss in music

We present a novel method for the compensation of long duration data gap...

Please sign up or login with your details

Forgot password? Click here to reset