Parameter Sensitivity of Deep-Feature based Evaluation Metrics for Audio Textures

08/23/2022
by   Chitralekha Gupta, et al.
0

Standard evaluation metrics such as the Inception score and Fréchet Audio Distance provide a general audio quality distance metric between the synthesized audio and reference clean audio. However, the sensitivity of these metrics to variations in the statistical parameters that define an audio texture is not well studied. In this work, we provide a systematic study of the sensitivity of some of the existing audio quality evaluation metrics to parameter variations in audio textures. Furthermore, we also study three more potentially parameter-sensitive metrics for audio texture synthesis, (a) a Gram matrix based distance, (b) an Accumulated Gram metric using a summarized version of the Gram matrices, and (c) a cochlear-model based statistical features metric. These metrics use deep features that summarize the statistics of any given audio texture, thus being inherently sensitive to variations in the statistical parameters that define an audio texture. We study and evaluate the sensitivity of existing standard metrics as well as Gram matrix and cochlear-model based metrics to control-parameter variations in audio textures across a wide range of texture and parameter types, and validate with subjective evaluation. We find that each of the metrics is sensitive to different sets of texture-parameter types. This is the first step towards investigating objective metrics for assessing parameter sensitivity in audio textures.

READ FULL TEXT
research
06/20/2018

Synthesizing Diverse, High-Quality Audio Textures

Texture synthesis techniques based on matching the Gram matrix of featur...
research
09/30/2020

Rethinking Evaluation Methodology for Audio-to-Score Alignment

This paper offers a precise, formal definition of an audio-to-score alig...
research
11/25/2020

MTCRNN: A multi-scale RNN for directed audio texture synthesis

Audio textures are a subset of environmental sounds, often defined as ha...
research
12/20/2018

Fréchet Audio Distance: A Metric for Evaluating Music Enhancement Algorithms

We propose the Fréchet Audio Distance (FAD), a novel, reference-free eva...
research
06/11/2021

Assessing Political Prudence of Open-domain Chatbots

Politically sensitive topics are still a challenge for open-domain chatb...
research
05/19/2019

An Objective Evaluation Metric for image fusion based on Del Operator

In this paper, a novel objective evaluation metric for image fusion is p...

Please sign up or login with your details

Forgot password? Click here to reset