DeepAI AI Chat
Log In Sign Up

CDPAM: Contrastive learning for perceptual audio similarity

by   Pranay Manocha, et al.

Many speech processing methods based on deep learning require an automatic and differentiable audio metric for the loss function. The DPAM approach of Manocha et al. learns a full-reference metric trained directly on human judgments, and thus correlates well with human perception. However, it requires a large number of human annotations and does not generalize well outside the range of perturbations on which it was trained. This paper introduces CDPAM, a metric that builds on and advances DPAM. The primary improvement is to combine contrastive learning and multi-dimensional representations to build robust models from limited data. In addition, we collect human judgments on triplet comparisons to improve generalization to a broader range of audio perturbations. CDPAM correlates well with human responses across nine varied datasets. We also show that adding this metric to existing speech synthesis and enhancement methods yields significant improvement, as measured by objective and subjective tests.


page 1

page 2

page 3

page 4


A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences

Assessment of many audio processing tasks relies on subjective evaluatio...

SAQAM: Spatial Audio Quality Assessment Metric

Audio quality assessment is critical for assessing the perceptual realis...

BWSNet: Automatic Perceptual Assessment of Audio Signals

This paper introduces BWSNet, a model that can be trained from raw human...

Perceptual audio loss function for deep learning

PESQ and POLQA , are standards are standards for automated assessment of...

Enhancing GAN-Based Vocoders with Contrastive Learning Under Data-limited Condition

Vocoder models have recently achieved substantial progress in generating...

DPLM: A Deep Perceptual Spatial-Audio Localization Metric

Subjective evaluations are critical for assessing the perceptual realism...

PerceptNet: Learning Perceptual Similarity of Haptic Textures in Presence of Unorderable Triplets

In order to design haptic icons or build a haptic vocabulary, we require...

Code Repositories


Perceptual Metrics of Audio - perceptually relevant loss function. DPAM and CDPAM

view repo