Dataset artefacts in anti-spoofing systems: a case study on the ASVspoof 2017 benchmark

10/15/2020
by   Bhusan Chettri, et al.
0

The Automatic Speaker Verification Spoofing and Countermeasures Challenges motivate research in protecting speech biometric systems against a variety of different access attacks. The 2017 edition focused on replay spoofing attacks, and involved participants building and training systems on a provided dataset (ASVspoof 2017). More than 60 research papers have so far been published with this dataset, but none have sought to answer why countermeasures appear successful in detecting spoofing attacks. This article shows how artefacts inherent to the dataset may be contributing to the apparent success of published systems. We first inspect the ASVspoof 2017 dataset and summarize various artefacts present in the dataset. Second, we demonstrate how countermeasure models can exploit these artefacts to appear successful in this dataset. Third, for reliable and robust performance estimates on this dataset we propose discarding nonspeech segments and silence before and after the speech utterance during training and inference. We create speech start and endpoint annotations in the dataset and demonstrate how using them helps countermeasure models become less vulnerable from being manipulated using artefacts found in the dataset. Finally, we provide several new benchmark results for both frame-level and utterance-level models that can serve as new baselines on this dataset.

READ FULL TEXT

page 1

page 5

research
12/06/2020

Multi-task Learning Based Spoofing-Robust Automatic Speaker Verification System

Spoofing attacks posed by generating artificial speech can severely degr...
research
05/24/2017

Anti-spoofing Methods for Automatic SpeakerVerification System

Growing interest in automatic speaker verification (ASV)systems has lead...
research
01/24/2022

Optimizing Tandem Speaker Verification and Anti-Spoofing Systems

As automatic speaker verification (ASV) systems are vulnerable to spoofi...
research
05/28/2019

Towards robust audio spoofing detection: a detailed comparison of traditional and learned features

Automatic speaker verification, like every other biometric system, is vu...
research
09/18/2023

Frame-to-Utterance Convergence: A Spectra-Temporal Approach for Unified Spoofing Detection

Voice spoofing attacks pose a significant threat to automated speaker ve...
research
04/11/2022

The PartialSpoof Database and Countermeasures for the Detection of Short Generated Audio Segments Embedded in a Speech Utterance

Automatic speaker verification is susceptible to various manipulations a...
research
09/15/2023

Improving Short Utterance Anti-Spoofing with AASIST2

The wav2vec 2.0 and integrated spectro-temporal graph attention network ...

Please sign up or login with your details

Forgot password? Click here to reset