Detecting Adversarial Attacks On Audio-Visual Speech Recognition

12/18/2019
by   Pingchuan Ma, et al.
22

Adversarial attacks pose a threat to deep learning models. However, research on adversarial detection methods, especially in the multi-modal domain, is very limited. In this work, we propose an efficient and straightforward detection method based on the temporal correlation between audio and video streams. The main idea is that the correlation between audio and video in adversarial examples will be lower than benign examples due to added adversarial noise. We use the synchronisation confidence score as a proxy for audio-visual correlation and based on it we can detect adversarial attacks. To the best of our knowledge, this is the first work on detection of adversarial attacks on audio-visual speech recognition models. We apply recent adversarial attacks on two audio-visual speech recognition models trained on the GRID and LRW datasets. The experimental results demonstrated that the proposed approach is an effective way for detecting such attacks.

READ FULL TEXT
research
09/11/2018

Isolated and Ensemble Audio Preprocessing Methods for Detecting Adversarial Examples against Automatic Speech Recognition

An adversarial attack is an exploitative process in which minute alterat...
research
10/03/2022

Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection

Audio-visual active speaker detection (AVASD) is well-developed, and now...
research
01/02/2018

Did you hear that? Adversarial Examples Against Automatic Speech Recognition

Speech is a common and effective way of communication between humans, an...
research
02/13/2020

Identifying Audio Adversarial Examples via Anomalous Pattern Detection

Audio processing models based on deep neural networks are susceptible to...
research
10/26/2019

Detection of Adversarial Attacks and Characterization of Adversarial Subspace

Adversarial attacks have always been a serious threat for any data-drive...
research
12/30/2022

Defense Against Adversarial Attacks on Audio DeepFake Detection

Audio DeepFakes are artificially generated utterances created using deep...
research
07/10/2019

Metamorphic Detection of Adversarial Examples in Deep Learning Models With Affine Transformations

Adversarial attacks are small, carefully crafted perturbations, impercep...

Please sign up or login with your details

Forgot password? Click here to reset