Adversarially Robust Video Perception by Seeing Motion

12/13/2022
by   Lingyu Zhang, et al.
0

Despite their excellent performance, state-of-the-art computer vision models often fail when they encounter adversarial examples. Video perception models tend to be more fragile under attacks, because the adversary has more places to manipulate in high-dimensional data. In this paper, we find one reason for video models' vulnerability is that they fail to perceive the correct motion under adversarial perturbations. Inspired by the extensive evidence that motion is a key factor for the human visual system, we propose to correct what the model sees by restoring the perceived motion information. Since motion information is an intrinsic structure of the video data, recovering motion signals can be done at inference time without any human annotation, which allows the model to adapt to unforeseen, worst-case inputs. Visualizations and empirical experiments on UCF-101 and HMDB-51 datasets show that restoring motion information in deep vision models improves adversarial robustness. Even under adaptive attacks where the adversary knows our defense, our algorithm is still effective. Our work provides new insight into robust video perception algorithms by using intrinsic structures from the data. Our webpage is available at https://motion4robust.cs.columbia.edu.

READ FULL TEXT

page 2

page 4

page 6

page 8

page 13

page 14

page 15

page 16

research
12/12/2022

Robust Perception through Equivariance

Deep networks for computer vision are not reliable when they encounter a...
research
04/10/2020

Luring of Adversarial Perturbations

The growing interest for adversarial examples, i.e. maliciously modified...
research
03/17/2020

Motion-Excited Sampler: Video Adversarial Attack with Sparked Prior

Deep neural networks are known to be susceptible to adversarial noise, w...
research
10/04/2022

Robustness Certification of Visual Perception Models via Camera Motion Smoothing

A vast literature shows that the learning-based visual perception model ...
research
03/11/2022

Perception Over Time: Temporal Dynamics for Robust Image Understanding

While deep learning surpasses human-level performance in narrow and spec...
research
05/04/2020

Robust Encodings: A Framework for Combating Adversarial Typos

Despite excellent performance on many tasks, NLP systems are easily fool...
research
06/05/2019

MNIST-C: A Robustness Benchmark for Computer Vision

We introduce the MNIST-C dataset, a comprehensive suite of 15 corruption...

Please sign up or login with your details

Forgot password? Click here to reset