Robust Perception through Equivariance

12/12/2022
by   Chengzhi Mao, et al.
0

Deep networks for computer vision are not reliable when they encounter adversarial examples. In this paper, we introduce a framework that uses the dense intrinsic constraints in natural images to robustify inference. By introducing constraints at inference time, we can shift the burden of robustness from training to the inference algorithm, thereby allowing the model to adjust dynamically to each individual image's unique and potentially novel characteristics at inference time. Among different constraints, we find that equivariance-based constraints are most effective, because they allow dense constraints in the feature space without overly constraining the representation at a fine-grained level. Our theoretical results validate the importance of having such dense constraints at inference time. Our empirical experiments show that restoring feature equivariance at inference time defends against worst-case adversarial perturbations. The method obtains improved adversarial robustness on four datasets (ImageNet, Cityscapes, PASCAL VOC, and MS-COCO) on image recognition, semantic segmentation, and instance segmentation tasks. Project page is available at equi4robust.cs.columbia.edu.

READ FULL TEXT

page 1

page 3

page 8

research
03/24/2017

Adversarial Examples for Semantic Segmentation and Object Detection

It has been well demonstrated that adversarial examples, i.e., natural i...
research
12/13/2022

Adversarially Robust Video Perception by Seeing Motion

Despite their excellent performance, state-of-the-art computer vision mo...
research
07/02/2020

Trace-Norm Adversarial Examples

White box adversarial perturbations are sought via iterative optimizatio...
research
06/20/2019

Improving the robustness of ImageNet classifiers using elements of human visual cognition

We investigate the robustness properties of image recognition models equ...
research
06/07/2023

PhenoBench – A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain

The production of food, feed, fiber, and fuel is a key task of agricultu...
research
03/20/2018

An Improved Evaluation Framework for Generative Adversarial Networks

In this paper, we propose an improved quantitative evaluation framework ...
research
05/23/2021

Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation

Recent studies imply that deep neural networks are vulnerable to adversa...

Please sign up or login with your details

Forgot password? Click here to reset