StyleAugment: Learning Texture De-biased Representations by Style Augmentation without Pre-defined Textures

08/24/2021
by   Sanghyuk Chun, et al.
0

Recent powerful vision classifiers are biased towards textures, while shape information is overlooked by the models. A simple attempt by augmenting training images using the artistic style transfer method, called Stylized ImageNet, can reduce the texture bias. However, Stylized ImageNet approach has two drawbacks in fidelity and diversity. First, the generated images show low image quality due to the significant semantic gap betweeen natural images and artistic paintings. Also, Stylized ImageNet training samples are pre-computed before training, resulting in showing the lack of diversity for each sample. We propose a StyleAugment by augmenting styles from the mini-batch. StyleAugment does not rely on the pre-defined style references, but generates augmented images on-the-fly by natural images in the mini-batch for the references. Hence, StyleAugment let the model observe abundant confounding cues for each image by on-the-fly the augmentation strategy, while the augmented images are more realistic than artistic style transferred images. We validate the effectiveness of StyleAugment in the ImageNet dataset with robustness benchmarks, such as texture de-biased accuracy, corruption robustness, natural adversarial samples, and occlusion robustness. StyleAugment shows better generalization performances than previous unsupervised de-biasing methods and state-of-the-art data augmentation methods in our experiments.

READ FULL TEXT
research
04/20/2021

Does enhanced shape bias improve neural network robustness to common corruptions?

Convolutional neural networks (CNNs) learn to extract representations of...
research
10/12/2020

Shape-Texture Debiased Neural Network Training

Shape and texture are two prominent and complementary cues for recognizi...
research
08/29/2023

WSAM: Visual Explanations from Style Augmentation as Adversarial Attacker and Their Influence in Image Classification

Currently, style augmentation is capturing attention due to convolutiona...
research
11/23/2019

Universal Adversarial Perturbations to Understand Robustness of Texture vs. Shape-biased Training

Convolutional Neural Networks (CNNs) used on image classification tasks ...
research
10/15/2021

Data Generation using Texture Co-occurrence and Spatial Self-Similarity for Debiasing

Classification models trained on biased datasets usually perform poorly ...
research
06/22/2021

Wallpaper Texture Generation and Style Transfer Based on Multi-label Semantics

Textures contain a wealth of image information and are widely used in va...
research
11/14/2022

Robustifying Deep Vision Models Through Shape Sensitization

Recent work has shown that deep vision models tend to be overly dependen...

Please sign up or login with your details

Forgot password? Click here to reset