Contemplating real-world object classification

03/08/2021
by   Ali Borji, et al.
0

Deep object recognition models have been very successful over benchmark datasets such as ImageNet. How accurate and robust are they to distribution shifts arising from natural and synthetic variations in datasets? Prior research on this problem has primarily focused on ImageNet variations (e.g., ImageNetV2, ImageNet-A). To avoid potential inherited biases in these studies, we take a different approach. Specifically, we reanalyze the ObjectNet dataset recently proposed by Barbu et al. containing objects in daily life situations. They showed a dramatic performance drop of the state of the art object recognition models on this dataset. Due to the importance and implications of their results regarding the generalization ability of deep models, we take a second look at their analysis. We find that applying deep models to the isolated objects, rather than the entire scene as is done in the original paper, results in around 20-30 numbers reported in Barbu et al., around 10-15 recovered, without any test time data augmentation. Despite this gain, however, we conclude that deep models still suffer drastically on the ObjectNet dataset. We also investigate the robustness of models against synthetic image perturbations such as geometric transformations (e.g., scale, rotation, translation), natural image distortions (e.g., impulse noise, blur) as well as adversarial attacks (e.g., FGSM and PGD-5). Our results indicate that limiting the object area as much as possible (i.e., from the entire image to the bounding box to the segmentation mask) leads to consistent improvement in accuracy and robustness.

READ FULL TEXT

page 24

page 26

page 29

page 30

page 32

page 36

page 39

page 40

research
04/04/2020

ObjectNet Dataset: Reanalysis and Correction

Recently, Barbu et al introduced a dataset called ObjectNet which includ...
research
07/01/2020

Measuring Robustness to Natural Distribution Shifts in Image Classification

We study how robust current ImageNet models are to distribution shifts a...
research
04/20/2023

Enhancing object detection robustness: A synthetic and natural perturbation approach

Robustness against real-world distribution shifts is crucial for the suc...
research
08/24/2021

OOWL500: Overcoming Dataset Collection Bias in the Wild

The hypothesis that image datasets gathered online "in the wild" can pro...
research
08/23/2022

How good are deep models in understanding the generated images?

My goal in this paper is twofold: to study how well deep models can unde...
research
03/19/2017

Multilevel Context Representation for Improving Object Recognition

In this work, we propose the combined usage of low- and high-level block...
research
07/24/2023

Does Progress On Object Recognition Benchmarks Improve Real-World Generalization?

For more than a decade, researchers have measured progress in object rec...

Please sign up or login with your details

Forgot password? Click here to reset