DeepAI AI Chat
Log In Sign Up

SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size

by   Jessica Yung, et al.

Before deploying machine learning models it is critical to assess their robustness. In the context of deep neural networks for image understanding, changing the object location, rotation and size may affect the predictions in non-trivial ways. In this work we perform a fine-grained analysis of robustness with respect to these factors of variation using SI-Score, a synthetic dataset. In particular, we investigate ResNets, Vision Transformers and CLIP, and identify interesting qualitative differences between these.


page 2

page 4


Object-centric Sampling for Fine-grained Image Classification

This paper proposes to go beyond the state-of-the-art deep convolutional...

Finding Differences Between Transformers and ConvNets Using Counterfactual Simulation Testing

Modern deep neural networks tend to be evaluated on static test sets. On...

Conviformers: Convolutionally guided Vision Transformer

Vision transformers are nowadays the de-facto preference for image class...

Identification of Fine-Grained Location Mentions in Crisis Tweets

Identification of fine-grained location mentions in crisis tweets is cen...

Let's Roll: Synthetic Dataset Analysis for Pedestrian Detection Across Different Shutter Types

Computer vision (CV) pipelines are typically evaluated on datasets proce...

Fine-Grained Visual Classification of Aircraft

This paper introduces FGVC-Aircraft, a new dataset containing 10,000 ima...

Interventional Robustness of Deep Latent Variable Models

The ability to learn disentangled representations that split underlying ...