Back-to-Bones: Rediscovering the Role of Backbones in Domain Generalization

09/02/2022
by   Simone Angarano, et al.
18

Domain Generalization (DG) studies the capability of a deep learning model to generalize to out-of-training distributions. In the last decade, literature has been massively filled with a collection of training methodologies that claim to obtain more abstract and robust data representations to tackle domain shifts. Recent research has provided a reproducible benchmark for DG, pointing out the effectiveness of naive empirical risk minimization (ERM) over existing algorithms. Nevertheless, researchers persist in using the same outdated feature extractors, and no attention has been given to the effects of different backbones yet. In this paper, we start back to backbones proposing a comprehensive analysis of their intrinsic generalization capabilities, so far ignored by the research community. We evaluate a wide variety of feature extractors, from standard residual solutions to transformer-based architectures, finding an evident linear correlation between large-scale single-domain classification accuracy and DG capability. Our extensive experimentation shows that by adopting competitive backbones in conjunction with effective data augmentation, plain ERM outperforms recent DG solutions and achieves state-of-the-art accuracy. Moreover, our additional qualitative studies reveal that novel backbones give more similar representations to same-class samples, separating different domains in the feature space. This boost in generalization capabilities leaves marginal room for DG algorithms and suggests a new paradigm for investigating the problem, placing backbones in the spotlight and encouraging the development of consistent algorithms on top of them.

READ FULL TEXT

page 7

page 19

page 20

research
12/09/2021

Adaptive Methods for Aggregated Domain Generalization

Domain generalization involves learning a classifier from a heterogeneou...
research
06/07/2021

OoD-Bench: Benchmarking and Understanding Out-of-Distribution Generalization Datasets and Algorithms

Deep learning has achieved tremendous success with independent and ident...
research
02/14/2023

Robust Representation Learning with Self-Distillation for Domain Generalization

Domain generalization is a challenging problem in machine learning, wher...
research
04/04/2023

Randomized Adversarial Style Perturbations for Domain Generalization

We propose a novel domain generalization technique, referred to as Rando...
research
05/09/2022

Localized Adversarial Domain Generalization

Deep learning methods can struggle to handle domain shifts not seen in t...
research
07/12/2023

Single Domain Generalization via Normalised Cross-correlation Based Convolutions

Deep learning techniques often perform poorly in the presence of domain ...
research
11/27/2021

Towards Principled Disentanglement for Domain Generalization

A fundamental challenge for machine learning models is generalizing to o...

Please sign up or login with your details

Forgot password? Click here to reset