Using learned optimizers to make models robust to input noise

06/08/2019
by   Luke Metz, et al.
0

State-of-the art vision models can achieve superhuman performance on image classification tasks when testing and training data come from the same distribution. However, when models are tested on corrupted images (e.g. due to scale changes, translations, or shifts in brightness or contrast), performance degrades significantly. Here, we explore the possibility of meta-training a learned optimizer that can train image classification models such that they are robust to common image corruptions. Specifically, we are interested training models that are more robust to noise distributions not present in the training data. We find that a learned optimizer meta-trained to produce models which are robust to Gaussian noise trains models that are more robust to Gaussian noise at other scales compared to traditional optimizers like Adam. The effect of meta-training is more complicated when targeting a more general set of noise distributions, but led to improved performance on half of held-out corruption tasks. Our results suggest that meta-learning provides a novel approach for studying and improving the robustness of deep learning models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/30/2023

Robust Meta Learning for Image based tasks

A machine learning model that generalizes well should obtain low errors ...
research
08/12/2020

Learning to Learn from Mistakes: Robust Optimization for Adversarial Noise

Sensitivity to adversarial noise hinders deployment of machine learning ...
research
04/12/2023

Diffusion models with location-scale noise

Diffusion Models (DMs) are powerful generative models that add Gaussian ...
research
11/17/2022

VeLO: Training Versatile Learned Optimizers by Scaling Up

While deep learning models have replaced hand-designed features across m...
research
09/22/2022

A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases

Learned optimizers – neural networks that are trained to act as optimize...
research
06/15/2022

A Meta-Analysis of Distributionally-Robust Models

State-of-the-art image classifiers trained on massive datasets (such as ...
research
03/12/2023

RotoGBML: Towards Out-of-Distribution Generalization for Gradient-Based Meta-Learning

Gradient-based meta-learning (GBML) algorithms are able to fast adapt to...

Please sign up or login with your details

Forgot password? Click here to reset