Midpoint Regularization: from High Uncertainty Training to Conservative Classification

06/26/2021
by   Hongyu Guo, et al.
0

Label Smoothing (LS) improves model generalization through penalizing models from generating overconfident output distributions. For each training sample the LS strategy smooths the one-hot encoded training signal by distributing its distribution mass over the non-ground truth classes. We extend this technique by considering example pairs, coined PLS. PLS first creates midpoint samples by averaging random sample pairs and then learns a smoothing distribution during training for each of these midpoint samples, resulting in midpoints with high uncertainty labels for training. We empirically show that PLS significantly outperforms LS, achieving up to 30 We also visualize that PLS produces very low winning softmax scores for both in and out of distribution samples.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/02/2020

Regularization via Adaptive Pairwise Label Smoothing

Label Smoothing (LS) is an effective regularizer to improve the generali...
research
06/19/2022

Gray Learning from Non-IID Data with Out-of-distribution Samples

The quality of the training data annotated by experts cannot be guarante...
research
06/19/2022

Out-of-distribution Detection by Cross-class Vicinity Distribution of In-distribution Data

Deep neural networks only learn to map in-distribution inputs to their c...
research
10/28/2022

An Approach for Noisy, Crowdsourced Datasets Utilizing Ensemble Modeling, 'Human Softmax' Distributions, and Entropic Measures of Uncertainty

Noisy, crowdsourced image datasets prove challenging, even for the best ...
research
01/07/2020

Regularization via Structural Label Smoothing

Regularization is an effective way to promote the generalization perform...
research
02/09/2021

Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection

Detecting out-of-distribution (OOD) examples is critical in many applica...
research
12/15/2020

Weakly Supervised Label Smoothing

We study Label Smoothing (LS), a widely used regularization technique, i...

Please sign up or login with your details

Forgot password? Click here to reset