Dropout can Simulate Exponential Number of Models for Sample Selection Techniques

02/26/2022
by   Lakshya, et al.
0

Following Coteaching, generally in the literature, two models are used in sample selection based approaches for training with noisy labels. Meanwhile, it is also well known that Dropout when present in a network trains an ensemble of sub-networks. We show how to leverage this property of Dropout to train an exponential number of shared models, by training a single model with Dropout. We show how we can modify existing two model-based sample selection methodologies to use an exponential number of shared models. Not only is it more convenient to use a single model with Dropout, but this approach also combines the natural benefits of Dropout with that of training an exponential number of models, leading to improved results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/09/2015

Efficient batchwise dropout training using submatrices

Dropout is a popular technique for regularizing artificial neural networ...
research
06/29/2015

Dropout as data augmentation

Dropout is typically interpreted as bagging a large number of models sha...
research
08/03/2021

MBDP: A Model-based Approach to Achieve both Robustness and Sample Efficiency via Double Dropout Planning

Model-based reinforcement learning is a widely accepted solution for sol...
research
09/13/2020

Machine Learning's Dropout Training is Distributionally Robust Optimal

This paper shows that dropout training in Generalized Linear Models is t...
research
02/18/2013

Maxout Networks

We consider the problem of designing models to leverage a recently intro...
research
11/01/2019

Kinetic foundation of the zero-inflated negative binomial model for single-cell RNA sequencing data

Single-cell RNA sequencing data have complex features such as dropout ev...
research
07/11/2014

Altitude Training: Strong Bounds for Single-Layer Dropout

Dropout training, originally designed for deep neural networks, has been...

Please sign up or login with your details

Forgot password? Click here to reset