Muddling Label Regularization: Deep Learning for Tabular Datasets

06/08/2021
by   Karim Lounici, et al.
0

Deep Learning (DL) is considered the state-of-the-art in computer vision, speech recognition and natural language processing. Until recently, it was also widely accepted that DL is irrelevant for learning tasks on tabular data, especially in the small sample regime where ensemble methods are acknowledged as the gold standard. We present a new end-to-end differentiable method to train a standard FFNN. Our method, Muddling labels for Regularization (), penalizes memorization through the generation of uninformative labels and the application of a differentiable close-form regularization scheme on the last hidden layer during training. outperforms classical NN and the gold standard (GBDT, RF) for regression and classification tasks on several datasets from the UCI database and Kaggle covering a large range of sample sizes and feature to sample ratios. Researchers and practitioners can use on its own as an off-the-shelf solution or integrate it into the most advanced ML pipelines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/25/2021

Scalable End-to-End RF Classification: A Case Study on Undersized Dataset Regularization by Convolutional-MST

Unlike areas such as computer vision and speech recognition where convol...
research
02/13/2020

Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges

Deep Learning (DL) techniques for Natural Language Processing have been ...
research
10/11/2021

Disturbing Target Values for Neural Network Regularization

Diverse regularization techniques have been developed such as L2 regular...
research
08/02/2019

DELTA: A DEep learning based Language Technology plAtform

In this paper we present DELTA, a deep learning based language technolog...
research
06/11/2020

Is deep learning necessary for simple classification tasks?

Automated machine learning (AutoML) and deep learning (DL) are two cutti...
research
06/22/2020

Bayesian Neural Networks: An Introduction and Survey

Neural Networks (NNs) have provided state-of-the-art results for many ch...
research
06/16/2020

On the Inference of Soft Biometrics from Typing Patterns Collected in a Multi-device Environment

In this paper, we study the inference of gender, major/minor (computer s...

Please sign up or login with your details

Forgot password? Click here to reset