On Regularization Properties of Artificial Datasets for Deep Learning

by   Karol Antczak, et al.

The paper discusses regularization properties of artificial data for deep learning. Artificial datasets allow to train neural networks in the case of a real data shortage. It is demonstrated that the artificial data generation process, described as injecting noise to high-level features, bears several similarities to existing regularization methods for deep neural networks. One can treat this property of artificial data as a kind of "deep" regularization. It is thus possible to regularize hidden layers of the network by generating the training data in a certain way.


DL-Reg: A Deep Learning Regularization Technique using Linear Regression

Regularization plays a vital role in the context of deep learning by pre...

Residual-Concatenate Neural Network with Deep Regularization Layers for Binary Classification

Many complex Deep Learning models are used with different variations for...

Visualization Regularizers for Neural Network based Image Recognition

The success of deep neural networks is mostly due their ability to learn...

Regularization for Deep Learning: A Taxonomy

Regularization is one of the crucial ingredients of deep learning, yet t...

The staircase property: How hierarchical structure can guide deep learning

This paper identifies a structural property of data distributions that e...

Applying Deep Learning to Derivatives Valuation

The universal approximation theorem of artificial neural networks states...

Invertible Neural Networks for Graph Prediction

In this work, we address conditional generation using deep invertible ne...