Neural Networks Regularization Through Representation Learning

by   Soufiane Belharbi, et al.

Neural network models and deep models are one of the leading and state of the art models in machine learning. Most successful deep neural models are the ones with many layers which highly increases their number of parameters. Training such models requires a large number of training samples which is not always available. One of the fundamental issues in neural networks is overfitting which is the issue tackled in this thesis. Such problem often occurs when the training of large models is performed using few training samples. Many approaches have been proposed to prevent the network from overfitting and improve its generalization performance such as data augmentation, early stopping, parameters sharing, unsupervised learning, dropout, batch normalization, etc. In this thesis, we tackle the neural network overfitting issue from a representation learning perspective by considering the situation where few training samples are available which is the case of many real world applications. We propose three contributions. The first one presented in chapter 2 is dedicated to dealing with structured output problems to perform multivariate regression when the output variable y contains structural dependencies between its components. The second contribution described in chapter 3 deals with the classification task where we propose to exploit prior knowledge about the internal representation of the hidden layers in neural networks. Our last contribution presented in chapter 4 showed the interest of transfer learning in applications where only few samples are available. In this contribution, we provide an automatic system based on such learning scheme with an application to medical domain. In this application, the task consists in localizing the third lumbar vertebra in a 3D CT scan. This work has been done in collaboration with the clinic Rouen Henri Becquerel Center who provided us with data.



There are no comments yet.



Neural Networks Regularization Through Class-wise Invariant Representation Learning

Training deep neural networks is known to require a large number of trai...

Reduction of Overfitting in Diabetes Prediction Using Deep Learning Neural Network

Augmented accuracy in prediction of diabetes will open up new frontiers ...

Regularized Deep Networks in Intelligent Transportation Systems: A Taxonomy and a Case Study

Intelligent Transportation Systems (ITS) are much correlated with data s...

Distributed Supervised Learning using Neural Networks

Distributed learning is the problem of inferring a function in the case ...

Overfitting Mechanism and Avoidance in Deep Neural Networks

Assisted by the availability of data and high performance computing, dee...

NeuCrowd: Neural Sampling Network for Representation Learning with Crowdsourced Labels

Representation learning approaches require a massive amount of discrimin...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.