Neural Networks Regularization Through Class-wise Invariant Representation Learning

09/06/2017
by   Soufiane Belharbi, et al.
0

Training deep neural networks is known to require a large number of training samples. However, in many applications only few training samples are available. In this work, we tackle the issue of training neural networks for classification task when few training samples are available. We attempt to solve this issue by proposing a new regularization term that constrains the hidden layers of a network to learn class-wise invariant representations. In our regularization framework, learning invariant representations is generalized to the class membership where samples with the same class should have the same representation. Numerical experiments over MNIST and its variants showed that our proposal helps improving the generalization of neural network particularly when trained with few samples. We provide the source code of our framework https://github.com/sbelharbi/learning-class-invariant-features .

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

07/13/2018

Neural Networks Regularization Through Representation Learning

Neural network models and deep models are one of the leading and state o...
11/28/2020

GradAug: A New Regularization Method for Deep Neural Networks

We propose a new regularization method to alleviate over-fitting in deep...
04/28/2015

Deep Neural Networks Regularization for Structured Output Prediction

A deep neural network model is a powerful framework for learning represe...
01/02/2018

Learning audio and image representations with bio-inspired trainable feature extractors

Recent advancements in pattern recognition and signal processing concern...
02/04/2019

Deep One-Class Classification Using Data Splitting

This paper introduces a generic method which enables to use conventional...
06/19/2017

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability

We propose a new technique, Singular Vector Canonical Correlation Analys...
01/19/2019

MOROCO: The Moldavian and Romanian Dialectal Corpus

In this work, we introduce the MOldavian and ROmanian Dialectal COrpus (...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.