Comparison of Training Methods for Deep Neural Networks

04/26/2015
by   Patrick O. Glauner, et al.
0

This report describes the difficulties of training neural networks and in particular deep neural networks. It then provides a literature review of training methods for deep neural networks, with a focus on pre-training. It focuses on Deep Belief Networks composed of Restricted Boltzmann Machines and Stacked Autoencoders and provides an outreach on further and alternative approaches. It also includes related practical recommendations from the literature on training them. In the second part, initial experiments using some of the covered methods are performed on two databases. In particular, experiments are performed on the MNIST hand-written digit dataset and on facial emotion data from a Kaggle competition. The results are discussed in the context of results reported in other research papers. An error rate lower than the best contribution to the Kaggle competition is achieved using an optimized Stacked Autoencoder.

READ FULL TEXT

page 38

page 40

research
08/26/2015

Deep Convolutional Neural Networks for Smile Recognition

This thesis describes the design and implementation of a smile detector ...
research
12/22/2016

How to Train Your Deep Neural Network with Dictionary Learning

Currently there are two predominant ways to train deep neural networks. ...
research
03/23/2016

A Tutorial on Deep Neural Networks for Intelligent Systems

Developing Intelligent Systems involves artificial intelligence approach...
research
01/01/2018

Accelerating Deep Learning with Memcomputing

Restricted Boltzmann machines (RBMs) and their extensions, often called ...
research
05/16/2016

Alternating optimization method based on nonnegative matrix factorizations for deep neural networks

The backpropagation algorithm for calculating gradients has been widely ...
research
04/12/2019

An Empirical Evaluation Study on the Training of SDC Features for Dense Pixel Matching

Training a deep neural network is a non-trivial task. Not only the tunin...
research
12/03/2009

Behavior and performance of the deep belief networks on image classification

We apply deep belief networks of restricted Boltzmann machines to bags o...

Please sign up or login with your details

Forgot password? Click here to reset