DeepAI AI Chat
Log In Sign Up

How to Train Your Deep Neural Network with Dictionary Learning

12/22/2016
by   Vanika Singhal, et al.
0

Currently there are two predominant ways to train deep neural networks. The first one uses restricted Boltzmann machine (RBM) and the second one autoencoders. RBMs are stacked in layers to form deep belief network (DBN); the final representation layer is attached to the target to complete the deep neural network. Autoencoders are nested one inside the other to form stacked autoencoders; once the stcaked autoencoder is learnt the decoder portion is detached and the target attached to the deepest layer of the encoder to form the deep neural network. This work proposes a new approach to train deep neural networks using dictionary learning as the basic building block; the idea is to use the features from the shallower layer as inputs for training the next deeper layer. One can use any type of dictionary learning (unsupervised, supervised, discriminative etc.) as basic units till the pre-final layer. In the final layer one needs to use the label consistent dictionary learning formulation for classification. We compare our proposed framework with existing state-of-the-art deep learning techniques on benchmark problems; we are always within the top 10 results. In actual problems of age and gender classification, we are better than the best known techniques.

READ FULL TEXT

page 1

page 2

page 3

page 4

01/31/2016

Greedy Deep Dictionary Learning

In this work we propose a new deep learning tool called deep dictionary ...
09/11/2018

Deep Micro-Dictionary Learning and Coding Network

In this paper, we propose a novel Deep Micro-Dictionary Learning and Cod...
10/04/2019

Stacked Autoencoder Based Deep Random Vector Functional Link Neural Network for Classification

Extreme learning machine (ELM), which can be viewed as a variant of Rand...
04/26/2015

Comparison of Training Methods for Deep Neural Networks

This report describes the difficulties of training neural networks and i...
08/08/2015

Deep Boosting: Joint Feature Selection and Analysis Dictionary Learning in Hierarchy

This work investigates how the traditional image classification pipeline...
05/05/2011

Rapid Feature Learning with Stacked Linear Denoisers

We investigate unsupervised pre-training of deep architectures as featur...