A Survey: Time Travel in Deep Learning Space: An Introduction to Deep Learning Models and How Deep Learning Models Evolved from the Initial Ideas

10/16/2015
by   Haohan Wang, et al.
0

This report will show the history of deep learning evolves. It will trace back as far as the initial belief of connectionism modelling of brain, and come back to look at its early stage realization: neural networks. With the background of neural network, we will gradually introduce how convolutional neural network, as a representative of deep discriminative models, is developed from neural networks, together with many practical techniques that can help in optimization of neural networks. On the other hand, we will also trace back to see the evolution history of deep generative models, to see how researchers balance the representation power and computation complexity to reach Restricted Boltzmann Machine and eventually reach Deep Belief Nets. Further, we will also look into the development history of modelling time series data with neural networks. We start with Time Delay Neural Networks and move further to currently famous model named Recurrent Neural Network and its extension Long Short Term Memory. We will also briefly look into how to construct deep recurrent neural networks. Finally, we will conclude this report with some interesting open-ended questions of deep neural networks.

READ FULL TEXT
research
01/19/2017

Deep Neural Networks - A Brief History

Introduction to deep neural networks and their history....
research
02/24/2017

On the Origin of Deep Learning

This paper is a review of the evolutionary history of deep learning mode...
research
11/07/2022

Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy

In recent years, deep learning has infiltrated every field it has touche...
research
03/03/2018

The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches

Deep learning has demonstrated tremendous success in variety of applicat...
research
09/11/2020

Applications of Deep Neural Networks

Deep learning is a group of exciting new technologies for neural network...
research
10/06/2017

The Recurrent Temporal Discriminative Restricted Boltzmann Machines

The recurrent temporal restricted Boltzmann machine (RTRBM) has been suc...
research
07/23/2015

Neural NILM: Deep Neural Networks Applied to Energy Disaggregation

Energy disaggregation estimates appliance-by-appliance electricity consu...

Please sign up or login with your details

Forgot password? Click here to reset