Efficient Deep Feature Learning and Extraction via StochasticNets

12/11/2015
by   Mohammad Javad Shafiee, et al.
0

Deep neural networks are a powerful tool for feature learning and extraction given their ability to model high-level abstractions in highly complex data. One area worth exploring in feature learning and extraction using deep neural networks is efficient neural connectivity formation for faster feature learning and extraction. Motivated by findings of stochastic synaptic connectivity formation in the brain as well as the brain's uncanny ability to efficiently represent information, we propose the efficient learning and extraction of features via StochasticNets, where sparsely-connected deep neural networks can be formed via stochastic connectivity between neurons. To evaluate the feasibility of such a deep neural network architecture for feature learning and extraction, we train deep convolutional StochasticNets to learn abstract features using the CIFAR-10 dataset, and extract the learned features from images to perform classification on the SVHN and STL-10 datasets. Experimental results show that features learned using deep convolutional StochasticNets, with fewer neural connections than conventional deep convolutional neural networks, can allow for better or comparable classification accuracy than conventional deep neural networks: relative test error decrease of 4.5 classification on the STL-10 dataset and 1 dataset. Furthermore, it was shown that the deep features extracted using deep convolutional StochasticNets can provide comparable classification accuracy even when only 10 it was also shown that significant gains in feature extraction speed can be achieved in embedded applications using StochasticNets. As such, StochasticNets allow for faster feature learning and extraction performance while facilitate for better or comparable accuracy performances.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/22/2015

StochasticNet: Forming Deep Neural Networks via Stochastic Connectivity

Deep neural networks is a branch in machine learning that has seen a met...
research
05/18/2020

Hybrid-DNNs: Hybrid Deep Neural Networks for Mixed Inputs

Rapid development of big data and high-performance computing have encour...
research
12/06/2014

Practice in Synonym Extraction at Large Scale

Synonym extraction is an important task in natural language processing a...
research
08/15/2016

Star-galaxy Classification Using Deep Convolutional Neural Networks

Most existing star-galaxy classifiers use the reduced summary informatio...
research
10/24/2017

Improving Accuracy of Nonparametric Transfer Learning via Vector Segmentation

Transfer learning using deep neural networks as feature extractors has b...
research
06/17/2013

Classifying and Visualizing Motion Capture Sequences using Deep Neural Networks

The gesture recognition using motion capture data and depth sensors has ...
research
10/07/2016

Indoor Space Recognition using Deep Convolutional Neural Network: A Case Study at MIT Campus

In this paper, we propose a robust and parsimonious approach using Deep ...

Please sign up or login with your details

Forgot password? Click here to reset