Evolving Unsupervised Deep Neural Networks for Learning Meaningful Representations

12/13/2017
by   Yanan Sun, et al.
0

Deep Learning (DL) aims at learning the meaningful representations. A meaningful representation refers to the one that gives rise to significant performance improvement of associated Machine Learning (ML) tasks by replacing the raw data as the input. However, optimal architecture design and model parameter estimation in DL algorithms are widely considered to be intractable. Evolutionary algorithms are much preferable for complex and non-convex problems due to its inherent characteristics of gradient-free and insensitivity to local optimum. In this paper, we propose a computationally economical algorithm for evolving unsupervised deep neural networks to efficiently learn meaningful representations, which is very suitable in the current Big Data era where sufficient labeled data for training is often expensive to acquire. In the proposed algorithm, finding an appropriate architecture and the initialized parameter values for a ML task at hand is modeled by one computational efficient gene encoding approach, which is employed to effectively model the task with a large number of parameters. In addition, a local search strategy is incorporated to facilitate the exploitation search for further improving the performance. Furthermore, a small proportion labeled data is utilized during evolution search to guarantee the learnt representations to be meaningful. The performance of the proposed algorithm has been thoroughly investigated over classification tasks. Specifically, error classification rate on MNIST with 1.15% is reached by the proposed algorithm consistently, which is a very promising result against state-of-the-art unsupervised DL algorithms.

READ FULL TEXT

page 5

page 10

page 13

page 19

research
10/30/2017

Evolving Deep Convolutional Neural Networks for Image Classification

Evolutionary computation methods have been successfully applied to neura...
research
07/30/2014

Automated Machine Learning on Big Data using Stochastic Algorithm Tuning

We introduce a means of automating machine learning (ML) for big data ta...
research
02/08/2023

Unsupervised Learning of Initialization in Deep Neural Networks via Maximum Mean Discrepancy

Despite the recent success of stochastic gradient descent in deep learni...
research
12/01/2017

Deep Learning for Metagenomic Data: using 2D Embeddings and Convolutional Neural Networks

Deep learning (DL) techniques have had unprecedented success when applie...
research
01/04/2021

Learning to Optimize Under Constraints with Unsupervised Deep Neural Networks

In this paper, we propose a machine learning (ML) method to learn how to...
research
10/21/2017

Superposed Episodic and Semantic Memory via Sparse Distributed Representation

The abilities to perceive, learn, and use generalities, similarities, cl...

Please sign up or login with your details

Forgot password? Click here to reset