EIGEN: Ecologically-Inspired GENetic Approach for Neural Network Structure Searching

06/05/2018
by   Jian Ren, et al.
0

Designing the structure of neural networks is considered one of the most challenging tasks in deep learning. Recently, a few approaches have been proposed to automatically search for the optimal structure of neural networks, however, they suffer from either prohibitive computation cost (e.g., 256 Hours on 250 GPU in [1]) or unsatisfactory performance compared to those of hand-crafted neural networks. In this paper, we propose an Ecologically-Inspired GENetic approach for neural network structure search (EIGEN), that includes succession, mimicry and gene duplication. Specifically, we first use primary succession to rapidly evolve a community of poor initialized neural network structures into a more diverse community, followed by a secondary succession stage for fine-grained searching based on the networks from the primary succession. Extinction is applied in both stages to reduce computational cost. Mimicry is employed during the entire evolution process to help the inferior networks imitate the behavior of a superior network and gene duplication is utilized to duplicate the learned blocks of novel structures, both of which help to find the better network structures. Extensive experimental results show that our proposed approach can achieve the similar or better performance compared to the existing genetic approaches with dramatically reduced computation cost. For example, the network discovered by our approach on CIFAR-100 dataset achieves 78.1 hours, compared to 77.0

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/03/2018

Optimization Algorithm Inspired Deep Neural Network Structure Design

Deep neural networks have been one of the dominant machine learning appr...
research
11/05/2022

Efficient Cavity Searching for Gene Network of Influenza A Virus

High order structures (cavities and cliques) of the gene network of infl...
research
06/03/2018

An Aggressive Genetic Programming Approach for Searching Neural Network Structure Under Computational Constraints

Recently, there emerged revived interests of designing automatic program...
research
10/31/2018

Structure Learning of Deep Neural Networks with Q-Learning

Recently, with convolutional neural networks gaining significant achieve...
research
04/19/2021

Coarse-to-Fine Searching for Efficient Generative Adversarial Networks

This paper studies the neural architecture search (NAS) problem for deve...
research
10/31/2019

Scaling structural learning with NO-BEARS to infer causal transcriptome networks

Constructing gene regulatory networks is a critical step in revealing di...
research
03/03/2023

Bespoke: A Block-Level Neural Network Optimization Framework for Low-Cost Deployment

As deep learning models become popular, there is a lot of need for deplo...

Please sign up or login with your details

Forgot password? Click here to reset