StressedNets: Efficient Feature Representations via Stress-induced Evolutionary Synthesis of Deep Neural Networks

by   Mohammad Javad Shafiee, et al.

The computational complexity of leveraging deep neural networks for extracting deep feature representations is a significant barrier to its widespread adoption, particularly for use in embedded devices. One particularly promising strategy to addressing the complexity issue is the notion of evolutionary synthesis of deep neural networks, which was demonstrated to successfully produce highly efficient deep neural networks while retaining modeling performance. Here, we further extend upon the evolutionary synthesis strategy for achieving efficient feature extraction via the introduction of a stress-induced evolutionary synthesis framework, where stress signals are imposed upon the synapses of a deep neural network during training to induce stress and steer the synthesis process towards the production of more efficient deep neural networks over successive generations and improved model fidelity at a greater efficiency. The proposed stress-induced evolutionary synthesis approach is evaluated on a variety of different deep neural network architectures (LeNet5, AlexNet, and YOLOv2) on different tasks (object classification and object detection) to synthesize efficient StressedNets over multiple generations. Experimental results demonstrate the efficacy of the proposed framework to synthesize StressedNets with significant improvement in network architecture efficiency (e.g., 40x for AlexNet and 33x for YOLOv2) and speed improvements (e.g., 5.5x inference speed-up for YOLOv2 on an Nvidia Tegra X1 mobile processor).


page 8

page 9

page 10


The Mating Rituals of Deep Neural Networks: Learning Compact Feature Representations through Sexual Evolutionary Synthesis

Evolutionary deep intelligence was recently proposed as a method for ach...

Complexity for deep neural networks and other characteristics of deep feature representations

We define a notion of complexity, motivated by considerations of circuit...

Exploring the Imposition of Synaptic Precision Restrictions For Evolutionary Synthesis of Deep Neural Networks

A key contributing factor to incredible success of deep neural networks ...

SquishedNets: Squishing SqueezeNet further for edge device scenarios via deep evolutionary synthesis

While deep neural networks have been shown in recent years to outperform...

FermiNets: Learning generative machines to generate efficient neural networks via generative synthesis

The tremendous potential exhibited by deep learning is often offset by a...

Evolution in Groups: A deeper look at synaptic cluster driven evolution of deep neural networks

A promising paradigm for achieving highly efficient deep neural networks...

Assessing Architectural Similarity in Populations of Deep Neural Networks

Evolutionary deep intelligence has recently shown great promise for prod...

Please sign up or login with your details

Forgot password? Click here to reset