AttoNets: Compact and Efficient Deep Neural Networks for the Edge via Human-Machine Collaborative Design

03/18/2019
by   Alexander Wong, et al.
0

While deep neural networks have achieved state-of-the-art performance across a large number of complex tasks, it remains a big challenge to deploy such networks for practical, on-device edge scenarios such as on mobile devices, consumer devices, drones, and vehicles. In this study, we take a deeper exploration into a human-machine collaborative design approach for creating highly efficient deep neural networks through a synergy between principled network design prototyping and machine-driven design exploration. The efficacy of human-machine collaborative design is demonstrated through the creation of AttoNets, a family of highly efficient deep neural networks for on-device edge deep learning. Each AttoNet possesses a human-specified network-level macro-architecture comprising of custom modules with unique machine-designed module-level macro-architecture and micro-architecture designs, all driven by human-specified design requirements. Experimental results for the task of object recognition showed that the AttoNets created via human-machine collaborative design has significantly fewer parameters and computational costs than state-of-the-art networks designed for efficiency while achieving noticeably higher accuracy (with the smallest AttoNet achieving 1.8 accuracy while requiring 10x fewer multiply-add operations and parameters than MobileNet-V1). Furthermore, the efficacy of the AttoNets is demonstrated for the task of instance-level object segmentation and object detection, where an AttoNet-based Mask R-CNN network was constructed with significantly fewer parameters and computational costs ( 5x fewer multiply-add operations and 2x fewer parameters) than a ResNet-50 based Mask R-CNN network.

READ FULL TEXT
research
10/18/2018

EdgeSpeechNets: Highly Efficient Deep Neural Networks for Speech Recognition on the Edge

Despite showing state-of-the-art performance, deep learning for speech r...
research
09/17/2018

FermiNets: Learning generative machines to generate efficient neural networks via generative synthesis

The tremendous potential exhibited by deep learning is often offset by a...
research
09/12/2019

Human-Machine Collaborative Design for Accelerated Design of Compact Deep Neural Networks for Autonomous Driving

An effective deep learning development process is critical for widesprea...
research
05/20/2019

Enabling Computer Vision Driven Assistive Devices for the Visually Impaired via Micro-architecture Design Exploration

Recent improvements in object detection have shown potential to aid in t...
research
06/29/2020

EmotionNet Nano: An Efficient Deep Convolutional Neural Network Design for Real-time Facial Expression Recognition

While recent advances in deep learning have led to significant improveme...
research
09/30/2020

AttendNets: Tiny Deep Image Recognition Neural Networks for the Edge via Visual Attention Condensers

While significant advances in deep learning has resulted in state-of-the...
research
01/16/2021

NNStreamer: Efficient and Agile Development of On-Device AI Systems

We propose NNStreamer, a software system that handles neural networks as...

Please sign up or login with your details

Forgot password? Click here to reset