Hardware/Software Co-Exploration of Neural Architectures

07/06/2019
by   Weiwen Jiang, et al.
9

We propose a novel hardware and software co-exploration framework for efficient neural architecture search (NAS). Different from existing hardware-aware NAS which assumes a fixed hardware design and explores the neural architecture search space only, our framework simultaneously explores both the architecture search space and the hardware design space to identify the best neural architecture and hardware pairs that maximize both test accuracy and hardware efficiency. Such a practice greatly opens up the design freedom and pushes forward the Pareto frontier between hardware efficiency and test accuracy for better design tradeoffs. The framework iteratively performs a two-level (fast and slow) exploration. Without lengthy training, the fast exploration can effectively fine-tune hyperparameters and prune inferior architectures in terms of hardware specifications, which significantly accelerates the NAS process. Then, the slow exploration trains candidates on a validation set and updates a controller using the reinforcement learning to maximize the expected accuracy together with the hardware efficiency. Experiments on ImageNet show that our co-exploration NAS can find the neural architectures and associated hardware design with the same accuracy, 35.24 higher throughput, 54.05 time, compared with the state-of-the-art hardware-aware NAS.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 8

research
01/31/2019

Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search

A fundamental question lies in almost every application of deep neural n...
research
10/31/2019

Device-Circuit-Architecture Co-Exploration for Computing-in-Memory Neural Accelerators

Co-exploration of neural architectures and hardware design is promising ...
research
10/31/2019

On Neural Architecture Search for Resource-Constrained Hardware Platforms

In the recent past, the success of Neural Architecture Search (NAS) has ...
research
02/10/2020

Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks

Neural Architecture Search (NAS) has demonstrated its power on various A...
research
05/05/2023

Neural Architecture Search for Intel Movidius VPU

Hardware-aware Neural Architecture Search (NAS) technologies have been p...
research
12/16/2020

Distilling Optimal Neural Networks: Rapid Search in Diverse Spaces

This work presents DONNA (Distilling Optimal Neural Network Architecture...
research
02/11/2020

Best of Both Worlds: AutoML Codesign of a CNN and its Hardware Accelerator

Neural architecture search (NAS) has been very successful at outperformi...

Please sign up or login with your details

Forgot password? Click here to reset