A Comprehensive Survey on Hardware-Aware Neural Architecture Search

01/22/2021
by   Hadjer Benmeziane, et al.
0

Neural Architecture Search (NAS) methods have been growing in popularity. These techniques have been fundamental to automate and speed up the time consuming and error-prone process of synthesizing novel Deep Learning (DL) architectures. NAS has been extensively studied in the past few years. Arguably their most significant impact has been in image classification and object detection tasks where the state of the art results have been obtained. Despite the significant success achieved to date, applying NAS to real-world problems still poses significant challenges and is not widely practical. In general, the synthesized Convolution Neural Network (CNN) architectures are too complex to be deployed in resource-limited platforms, such as IoT, mobile, and embedded systems. One solution growing in popularity is to use multi-objective optimization algorithms in the NAS search strategy by taking into account execution latency, energy consumption, memory footprint, etc. This kind of NAS, called hardware-aware NAS (HW-NAS), makes searching the most efficient architecture more complicated and opens several questions. In this survey, we provide a detailed review of existing HW-NAS research and categorize them according to four key dimensions: the search space, the search strategy, the acceleration technique, and the hardware cost estimation strategies. We further discuss the challenges and limitations of existing approaches and potential future directions. This is the first survey paper focusing on hardware-aware NAS. We hope it serves as a valuable reference for the various techniques and algorithms discussed and paves the road for future research towards hardware-aware NAS.

READ FULL TEXT

page 1

page 10

page 30

research
03/08/2023

HyT-NAS: Hybrid Transformers Neural Architecture Search for Edge Devices

Vision Transformers have enabled recent attention-based Deep Learning (D...
research
10/12/2022

Energy Consumption-Aware Tabular Benchmarks for Neural Architecture Search

The demand for large-scale computational resources for Neural Architectu...
research
05/05/2023

Neural Architecture Search for Intel Movidius VPU

Hardware-aware Neural Architecture Search (NAS) technologies have been p...
research
08/25/2020

A Survey on Evolutionary Neural Architecture Search

Deep Neural Networks (DNNs) have achieved great success in many applicat...
research
08/02/2019

AutoML: A Survey of the State-of-the-Art

Deep learning has penetrated all aspects of our lives and brought us gre...
research
08/29/2018

Searching Toward Pareto-Optimal Device-Aware Neural Architectures

Recent breakthroughs in Neural Architectural Search (NAS) have achieved ...
research
06/01/2020

A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions

Deep learning has made major breakthroughs and progress in many fields. ...

Please sign up or login with your details

Forgot password? Click here to reset