HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search

02/23/2021
by   Niv Nayman, et al.
0

Realistic use of neural networks often requires adhering to multiple constraints on latency, energy and memory among others. A popular approach to find fitting networks is through constrained Neural Architecture Search (NAS), however, previous methods enforce the constraint only softly. Therefore, the resulting networks do not exactly adhere to the resource constraint and their accuracy is harmed. In this work we resolve this by introducing Hard Constrained diffeRentiable NAS (HardCoRe-NAS), that is based on an accurate formulation of the expected resource requirement and a scalable search method that satisfies the hard constraint throughout the search. Our experiments show that HardCoRe-NAS generates state-of-the-art architectures, surpassing other NAS methods, while strictly satisfying the hard resource constraints without any tuning required.

READ FULL TEXT
research
08/30/2022

You Only Search Once: On Lightweight Differentiable Architecture Search for Resource-Constrained Embedded Platforms

Benefiting from the search efficiency, differentiable neural architectur...
research
07/07/2020

GOLD-NAS: Gradual, One-Level, Differentiable

There has been a large literature of neural architecture search, but mos...
research
04/04/2023

Data Aware Neural Architecture Search

Neural Architecture Search (NAS) is a popular tool for automatically gen...
research
06/12/2018

Resource-Efficient Neural Architect

Neural Architecture Search (NAS) is a laborious process. Prior work on a...
research
04/15/2022

Resource-Constrained Neural Architecture Search on Tabular Datasets

The best neural architecture for a given machine learning problem depend...
research
06/01/2022

Multi-Complexity-Loss DNAS for Energy-Efficient and Memory-Constrained Deep Neural Networks

Neural Architecture Search (NAS) is increasingly popular to automaticall...
research
10/21/2020

MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers

Executing machine learning workloads locally on resource constrained mic...

Please sign up or login with your details

Forgot password? Click here to reset