Weight-sharing Supernet for Searching Specialized Acoustic Event Classification Networks Across Device Constraints

03/18/2023
by   Guan-Ting Lin, et al.
0

Acoustic Event Classification (AEC) has been widely used in devices such as smart speakers and mobile phones for home safety or accessibility support. As AEC models run on more and more devices with diverse computation resource constraints, it became increasingly expensive to develop models that are tuned to achieve optimal accuracy/computation trade-off for each given computation resource constraint. In this paper, we introduce a Once-For-All (OFA) Neural Architecture Search (NAS) framework for AEC. Specifically, we first train a weight-sharing supernet that supports different model architectures, followed by automatically searching for a model given specific computational resource constraints. Our experimental results showed that by just training once, the resulting model from NAS significantly outperforms both models trained individually from scratch and knowledge distillation (25.4 improvement). We also found that the benefit of weight-sharing supernet training of ultra-small models comes not only from searching but from optimization.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/31/2019

HM-NAS: Efficient Neural Architecture Search via Hierarchical Masking

The use of automatic methods, often referred to as Neural Architecture S...
research
02/16/2021

AlphaNet: Improved Training of Supernet with Alpha-Divergence

Weight-sharing neural architecture search (NAS) is an effective techniqu...
research
01/06/2020

Deeper Insights into Weight Sharing in Neural Architecture Search

With the success of deep neural networks, Neural Architecture Search (NA...
research
08/06/2021

AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing

Architecture performance predictors have been widely used in neural arch...
research
12/24/2019

BETANAS: BalancEd TrAining and selective drop for Neural Architecture Search

Automatic neural architecture search techniques are becoming increasingl...
research
09/27/2022

Towards Regression-Free Neural Networks for Diverse Compute Platforms

With the shift towards on-device deep learning, ensuring a consistent be...
research
03/17/2018

Multi-device, Multi-tenant Model Selection with GP-EI

Bayesian optimization is the core technique behind the emergence of Auto...

Please sign up or login with your details

Forgot password? Click here to reset