Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search

02/28/2021
by   Guoyang Xie, et al.
0

Due to limited computational cost and energy consumption, most neural network models deployed in mobile devices are tiny. However, tiny neural networks are commonly very vulnerable to attacks. Current research has proved that larger model size can improve robustness, but little research focuses on how to enhance the robustness of tiny neural networks. Our work focuses on how to improve the robustness of tiny neural networks without seriously deteriorating of clean accuracy under mobile-level resources. To this end, we propose a multi-objective oneshot network architecture search (NAS) algorithm to obtain the best trade-off networks in terms of the adversarial accuracy, the clean accuracy and the model size. Specifically, we design a novel search space based on new tiny blocks and channels to balance model size and adversarial performance. Moreover, since the supernet significantly affects the performance of subnets in our NAS algorithm, we reveal the insights into how the supernet helps to obtain the best subnet under white-box adversarial attacks. Concretely, we explore a new adversarial training paradigm by analyzing the adversarial transferability, the width of the supernet and the difference between training the subnets from scratch and fine-tuning. Finally, we make a statistical analysis for the layer-wise combination of certain blocks and channels on the first non-dominated front, which can serve as a guideline to design tiny neural network architectures for the resilience of adversarial perturbations.

READ FULL TEXT

page 1

page 3

page 5

research
11/19/2020

Effective, Efficient and Robust Neural Architecture Search

Recent advances in adversarial attacks show the vulnerability of deep ne...
research
10/12/2022

Energy Consumption-Aware Tabular Benchmarks for Neural Architecture Search

The demand for large-scale computational resources for Neural Architectu...
research
12/16/2021

Learning Interpretable Models Through Multi-Objective Neural Architecture Search

Monumental advances in deep learning have led to unprecedented achieveme...
research
02/03/2021

Learning Diverse-Structured Networks for Adversarial Robustness

In adversarial training (AT), the main focus has been the objective and ...
research
06/11/2023

Neural Architecture Design and Robustness: A Dataset

Deep learning models have proven to be successful in a wide range of mac...
research
08/16/2021

Neural Architecture Dilation for Adversarial Robustness

With the tremendous advances in the architecture and scale of convolutio...
research
08/03/2020

Anti-Bandit Neural Architecture Search for Model Defense

Deep convolutional neural networks (DCNNs) have dominated as the best pe...

Please sign up or login with your details

Forgot password? Click here to reset