Neural Architecture Search over Decentralized Data

02/15/2020
by   Mengwei Xu, et al.
0

To preserve user privacy while enabling mobile intelligence, techniques have been proposed to train deep neural networks on decentralized data. However, training over decentralized data makes the design of neural architecture quite difficult as it already was. Such difficulty is further amplified when designing and deploying different neural architectures for heterogeneous mobile platforms. In this work, we propose an automatic neural architecture search into the decentralized training, as a new DNN training paradigm called Federated Neural Architecture Search, namely federated NAS. To deal with the primary challenge of limited on-client computational and communication resources, we present FedNAS, a highly optimized framework for efficient federated NAS. FedNAS fully exploits the key opportunity of insufficient model candidate re-training during the architecture search process, and incorporates three key optimizations: parallel candidates training on partial clients, early dropping candidates with inferior performance, and dynamic round numbers. Tested on large-scale datasets and typical CNN architectures, FedNAS achieves comparable model accuracy as state-of-the-art NAS algorithm that trains models with centralized data, and also reduces the client cost by up to two orders of magnitude compared to a straightforward design of federated NAS.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/06/2020

FDNAS: Improving Data Privacy and Model Diversity in AutoML

To prevent the leakage of private information while enabling automated m...
research
12/17/2019

Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data

This paper investigates the intriguing question of whether we can create...
research
06/06/2020

Conditional Neural Architecture Search

Designing resource-efficient Deep Neural Networks (DNNs) is critical to ...
research
10/16/2022

FAQS: Communication-efficient Federate DNN Architecture and Quantization Co-Search for personalized Hardware-aware Preferences

Due to user privacy and regulatory restrictions, federate learning (FL) ...
research
12/30/2019

Neural Architecture Search on Acoustic Scene Classification

Convolutional neural networks are widely adopted in Acoustic Scene Class...
research
05/19/2023

ALT: An Automatic System for Long Tail Scenario Modeling

In this paper, we consider the problem of long tail scenario modeling wi...
research
03/29/2022

AutoCoMet: Smart Neural Architecture Search via Co-Regulated Shaping Reinforcement

Designing suitable deep model architectures, for AI-driven on-device app...

Please sign up or login with your details

Forgot password? Click here to reset