DeepAI AI Chat
Log In Sign Up

BayesFT: Bayesian Optimization for Fault Tolerant Neural Network Architecture

by   Nanyang Ye, et al.
University of Cambridge
Shanghai Jiao Tong University

To deploy deep learning algorithms on resource-limited scenarios, an emerging device-resistive random access memory (ReRAM) has been regarded as promising via analog computing. However, the practicability of ReRAM is primarily limited due to the weight drifting of ReRAM neural networks due to multi-factor reasons, including manufacturing, thermal noises, and etc. In this paper, we propose a novel Bayesian optimization method for fault tolerant neural network architecture (BayesFT). For neural architecture search space design, instead of conducting neural architecture search on the whole feasible neural architecture search space, we first systematically explore the weight drifting tolerance of different neural network components, such as dropout, normalization, number of layers, and activation functions in which dropout is found to be able to improve the neural network robustness to weight drifting. Based on our analysis, we propose an efficient search space by only searching for dropout rates for each layer. Then, we use Bayesian optimization to search for the optimal neural architecture robust to weight drifting. Empirical experiments demonstrate that our algorithmic framework has outperformed the state-of-the-art methods by up to 10 times on various tasks, such as image classification and object detection.


page 2

page 6


Efficient Neural Architecture Search with Network Morphism

While neural architecture search (NAS) has drawn increasing attention fo...

Deep Neural Architecture Search with Deep Graph Bayesian Optimization

Bayesian optimization (BO) is an effective method of finding the global ...

Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detection

Recently, Neural Architecture Search has achieved great success in large...

Unified Probabilistic Neural Architecture and Weight Ensembling Improves Model Robustness

Robust machine learning models with accurately calibrated uncertainties ...

GNAS: A Generalized Neural Network Architecture Search Framework

In practice, the problems encountered in training NAS (Neural Architectu...

An Efficient End-to-End 3D Model Reconstruction based on Neural Architecture Search

Using neural networks to represent 3D objects has become popular. Howeve...

Fitness Landscape Footprint: A Framework to Compare Neural Architecture Search Problems

Neural architecture search is a promising area of research dedicated to ...