Unified Probabilistic Neural Architecture and Weight Ensembling Improves Model Robustness

10/08/2022
by   Sumegha Premchandar, et al.
0

Robust machine learning models with accurately calibrated uncertainties are crucial for safety-critical applications. Probabilistic machine learning and especially the Bayesian formalism provide a systematic framework to incorporate robustness through the distributional estimates and reason about uncertainty. Recent works have shown that approximate inference approaches that take the weight space uncertainty of neural networks to generate ensemble prediction are the state-of-the-art. However, architecture choices have mostly been ad hoc, which essentially ignores the epistemic uncertainty from the architecture space. To this end, we propose a Unified probabilistic architecture and weight ensembling Neural Architecture Search (UraeNAS) that leverages advances in probabilistic neural architecture search and approximate Bayesian inference to generate ensembles form the joint distribution of neural network architectures and weights. The proposed approach showed a significant improvement both with in-distribution (0.86 out-of-distribution (2.43 baseline deterministic approach.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/26/2021

AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification

Deep neural networks are powerful predictors for a variety of tasks. How...
research
06/11/2020

NADS: Neural Architecture Distribution Search for Uncertainty Awareness

Machine learning (ML) systems often encounter Out-of-Distribution (OoD) ...
research
09/30/2022

BayesFT: Bayesian Optimization for Fault Tolerant Neural Network Architecture

To deploy deep learning algorithms on resource-limited scenarios, an eme...
research
06/23/2021

Bayesian Deep Learning Hyperparameter Search for Robust Function Mapping to Polynomials with Noise

Advances in neural architecture search, as well as explainability and in...
research
03/04/2022

WPNAS: Neural Architecture Search by jointly using Weight Sharing and Predictor

Weight sharing based and predictor based methods are two major types of ...
research
10/08/2021

Pathologies in priors and inference for Bayesian transformers

In recent years, the transformer has established itself as a workhorse i...
research
02/06/2020

Variational Depth Search in ResNets

One-shot neural architecture search allows joint learning of weights and...

Please sign up or login with your details

Forgot password? Click here to reset