DeepAI AI Chat
Log In Sign Up

Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search

by   Arber Zela, et al.
University of Freiburg

While existing work on neural architecture search (NAS) tunes hyperparameters in a separate post-processing step, we demonstrate that architectural choices and other hyperparameter settings interact in a way that can render this separation suboptimal. Likewise, we demonstrate that the common practice of using very few epochs during the main NAS and much larger numbers of epochs during a post-processing step is inefficient due to little correlation in the relative rankings for these two training regimes. To combat both of these problems, we propose to use a recent combination of Bayesian optimization and Hyperband for efficient joint neural architecture and hyperparameter search.


Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization

Neural architecture search (NAS) and hyperparameter optimization (HPO) m...

An Asymptotically Optimal Multi-Armed Bandit Algorithm and Hyperparameter Optimization

The evaluation of hyperparameters, neural architectures, or data augment...

BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models

Neural architecture search (NAS) has shown promising results discovering...

Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture

Physics-informed neural networks (PINNs) are revolutionizing science and...

Automated Robustness with Adversarial Training as a Post-Processing Step

Adversarial training is a computationally expensive task and hence searc...

HANF: Hyperparameter And Neural Architecture Search in Federated Learning

Automated machine learning (AutoML) is an important step to make machine...

Revisiting the Train Loss: an Efficient Performance Estimator for Neural Architecture Search

Reliable yet efficient evaluation of generalisation performance of a pro...