Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL

by   Lucas Zimmer, et al.

While early AutoML frameworks focused on optimizing traditional ML pipelines and their hyperparameters, a recent trend in AutoML is to focus on neural architecture search. In this paper, we introduce Auto-PyTorch, which brings the best of these two worlds together by jointly and robustly optimizing the architecture of networks and the training hyperparameters to enable fully automated deep learning (AutoDL). Auto-PyTorch achieves state-of-the-art performance on several tabular benchmarks by combining multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) and common baselines for tabular data. To thoroughly study our assumptions on how to design such an AutoDL system, we additionally introduce a new benchmark on learning curves for DNNs, dubbed LCBench, and run extensive ablation studies of the full Auto-PyTorch on typical AutoML benchmarks, eventually showing that Auto-PyTorch performs better than several state-of-the-art competitors on average.



page 7


Efficient Automated Deep Learning for Time Series Forecasting

Recent years have witnessed tremendously improved efficiency of Automate...

Auto-Surprise: An Automated Recommender-System (AutoRecSys) Library with Tree of Parzens Estimator (TPE) Optimization

We introduce Auto-Surprise, an Automated Recommender System library. Aut...

Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture

Physics-informed neural networks (PINNs) are revolutionizing science and...

Auto-MVCNN: Neural Architecture Search for Multi-view 3D Shape Recognition

In 3D shape recognition, multi-view based methods leverage human's persp...

Evolutionary Neural AutoML for Deep Learning

Deep neural networks (DNNs) have produced state-of-the-art results in ma...

Auto-Sklearn 2.0: The Next Generation

Automated Machine Learning, which supports practitioners and researchers...

SplitNets: Designing Neural Architectures for Efficient Distributed Computing on Head-Mounted Systems

We design deep neural networks (DNNs) and corresponding networks' splitt...

Code Repositories


Automatic architecture search and hyperparameter optimization for PyTorch

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.