Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL

06/24/2020
by   Lucas Zimmer, et al.
105

While early AutoML frameworks focused on optimizing traditional ML pipelines and their hyperparameters, a recent trend in AutoML is to focus on neural architecture search. In this paper, we introduce Auto-PyTorch, which brings the best of these two worlds together by jointly and robustly optimizing the architecture of networks and the training hyperparameters to enable fully automated deep learning (AutoDL). Auto-PyTorch achieves state-of-the-art performance on several tabular benchmarks by combining multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) and common baselines for tabular data. To thoroughly study our assumptions on how to design such an AutoDL system, we additionally introduce a new benchmark on learning curves for DNNs, dubbed LCBench, and run extensive ablation studies of the full Auto-PyTorch on typical AutoML benchmarks, eventually showing that Auto-PyTorch performs better than several state-of-the-art competitors on average.

READ FULL TEXT

Authors

page 7

05/11/2022

Efficient Automated Deep Learning for Time Series Forecasting

Recent years have witnessed tremendously improved efficiency of Automate...
08/19/2020

Auto-Surprise: An Automated Recommender-System (AutoRecSys) Library with Tree of Parzens Estimator (TPE) Optimization

We introduce Auto-Surprise, an Automated Recommender System library. Aut...
05/27/2022

Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture

Physics-informed neural networks (PINNs) are revolutionizing science and...
12/10/2020

Auto-MVCNN: Neural Architecture Search for Multi-view 3D Shape Recognition

In 3D shape recognition, multi-view based methods leverage human's persp...
02/18/2019

Evolutionary Neural AutoML for Deep Learning

Deep neural networks (DNNs) have produced state-of-the-art results in ma...
07/08/2020

Auto-Sklearn 2.0: The Next Generation

Automated Machine Learning, which supports practitioners and researchers...
04/10/2022

SplitNets: Designing Neural Architectures for Efficient Distributed Computing on Head-Mounted Systems

We design deep neural networks (DNNs) and corresponding networks' splitt...

Code Repositories

Auto-PyTorch

Automatic architecture search and hyperparameter optimization for PyTorch


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.