Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL

06/24/2020
by   Lucas Zimmer, et al.
105

While early AutoML frameworks focused on optimizing traditional ML pipelines and their hyperparameters, a recent trend in AutoML is to focus on neural architecture search. In this paper, we introduce Auto-PyTorch, which brings the best of these two worlds together by jointly and robustly optimizing the architecture of networks and the training hyperparameters to enable fully automated deep learning (AutoDL). Auto-PyTorch achieves state-of-the-art performance on several tabular benchmarks by combining multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) and common baselines for tabular data. To thoroughly study our assumptions on how to design such an AutoDL system, we additionally introduce a new benchmark on learning curves for DNNs, dubbed LCBench, and run extensive ablation studies of the full Auto-PyTorch on typical AutoML benchmarks, eventually showing that Auto-PyTorch performs better than several state-of-the-art competitors on average.

READ FULL TEXT
research
05/11/2022

Efficient Automated Deep Learning for Time Series Forecasting

Recent years have witnessed tremendously improved efficiency of Automate...
research
08/19/2020

Auto-Surprise: An Automated Recommender-System (AutoRecSys) Library with Tree of Parzens Estimator (TPE) Optimization

We introduce Auto-Surprise, an Automated Recommender System library. Aut...
research
05/27/2022

Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture

Physics-informed neural networks (PINNs) are revolutionizing science and...
research
12/10/2020

Auto-MVCNN: Neural Architecture Search for Multi-view 3D Shape Recognition

In 3D shape recognition, multi-view based methods leverage human's persp...
research
02/18/2019

Evolutionary Neural AutoML for Deep Learning

Deep neural networks (DNNs) have produced state-of-the-art results in ma...
research
11/05/2021

NAS-Bench-x11 and the Power of Learning Curves

While early research in neural architecture search (NAS) required extrem...
research
07/08/2020

Auto-Sklearn 2.0: The Next Generation

Automated Machine Learning, which supports practitioners and researchers...

Please sign up or login with your details

Forgot password? Click here to reset