DeepAI AI Chat
Log In Sign Up

Deep-n-Cheap: An Automated Search Framework for Low Complexity Deep Learning

03/27/2020
by   Sourya Dey, et al.
University of Southern California
0

We present Deep-n-Cheap – an open-source AutoML framework to search for deep learning models. This search includes both architecture and training hyperparameters, and supports convolutional neural networks and multi-layer perceptrons. Our framework is targeted for deployment on both benchmark and custom datasets, and as a result, offers a greater degree of search space customizability as compared to a more limited search over only pre-existing models from literature. We also introduce the technique of 'search transfer', which demonstrates the generalization capabilities of the models found by our framework to multiple datasets. Deep-n-Cheap includes a user-customizable complexity penalty which trades off performance with training time or number of parameters. Specifically, our framework results in models offering performance comparable to state-of-the-art while taking 1-2 orders of magnitude less time to train than models from other AutoML and model search frameworks. Additionally, this work investigates and develops various insights regarding the search process. In particular, we show the superiority of a greedy strategy and justify our choice of Bayesian optimization as the primary search methodology over random / grid search.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/25/2016

CMA-ES for Hyperparameter Optimization of Deep Neural Networks

Hyperparameters of deep neural networks are often optimized by grid sear...
04/28/2017

DeepArchitect: Automatically Designing and Training Deep Architectures

In deep learning, performance is strongly affected by the choice of arch...
02/27/2023

An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters

In this paper, we propose an algorithmic framework to automatically gene...
03/13/2020

AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data

We introduce AutoGluon-Tabular, an open-source AutoML framework that req...
07/03/2019

HyperNOMAD: Hyperparameter optimization of deep neural networks using mesh adaptive direct search

The performance of deep neural networks is highly sensitive to the choic...
03/30/2022

AdaGrid: Adaptive Grid Search for Link Prediction Training Objective

One of the most important factors that contribute to the success of a ma...