DeepAI
Log In Sign Up

MOFA: Modular Factorial Design for Hyperparameter Optimization

11/18/2020
by   Bo Xiong, et al.
0

Automated hyperparameter optimization (HPO) has shown great power in many machine learning applications. While existing methods suffer from model selection, parallelism, or sample efficiency, this paper presents a new HPO method, MOdular FActorial Design (MOFA), to address these issues simultaneously. The major idea is to use techniques from Experimental Designs to improve sample efficiency of model-free methods. Particularly, MOFA runs with four modules in each iteration: (1) an Orthogonal Latin Hypercube (OLH)-based sampler preserving both univariate projection uniformity and orthogonality; (2) a highly parallelized evaluator; (3) a transformer to collapse the OLH performance table into a specified Fractional Factorial Design–Orthogonal Array (OA); (4) an analyzer including Factorial Performance Analysis and Factorial Importance Analysis to narrow down the search space. We theoretically and empirically show that MOFA has great advantages over existing model-based and model-free methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/16/2019

Weighted Sampling for Combined Model Selection and Hyperparameter Tuning

The combined algorithm selection and hyperparameter tuning (CASH) proble...
11/18/2018

Policy Optimization with Model-based Explorations

Model-free reinforcement learning methods such as the Proximal Policy Op...
04/30/2021

A Comparison of Model-Free Phase I Dose Escalation Designs for Dual-Agent Combination Therapies

It is increasingly common for therapies in oncology to be given in combi...
03/19/2018

Simple random search provides a competitive approach to reinforcement learning

A common belief in model-free reinforcement learning is that methods bas...
02/21/2017

Sample Efficient Policy Search for Optimal Stopping Domains

Optimal stopping problems consider the question of deciding when to stop...
02/29/2016

Modular Tracking Framework: A Unified Approach to Registration based Tracking

This paper presents a modular, extensible and highly efficient open sour...