Log In Sign Up

MOFA: Modular Factorial Design for Hyperparameter Optimization

by   Bo Xiong, et al.

Automated hyperparameter optimization (HPO) has shown great power in many machine learning applications. While existing methods suffer from model selection, parallelism, or sample efficiency, this paper presents a new HPO method, MOdular FActorial Design (MOFA), to address these issues simultaneously. The major idea is to use techniques from Experimental Designs to improve sample efficiency of model-free methods. Particularly, MOFA runs with four modules in each iteration: (1) an Orthogonal Latin Hypercube (OLH)-based sampler preserving both univariate projection uniformity and orthogonality; (2) a highly parallelized evaluator; (3) a transformer to collapse the OLH performance table into a specified Fractional Factorial Design–Orthogonal Array (OA); (4) an analyzer including Factorial Performance Analysis and Factorial Importance Analysis to narrow down the search space. We theoretically and empirically show that MOFA has great advantages over existing model-based and model-free methods.


page 1

page 2

page 3

page 4


Weighted Sampling for Combined Model Selection and Hyperparameter Tuning

The combined algorithm selection and hyperparameter tuning (CASH) proble...

Policy Optimization with Model-based Explorations

Model-free reinforcement learning methods such as the Proximal Policy Op...

A Comparison of Model-Free Phase I Dose Escalation Designs for Dual-Agent Combination Therapies

It is increasingly common for therapies in oncology to be given in combi...

Simple random search provides a competitive approach to reinforcement learning

A common belief in model-free reinforcement learning is that methods bas...

Sample Efficient Policy Search for Optimal Stopping Domains

Optimal stopping problems consider the question of deciding when to stop...

Modular Tracking Framework: A Unified Approach to Registration based Tracking

This paper presents a modular, extensible and highly efficient open sour...