On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learning

by   Baohe Zhang, et al.

Model-based Reinforcement Learning (MBRL) is a promising framework for learning control in a data-efficient manner. MBRL algorithms can be fairly complex due to the separate dynamics modeling and the subsequent planning algorithm, and as a result, they often possess tens of hyperparameters and architectural choices. For this reason, MBRL typically requires significant human expertise before it can be applied to new problems and domains. To alleviate this problem, we propose to use automatic hyperparameter optimization (HPO). We demonstrate that this problem can be tackled effectively with automated HPO, which we demonstrate to yield significantly improved performance compared to human experts. In addition, we show that tuning of several MBRL hyperparameters dynamically, i.e. during the training itself, further improves the performance compared to using static hyperparameters which are kept fixed for the whole training. Finally, our experiments provide valuable insights into the effects of several hyperparameters, such as plan horizon or learning rate and their influence on the stability of training and resulting rewards.



page 6

page 12


To tune or not to tune? An Approach for Recommending Important Hyperparameters

Novel technologies in automated machine learning ease the complexity of ...

Hyp-RL : Hyperparameter Optimization by Reinforcement Learning

Hyperparameter tuning is an omnipresent problem in machine learning as i...

Efficient Hyperparameter Tuning with Dynamic Accuracy Derivative-Free Optimization

Many machine learning solutions are framed as optimization problems whic...

Hyperparameter Optimisation with Early Termination of Poor Performers

It is typical for a machine learning system to have numerous hyperparame...

On Effective Scheduling of Model-based Reinforcement Learning

Model-based reinforcement learning has attracted wide attention due to i...

Population Based Training of Neural Networks

Neural networks dominate the modern machine learning landscape, but thei...

How much progress have we made in neural network training? A New Evaluation Protocol for Benchmarking Optimizers

Many optimizers have been proposed for training deep neural networks, an...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.