Classified Regression for Bayesian Optimization: Robot Learning with Unknown Penalties

07/24/2019
by   Alonso Marco, et al.
3

Learning robot controllers by minimizing a black-box objective cost using Bayesian optimization (BO) can be time-consuming and challenging. It is very often the case that some roll-outs result in failure behaviors, causing premature experiment detention. In such cases, the designer is forced to decide on heuristic cost penalties because the acquired data is often scarce, or not comparable with that of the stable policies. To overcome this, we propose a Bayesian model that captures exactly what we know about the cost of unstable controllers prior to data collection: Nothing, except that it should be a somewhat large number. The resulting Bayesian model, approximated with a Gaussian process, predicts high cost values in regions where failures are likely to occur. In this way, the model guides the BO exploration toward regions of stability. We demonstrate the benefits of the proposed model in several illustrative and statistical synthetic benchmarks, and also in experiments on a real robotic platform. In addition, we propose and experimentally validate a new BO method to account for unknown constraints. Such method is an extension of Max-Value Entropy Search, a recent information-theoretic method, to solve unconstrained global optimization problems.

READ FULL TEXT

page 23

page 24

page 28

page 29

research
02/18/2015

Predictive Entropy Search for Bayesian Optimization with Unknown Constraints

Unknown constraints arise in many types of expensive black-box optimizat...
research
10/16/2020

Robot Learning with Crash Constraints

In the past decade, numerous machine learning algorithms have been shown...
research
10/15/2019

Constrained Bayesian Optimization with Max-Value Entropy Search

Bayesian optimization (BO) is a model-based approach to sequentially opt...
research
01/24/2019

Multi-fidelity Bayesian Optimization with Max-value Entropy Search

Bayesian optimization (BO) is an effective tool for black-box optimizati...
research
03/06/2017

Max-value Entropy Search for Efficient Bayesian Optimization

Entropy Search (ES) and Predictive Entropy Search (PES) are popular and ...
research
03/07/2016

Unscented Bayesian Optimization for Safe Robot Grasping

We address the robot grasp optimization problem of unknown objects consi...
research
05/15/2020

Excursion Search for Constrained Bayesian Optimization under a Limited Budget of Failures

When learning to ride a bike, a child falls down a number of times befor...

Please sign up or login with your details

Forgot password? Click here to reset