Robot Learning with Crash Constraints

10/16/2020
by   Alonso Marco, et al.
0

In the past decade, numerous machine learning algorithms have been shown to successfully learn optimal policies to control real robotic systems. However, it is not rare to encounter failing behaviors as the learning loop progresses. Specifically, in robot applications where failing is undesired but not catastrophic, many algorithms struggle with leveraging data obtained from failures. This is usually caused by (i) the failed experiment ending prematurely, or (ii) the acquired data being scarce or corrupted. Both complicate the design of proper reward functions to penalize failures. In this paper, we propose a framework that addresses those issues. We consider failing behaviors as those that violate a constraint and address the problem of "learning with crash constraints", where no data is obtained upon constraint violation. The no-data case is addressed by a novel GP model (GPCR) for the constraint that combines discrete events (failure/success) with continuous observations (only obtained upon success). We demonstrate the effectiveness of our framework on simulated benchmarks and on a real jumping quadruped, where the constraint boundary is unknown a priori. Experimental data is collected, by means of constrained Bayesian optimization, directly on the real robot. Our results outperform manual tuning and GPCR proves useful on estimating the constraint boundary.

READ FULL TEXT

page 1

page 7

research
07/24/2019

Classified Regression for Bayesian Optimization: Robot Learning with Unknown Penalties

Learning robot controllers by minimizing a black-box objective cost usin...
research
10/14/2021

VABO: Violation-Aware Bayesian Optimization for Closed-Loop Control Performance Optimization with Unmodeled Constraints

We study the problem of performance optimization of closed-loop control ...
research
03/22/2014

Bayesian Optimization with Unknown Constraints

Recent work on Bayesian optimization has shown its effectiveness in glob...
research
05/15/2020

Excursion Search for Constrained Bayesian Optimization under a Limited Budget of Failures

When learning to ride a bike, a child falls down a number of times befor...
research
01/28/2023

Violation-Aware Contextual Bayesian Optimization for Controller Performance Optimization with Unmodeled Constraints

We study the problem of performance optimization of closed-loop control ...
research
04/12/2023

Primal-Dual Contextual Bayesian Optimization for Control System Online Optimization with Time-Average Constraints

This paper studies the problem of online performance optimization of con...

Please sign up or login with your details

Forgot password? Click here to reset