Log In Sign Up

Leveraging Forward Model Prediction Error for Learning Control

by   Sarah Bechtle, et al.

Learning for model based control can be sample-efficient and generalize well, however successfully learning models and controllers that represent the problem at hand can be challenging for complex tasks. Using inaccurate models for learning can lead to sub-optimal solutions, that are unlikely to perform well in practice. In this work, we present a learning approach which iterates between model learning and data collection and leverages forward model prediction error for learning control. We show how using the controller's prediction as input to a forward model can create a differentiable connection between the controller and the model, allowing us to formulate a loss in the state space. This lets us include forward model prediction error during controller learning and we show that this creates a loss objective that significantly improves learning on different motor control tasks. We provide empirical and theoretical results that show the benefits of our method and present evaluations in simulation for learning control on a 7 DoF manipulator and an underactuated 12 DoF quadruped. We show that our approach successfully learns controllers for challenging motor control tasks involving contact switching.


page 1

page 2

page 3

page 4


Learning Task Agnostic Sufficiently Accurate Models

For complex real-world systems, designing controllers are a difficult ta...

Lyapunov Design for Robust and Efficient Robotic Reinforcement Learning

Recent advances in the reinforcement learning (RL) literature have enabl...

Synthesizing Neural Network Controllers with Probabilistic Model based Reinforcement Learning

We present an algorithm for rapidly learning controllers for robotics sy...

Learning Reactive and Predictive Differentiable Controllers for Switching Linear Dynamical Models

Humans leverage the dynamics of the environment and their own bodies to ...

Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contexts

We explore the reinforcement learning approach to designing controllers ...

Thrust Direction Control of an Underactuated Oscillating Swimming Robot

The Modboat is an autonomous surface robot that turns the oscillation of...