Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robot

03/06/2020
by   Napat Karnchanachari, et al.
0

Model Predictive Control (MPC) is a powerful control technique that handles constraints, takes the system's dynamics into account, and optimizes for a given cost function. In practice, however, it often requires an expert to craft and tune this cost function and find trade-offs between different state penalties to satisfy simple high level objectives. In this paper, we use Reinforcement Learning and in particular value learning to approximate the value function given only high level objectives, which can be sparse and binary. Building upon previous works, we present improvements that allowed us to successfully deploy the method on a real world unmanned ground vehicle. Our experiments show that our method can learn the cost function from scratch and without human intervention, while reaching a performance level similar to that of an expert-tuned MPC. We perform a quantitative comparison of these methods with standard MPC approaches both in simulation and on the real robot.

READ FULL TEXT
research
08/10/2021

An experimental study of two predictive reinforcement learning methods and comparison with model-predictive control

Reinforcement learning (RL) has been successfully used in various simula...
research
12/10/2020

Blending MPC Value Function Approximation for Efficient Reinforcement Learning

Model-Predictive Control (MPC) is a powerful tool for controlling comple...
research
10/27/2017

Declarative vs Rule-based Control for Flocking Dynamics

The popularity of rule-based flocking models, such as Reynolds' classic ...
research
07/20/2022

Governor: a Reference Generator for Nonlinear Model Predictive Control in Legged Robots

Model Predictive Control (MPC) approaches are widely used in robotics, s...
research
09/22/2022

Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation

Despite decades of research, existing navigation systems still face real...
research
12/05/2022

Learning Sampling Distributions for Model Predictive Control

Sampling-based methods have become a cornerstone of contemporary approac...
research
04/23/2021

Optimal Cost Design for Model Predictive Control

Many robotics domains use some form of nonconvex model predictive contro...

Please sign up or login with your details

Forgot password? Click here to reset