Local Policy Optimization for Trajectory-Centric Reinforcement Learning

by   Patrik Kolaric, et al.
The University of Texas at Arlington

The goal of this paper is to present a method for simultaneous trajectory and local stabilizing policy optimization to generate local policies for trajectory-centric model-based reinforcement learning (MBRL). This is motivated by the fact that global policy optimization for non-linear systems could be a very challenging problem both algorithmically and numerically. However, a lot of robotic manipulation tasks are trajectory-centric, and thus do not require a global model or policy. Due to inaccuracies in the learned model estimates, an open-loop trajectory optimization process mostly results in very poor performance when used on the real system. Motivated by these problems, we try to formulate the problem of trajectory optimization and local policy synthesis as a single optimization problem. It is then solved simultaneously as an instance of nonlinear programming. We provide some results for analysis as well as achieved performance of the proposed technique under some simplifying assumptions.


Learning Off-Policy with Online Planning

We propose Learning Off-Policy with Online Planning (LOOP), combining th...

Decoupled Data Based Approach for Learning to Control Nonlinear Dynamical Systems

This paper addresses the problem of learning the optimal control policy ...

TAMOLS: Terrain-Aware Motion Optimization for Legged Systems

Terrain geometry is, in general, non-smooth, non-linear, non-convex, and...

Guided Policy Search using Sequential Convex Programming for Initialization of Trajectory Optimization Algorithms

Nonlinear trajectory optimization algorithms have been developed to hand...

Local Trajectory Stabilization for Dexterous Manipulation via Piecewise Affine Approximations

We propose a model-based approach to design feedback policies for dexter...

The Power of Learned Locally Linear Models for Nonlinear Policy Optimization

A common pipeline in learning-based control is to iteratively estimate a...

CACTO: Continuous Actor-Critic with Trajectory Optimization – Towards global optimality

This paper presents a novel algorithm for the continuous control of dyna...

Please sign up or login with your details

Forgot password? Click here to reset