DeepAI
Log In Sign Up

PLAS: Latent Action Space for Offline Reinforcement Learning

11/14/2020
by   Wenxuan Zhou, et al.
0

The goal of offline reinforcement learning is to learn a policy from a fixed dataset, without further interactions with the environment. This setting will be an increasingly more important paradigm for real-world applications of reinforcement learning such as robotics, in which data collection is slow and potentially dangerous. Existing off-policy algorithms have limited performance on static datasets due to extrapolation errors from out-of-distribution actions. This leads to the challenge of constraining the policy to select actions within the support of the dataset during training. We propose to simply learn the Policy in the Latent Action Space (PLAS) such that this requirement is naturally satisfied. We evaluate our method on continuous control benchmarks in simulation and a deformable object manipulation task with a physical robot. We demonstrate that our method provides competitive performance consistently across various continuous control tasks and different types of datasets, outperforming existing offline reinforcement learning methods with explicit constraints. Videos and code are available at https://sites.google.com/view/latent-policy.

READ FULL TEXT

page 6

page 14

06/14/2022

Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning

Offline reinforcement learning (RL) extends the paradigm of classical RL...
07/05/2021

The Least Restriction for Offline Reinforcement Learning

Many practical applications of reinforcement learning (RL) constrain the...
03/29/2021

LASER: Learning a Latent Action Space for Efficient Reinforcement Learning

The process of learning a manipulation task depends strongly on the acti...
11/15/2021

Exploiting Action Impact Regularity and Partially Known Models for Offline Reinforcement Learning

Offline reinforcement learning-learning a policy from a batch of data-is...
08/02/2018

Learning Actionable Representations from Visual Observations

In this work we explore a new approach for robots to teach themselves ab...
11/20/2019

Evaluating task-agnostic exploration for fixed-batch learning of arbitrary future tasks

Deep reinforcement learning has been shown to solve challenging tasks wh...
03/02/2021

Offline Reinforcement Learning with Pseudometric Learning

Offline Reinforcement Learning methods seek to learn a policy from logge...