Weakly Supervised Disentangled Representation for Goal-conditioned Reinforcement Learning

02/28/2022
by   Zhifeng Qian, et al.
6

Goal-conditioned reinforcement learning is a crucial yet challenging algorithm which enables agents to achieve multiple user-specified goals when learning a set of skills in a dynamic environment. However, it typically requires millions of the environmental interactions explored by agents, which is sample-inefficient. In the paper, we propose a skill learning framework DR-GRL that aims to improve the sample efficiency and policy generalization by combining the Disentangled Representation learning and Goal-conditioned visual Reinforcement Learning. In a weakly supervised manner, we propose a Spatial Transform AutoEncoder (STAE) to learn an interpretable and controllable representation in which different parts correspond to different object attributes (shape, color, position). Due to the high controllability of the representations, STAE can simply recombine and recode the representations to generate unseen goals for agents to practice themselves. The manifold structure of the learned representation maintains consistency with the physical position, which is beneficial for reward calculation. We empirically demonstrate that DR-GRL significantly outperforms the previous methods in sample efficiency and policy generalization. In addition, DR-GRL is also easy to expand to the real robot.

READ FULL TEXT

page 1

page 5

page 6

page 7

research
07/12/2018

Visual Reinforcement Learning with Imagined Goals

For an autonomous agent to fulfill a wide range of user-specified goals ...
research
07/20/2023

Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planning

Goal-Conditioned Reinforcement Learning (GCRL) can enable agents to spon...
research
04/16/2019

Object-Oriented Dynamics Learning through Multi-Level Abstraction

Object-based approaches for learning action-conditioned dynamics has dem...
research
11/01/2022

Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning

Goal-conditioned reinforcement learning (RL) is a promising direction fo...
research
05/04/2022

State Representation Learning for Goal-Conditioned Reinforcement Learning

This paper presents a novel state representation for reward-free Markov ...
research
10/23/2019

Contextual Imagined Goals for Self-Supervised Robotic Learning

While reinforcement learning provides an appealing formalism for learnin...
research
04/11/2022

Learning Object-Centered Autotelic Behaviors with Graph Neural Networks

Although humans live in an open-ended world and endlessly face new chall...

Please sign up or login with your details

Forgot password? Click here to reset