PALMER: Perception-Action Loop with Memory for Long-Horizon Planning

12/08/2022
by   Onur Beker, et al.
0

To achieve autonomy in a priori unknown real-world scenarios, agents should be able to: i) act from high-dimensional sensory observations (e.g., images), ii) learn from past experience to adapt and improve, and iii) be capable of long horizon planning. Classical planning algorithms (e.g. PRM, RRT) are proficient at handling long-horizon planning. Deep learning based methods in turn can provide the necessary representations to address the others, by modeling statistical contingencies between observations. In this direction, we introduce a general-purpose planning algorithm called PALMER that combines classical sampling-based planning algorithms with learning-based perceptual representations. For training these perceptual representations, we combine Q-learning with contrastive representation learning to create a latent space where the distance between the embeddings of two states captures how easily an optimal policy can traverse between them. For planning with these perceptual representations, we re-purpose classical sampling-based planning algorithms to retrieve previously observed trajectory segments from a replay buffer and restitch them into approximately optimal paths that connect any given pair of start and goal states. This creates a tight feedback loop between representation learning, memory, reinforcement learning, and sampling-based planning. The end result is an experiential framework for long-horizon planning that is significantly more robust and sample efficient compared to existing methods.

READ FULL TEXT

page 2

page 7

page 8

page 9

page 23

page 24

research
03/13/2020

Sparse Graphical Memory for Robust Planning

To operate effectively in the real world, artificial agents must act fro...
research
03/07/2023

Sample-efficient Real-time Planning with Curiosity Cross-Entropy Method and Contrastive Learning

Model-based reinforcement learning (MBRL) with real-time planning has sh...
research
06/12/2019

Search on the Replay Buffer: Bridging Planning and Reinforcement Learning

The history of learning for control has been an exciting back and forth ...
research
10/16/2018

Learning abstract planning domains and mappings to real world perceptions

Most of the works on planning and learning, e.g., planning by (model bas...
research
09/28/2019

Regression Planning Networks

Recent learning-to-plan methods have shown promising results on planning...
research
09/14/2021

Comparing Reconstruction- and Contrastive-based Models for Visual Task Planning

Learning state representations enables robotic planning directly from ra...
research
03/24/2022

Augment-Connect-Explore: a Paradigm for Visual Action Planning with Data Scarcity

Visual action planning particularly excels in applications where the sta...

Please sign up or login with your details

Forgot password? Click here to reset