Sparse Graphical Memory for Robust Planning

To operate effectively in the real world, artificial agents must act from raw sensory input such as images and achieve diverse goals across long time-horizons. On the one hand, recent strides in deep reinforcement and imitation learning have demonstrated impressive ability to learn goal-conditioned policies from high-dimensional image input, though only for short-horizon tasks. On the other hand, classical graphical methods like A* search are able to solve long-horizon tasks, but assume that the graph structure is abstracted away from raw sensory input and can only be constructed with task-specific priors. We wish to combine the strengths of deep learning and classical planning to solve long-horizon tasks from raw sensory input. To this end, we introduce Sparse Graphical Memory (SGM), a new data structure that stores observations and feasible transitions in a sparse memory. SGM can be combined with goal-conditioned RL or imitative agents to solve long-horizon tasks across a diverse set of domains. We show that SGM significantly outperforms current state of the art methods on long-horizon, sparse-reward visual navigation tasks. Project video and code are available at


page 6

page 8

page 9

page 10

page 11


Model-Based Reinforcement Learning via Latent-Space Collocation

The ability to plan into the future while utilizing only raw high-dimens...

PALMER: Perception-Action Loop with Memory for Long-Horizon Planning

To achieve autonomy in a priori unknown real-world scenarios, agents sho...

Imitating Graph-Based Planning with Goal-Conditioned Policies

Recently, graph-based planning algorithms have gained much attention to ...

Broadly-Exploring, Local-Policy Trees for Long-Horizon Task Planning

Long-horizon planning in realistic environments requires the ability to ...

Deep Abstract Q-Networks

We examine the problem of learning and planning on high-dimensional doma...

Search on the Replay Buffer: Bridging Planning and Reinforcement Learning

The history of learning for control has been an exciting back and forth ...

Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors

The ability to predict and plan into the future is fundamental for agent...

Please sign up or login with your details

Forgot password? Click here to reset