On the Complexity of Exploration in Goal-Driven Navigation

11/16/2018
by   Maruan Al-Shedivat, et al.
10

Building agents that can explore their environments intelligently is a challenging open problem. In this paper, we make a step towards understanding how a hierarchical design of the agent's policy can affect its exploration capabilities. First, we design EscapeRoom environments, where the agent must figure out how to navigate to the exit by accomplishing a number of intermediate tasks (subgoals), such as finding keys or opening doors. Our environments are procedurally generated and vary in complexity, which can be controlled by the number of subgoals and relationships between them. Next, we propose to measure the complexity of each environment by constructing dependency graphs between the goals and analytically computing hitting times of a random walk in the graph. We empirically evaluate Proximal Policy Optimization (PPO) with sparse and shaped rewards, a variation of policy sketches, and a hierarchical version of PPO (called HiPPO) akin to h-DQN. We show that analytically estimated hitting time in goal dependency graphs is an informative metric of the environment complexity. We conjecture that the result should hold for environments other than navigation. Finally, we show that solving environments beyond certain level of complexity requires hierarchical approaches.

READ FULL TEXT
research
11/23/2022

Predicting Topological Maps for Visual Navigation in Unexplored Environments

We propose a robotic learning system for autonomous exploration and navi...
research
02/08/2023

Learning Graph-Enhanced Commander-Executor for Multi-Agent Navigation

This paper investigates the multi-agent navigation problem, which requir...
research
06/23/2022

Walk the Random Walk: Learning to Discover and Reach Goals Without Supervision

Learning a diverse set of skills by interacting with an environment with...
research
02/13/2023

Improving robot navigation in crowded environments using intrinsic rewards

Autonomous navigation in crowded environments is an open problem with ma...
research
03/05/2019

Learning Exploration Policies for Navigation

Numerous past works have tackled the problem of task-driven navigation. ...
research
09/18/2023

Learning Spatial and Temporal Hierarchies: Hierarchical Active Inference for navigation in Multi-Room Maze Environments

Cognitive maps play a crucial role in facilitating flexible behaviour by...
research
06/23/2023

Inferring Hierarchical Structure in Multi-Room Maze Environments

Cognitive maps play a crucial role in facilitating flexible behaviour by...

Please sign up or login with your details

Forgot password? Click here to reset