What do navigation agents learn about their environment?

06/17/2022
by   Kshitij Dwivedi, et al.
0

Today's state of the art visual navigation agents typically consist of large deep learning models trained end to end. Such models offer little to no interpretability about the learned skills or the actions of the agent taken in response to its environment. While past works have explored interpreting deep learning models, little attention has been devoted to interpreting embodied AI systems, which often involve reasoning about the structure of the environment, target characteristics and the outcome of one's actions. In this paper, we introduce the Interpretability System for Embodied agEnts (iSEE) for Point Goal and Object Goal navigation agents. We use iSEE to probe the dynamic representations produced by these agents for the presence of information about the agent as well as the environment. We demonstrate interesting insights about navigation agents using iSEE, including the ability to encode reachable locations (to avoid obstacles), visibility of the target, progress from the initial spawn location as well as the dramatic effect on the behaviors of agents when we mask out critical individual neurons. The code is available at: https://github.com/allenai/iSEE

READ FULL TEXT

page 4

page 6

page 13

research
06/30/2022

Visual Pre-training for Navigation: What Can We Learn from Noise?

A powerful paradigm for sensorimotor control is to predict actions from ...
research
04/28/2021

Pushing it out of the Way: Interactive Visual Navigation

We have observed significant progress in visual navigation for embodied ...
research
06/08/2021

RobustNav: Towards Benchmarking Robustness in Embodied Navigation

As an attempt towards assessing the robustness of embodied navigation ag...
research
01/02/2022

The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agents

The last few years have witnessed substantial progress in the field of e...
research
11/03/2020

Guided Navigation from Multiple Viewpoints using Qualitative Spatial Reasoning

Navigation is an essential ability for mobile agents to be completely au...
research
11/18/2022

Ask4Help: Learning to Leverage an Expert for Embodied Tasks

Embodied AI agents continue to become more capable every year with the a...
research
04/08/2021

Auxiliary Tasks and Exploration Enable ObjectNav

ObjectGoal Navigation (ObjectNav) is an embodied task wherein agents are...

Please sign up or login with your details

Forgot password? Click here to reset