DeepAI AI Chat
Log In Sign Up

Focus on Impact: Indoor Exploration with Intrinsic Motivation

by   Roberto Bigazzi, et al.
University of Modena and Reggio Emilia

Exploration of indoor environments has recently experienced a significant interest, also thanks to the introduction of deep neural agents built in a hierarchical fashion and trained with Deep Reinforcement Learning (DRL) on simulated environments. Current state-of-the-art methods employ a dense extrinsic reward that requires the complete a priori knowledge of the layout of the training environment to learn an effective exploration policy. However, such information is expensive to gather in terms of time and resources. In this work, we propose to train the model with a purely intrinsic reward signal to guide exploration, which is based on the impact of the robot's actions on the environment. So far, impact-based rewards have been employed for simple tasks and in procedurally generated synthetic environments with countable states. Since the number of states observable by the agent in realistic indoor environments is non-countable, we include a neural-based density model and replace the traditional count-based regularization with an estimated pseudo-count of previously visited states. The proposed exploration approach outperforms DRL-based competitors relying on intrinsic rewards and surpasses the agents trained with a dense extrinsic reward computed with the environment layouts. We also show that a robot equipped with the proposed approach seamlessly adapts to point-goal navigation and real-world deployment.


page 1

page 3

page 7


RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments

Exploration in sparse reward environments remains one of the key challen...

Unifying Count-Based Exploration and Intrinsic Motivation

We consider an agent's uncertainty about its environment and the problem...

Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learning

This paper investigates exploration strategies of Deep Reinforcement Lea...

Improving robot navigation in crowded environments using intrinsic rewards

Autonomous navigation in crowded environments is an open problem with ma...

Exploration of Indoor Environments Predicting the Layout of Partially Observed Rooms

We consider exploration tasks in which an autonomous mobile robot increm...

Spot the Difference: A Novel Task for Embodied Agents in Changing Environments

Embodied AI is a recent research area that aims at creating intelligent ...

Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environments

Exploration under sparse reward is a long-standing challenge of model-fr...