Symmetry-aware Neural Architecture for Embodied Visual Navigation

12/17/2021
by   Shuang Liu, et al.
0

Visual exploration is a task that seeks to visit all the navigable areas of an environment as quickly as possible. The existing methods employ deep reinforcement learning (RL) as the standard tool for the task. However, they tend to be vulnerable to statistical shifts between the training and test data, resulting in poor generalization over novel environments that are out-of-distribution (OOD) from the training data. In this paper, we attempt to improve the generalization ability by utilizing the inductive biases available for the task. Employing the active neural SLAM (ANS) that learns exploration policies with the advantage actor-critic (A2C) method as the base framework, we first point out that the mappings represented by the actor and the critic should satisfy specific symmetries. We then propose a network design for the actor and the critic to inherently attain these symmetries. Specifically, we use G-convolution instead of the standard convolution and insert the semi-global polar pooling (SGPP) layer, which we newly design in this study, in the last section of the critic network. Experimental results show that our method increases area coverage by 8.1 m^2 when trained on the Gibson dataset and tested on the MP3D dataset, establishing the new state-of-the-art.

READ FULL TEXT

page 7

page 12

research
06/14/2021

Analysis of a Target-Based Actor-Critic Algorithm with Linear Function Approximation

Actor-critic methods integrating target networks have exhibited a stupen...
research
02/07/2019

The Actor-Advisor: Policy Gradient With Off-Policy Advice

Actor-critic algorithms learn an explicit policy (actor), and an accompa...
research
01/26/2021

Finite Sample Analysis of Two-Time-Scale Natural Actor-Critic Algorithm

Actor-critic style two-time-scale algorithms are very popular in reinfor...
research
10/26/2018

Deep Intrinsically Motivated Continuous Actor-Critic for Efficient Robotic Visuomotor Skill Learning

In this paper, we present a new intrinsically motivated actor-critic alg...
research
11/25/2020

Symmetry-Aware Actor-Critic for 3D Molecular Design

Automating molecular design using deep reinforcement learning (RL) has t...
research
11/11/2019

DRiLLS: Deep Reinforcement Learning for Logic Synthesis

Logic synthesis requires extensive tuning of the synthesis optimization ...
research
10/03/2022

Latent State Marginalization as a Low-cost Approach for Improving Exploration

While the maximum entropy (MaxEnt) reinforcement learning (RL) framework...

Please sign up or login with your details

Forgot password? Click here to reset