DeepAI AI Chat
Log In Sign Up

Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation

by   Kenzo Lobos-Tsunekawa, et al.
The University of Tokyo

Reinforcement Learning (RL), among other learning-based methods, represents powerful tools to solve complex robotic tasks (e.g., actuation, manipulation, navigation, etc.), with the need for real-world data to train these systems as one of its most important limitations. The use of simulators is one way to address this issue, yet knowledge acquired in simulations does not work directly in the real-world, which is known as the sim-to-real transfer problem. While previous works focus on the nature of the images used as observations (e.g., textures and lighting), which has proven useful for a sim-to-sim transfer, they neglect other concerns regarding said observations, such as precise geometrical meanings, failing at robot-to-robot, and thus in sim-to-real transfers. We propose a method that learns on an observation space constructed by point clouds and environment randomization, generalizing among robots and simulators to achieve sim-to-real, while also addressing partial observability. We demonstrate the benefits of our methodology on the point goal navigation task, in which our method proves to be highly unaffected to unseen scenarios produced by robot-to-robot transfer, outperforms image-based baselines in robot-randomized experiments, and presents high performances in sim-to-sim conditions. Finally, we perform several experiments to validate the sim-to-real transfer to a physical domestic robot platform, confirming the out-of-the-box performance of our system.


page 1

page 5

page 7


Asymmetric Actor Critic for Image-Based Robot Learning

Deep reinforcement learning (RL) has proven a powerful technique in many...

ViNG: Learning Open-World Navigation with Visual Goals

We propose a learning-based navigation system for reaching visually indi...

ReProHRL: Towards Multi-Goal Navigation in the Real World using Hierarchical Agents

Robots have been successfully used to perform tasks with high precision....

Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

This paper was motivated by the problem of how to make robots fuse and t...

Learning Manipulation Skills Via Hierarchical Spatial Attention

Learning generalizable skills in robotic manipulation has long been chal...

Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks

Randomization is currently a widely used approach in Sim2Real transfer f...

Rule-Based Reinforcement Learning for Efficient Robot Navigation with Space Reduction

For real-world deployments, it is critical to allow robots to navigate i...