Navigation Agents for the Visually Impaired: A Sidewalk Simulator and Experiments

10/29/2019
by   Martin Weiss, et al.
18

Millions of blind and visually-impaired (BVI) people navigate urban environments every day, using smartphones for high-level path-planning and white canes or guide dogs for local information. However, many BVI people still struggle to travel to new places. In our endeavor to create a navigation assistant for the BVI, we found that existing Reinforcement Learning (RL) environments were unsuitable for the task. This work introduces SEVN, a sidewalk simulation environment and a neural network-based approach to creating a navigation agent. SEVN contains panoramic images with labels for house numbers, doors, and street name signs, and formulations for several navigation tasks. We study the performance of an RL algorithm (PPO) in this setting. Our policy model fuses multi-modal observations in the form of variable resolution images, visible text, and simulated GPS data to navigate to a goal door. We hope that this dataset, simulator, and experimental results will provide a foundation for further research into the creation of agents that can assist members of the BVI community with outdoor navigation.

READ FULL TEXT

page 2

page 13

research
09/23/2021

Hierarchies of Planning and Reinforcement Learning for Robot Navigation

Solving robotic navigation tasks via reinforcement learning (RL) is chal...
research
04/05/2017

Supporting Navigation of Outdoor Shopping Complexes for Visually-impaired Users through Multi-modal Data Fusion

Outdoor shopping complexes (OSC) are extremely difficult for people with...
research
06/13/2019

Cross-View Policy Learning for Street Navigation

The ability to navigate from visual observations in unfamiliar environme...
research
12/06/2019

A pedestrian path-planning model in accordance with obstacle's danger with reinforcement learning

Most microscopic pedestrian navigation models use the concept of "forces...
research
07/27/2021

Angel's Girl for Blind Painters: an Efficient Painting Navigation System Validated by Multimodal Evaluation Approach

For people who ardently love painting but unfortunately have visual impa...
research
09/07/2023

Navigation Through Endoluminal Channels Using Q-Learning

In this paper, we present a novel approach to navigating endoluminal cha...
research
10/07/2022

Learning a Visually Grounded Memory Assistant

We introduce a novel interface for large scale collection of human memor...

Please sign up or login with your details

Forgot password? Click here to reset