Success Weighted by Completion Time: A Dynamics-Aware Evaluation Criteria for Embodied Navigation

by   Naoki Yokoyama, et al.

We present Success weighted by Completion Time (SCT), a new metric for evaluating navigation performance for mobile robots. Several related works on navigation have used Success weighted by Path Length (SPL) as the primary method of evaluating the path an agent makes to a goal location, but SPL is limited in its ability to properly evaluate agents with complex dynamics. In contrast, SCT explicitly takes the agent's dynamics model into consideration, and aims to accurately capture how well the agent has approximated the fastest navigation behavior afforded by its dynamics. While several embodied navigation works use point-turn dynamics, we focus on unicycle-cart dynamics for our agent, which better exemplifies the dynamics model of popular mobile robotics platforms (e.g., LoCoBot, TurtleBot, Fetch, etc.). We also present RRT*-Unicycle, an algorithm for unicycle dynamics that estimates the fastest collision-free path and completion time from a starting pose to a goal location in an environment containing obstacles. We experiment with deep reinforcement learning and reward shaping to train and compare the navigation performance of agents with different dynamics models. In evaluating these agents, we show that in contrast to SPL, SCT is able to capture the advantages in navigation speed a unicycle model has over a simpler point-turn model of dynamics. Lastly, we show that we can successfully deploy our trained models and algorithms outside of simulation in the real world. We embody our agents in an real robot to navigate an apartment, and show that they can generalize in a zero-shot manner.


page 1

page 4

page 5

page 6

page 7


ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings

We present a scalable approach for learning open-world object-goal navig...

How To Train Your HERON

In this paper we apply Deep Reinforcement Learning (Deep RL) and Domain ...

Guided Navigation from Multiple Viewpoints using Qualitative Spatial Reasoning

Navigation is an essential ability for mobile agents to be completely au...

Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents

Recent work has presented embodied agents that can navigate to point-goa...

Tracking and Planning with Spatial World Models

We introduce a method for real-time navigation and tracking with differe...

Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigation

A key challenge on the path to developing agents that learn complex huma...

Real-Time Decentralized Navigation of Nonholonomic Agents Using Shifted Yielding Areas

We present a lightweight, decentralized algorithm for navigating multipl...

Please sign up or login with your details

Forgot password? Click here to reset