InCrowdFormer: On-Ground Pedestrian World Model From Egocentric Views

03/16/2023
by   Mai Nishimura, et al.
0

We introduce an on-ground Pedestrian World Model, a computational model that can predict how pedestrians move around an observer in the crowd on the ground plane, but from just the egocentric-views of the observer. Our model, InCrowdFormer, fully leverages the Transformer architecture by modeling pedestrian interaction and egocentric to top-down view transformation with attention, and autoregressively predicts on-ground positions of a variable number of people with an encoder-decoder architecture. We encode the uncertainties arising from unknown pedestrian heights with latent codes to predict the posterior distributions of pedestrian positions. We validate the effectiveness of InCrowdFormer on a novel prediction benchmark of real movements. The results show that InCrowdFormer accurately predicts the future coordination of pedestrians. To the best of our knowledge, InCrowdFormer is the first-of-its-kind pedestrian world model which we believe will benefit a wide range of egocentric-view applications including crowd navigation, tracking, and synthesis.

READ FULL TEXT

page 1

page 7

page 8

page 9

research
10/12/2022

ViewBirdiformer: Learning to recover ground-plane crowd trajectories and ego-motion from a single ego-centric view

We introduce a novel learning-based method for view birdification, the t...
research
07/15/2021

Learning Sparse Interaction Graphs of Partially Observed Pedestrians for Trajectory Prediction

Multi-pedestrian trajectory prediction is an indispensable safety elemen...
research
11/09/2021

View Birdification in the Crowd: Ground-Plane Localization from Perceived Movements

We introduce view birdification, the problem of recovering ground-plane ...
research
10/30/2022

Context-empowered Visual Attention Prediction in Pedestrian Scenarios

Effective and flexible allocation of visual attention is key for pedestr...
research
11/12/2020

Anticipatory Navigation in Crowds by Probabilistic Prediction of Pedestrian Future Movements

Critical for the coexistence of humans and robots in dynamic environment...
research
02/05/2016

Wayfinding and cognitive maps for pedestrian models

Usually, routing models in pedestrian dynamics assume that agents have f...
research
09/17/2018

Building Prior Knowledge: A Markov Based Pedestrian Prediction Model Using Urban Environmental Data

Autonomous Vehicles navigating in urban areas have a need to understand ...

Please sign up or login with your details

Forgot password? Click here to reset