Log In Sign Up

Hidden Footprints: Learning Contextual Walkability from 3D Human Trails

by   Jin Sun, et al.

Predicting where people can walk in a scene is important for many tasks, including autonomous driving systems and human behavior analysis. Yet learning a computational model for this purpose is challenging due to semantic ambiguity and a lack of labeled data: current datasets only tell you where people are, not where they could be. We tackle this problem by leveraging information from existing datasets, without additional labeling. We first augment the set of valid, labeled walkable regions by propagating person observations between images, utilizing 3D information to create what we call hidden footprints. However, this augmented data is still sparse. We devise a training strategy designed for such sparse labels, combining a class-balanced classification loss with a contextual adversarial loss. Using this strategy, we demonstrate a model that learns to predict a walkability map from a single image. We evaluate our model on the Waymo and Cityscapes datasets, demonstrating superior performance compared to baselines and state-of-the-art models.


page 1

page 3

page 5

page 7

page 9

page 13

page 14


Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels

While neural networks trained for semantic segmentation are essential fo...

Augmented Bi-path Network for Few-shot Learning

Few-shot Learning (FSL) which aims to learn from few labeled training da...

What Face and Body Shapes Can Tell About Height

Recovering a person's height from a single image is important for virtua...

SIMBAR: Single Image-Based Scene Relighting For Effective Data Augmentation For Automated Driving Vision Tasks

Real-world autonomous driving datasets comprise of images aggregated fro...

Learning to Reconstruct People in Clothing from a Single RGB Camera

We present a learning-based model to infer the personalized 3D shape of ...

DefogGAN: Predicting Hidden Information in the StarCraft Fog of War with Generative Adversarial Nets

We propose DefogGAN, a generative approach to the problem of inferring s...