DeepAI
Log In Sign Up

Hidden Footprints: Learning Contextual Walkability from 3D Human Trails

08/19/2020
by   Jin Sun, et al.
9

Predicting where people can walk in a scene is important for many tasks, including autonomous driving systems and human behavior analysis. Yet learning a computational model for this purpose is challenging due to semantic ambiguity and a lack of labeled data: current datasets only tell you where people are, not where they could be. We tackle this problem by leveraging information from existing datasets, without additional labeling. We first augment the set of valid, labeled walkable regions by propagating person observations between images, utilizing 3D information to create what we call hidden footprints. However, this augmented data is still sparse. We devise a training strategy designed for such sparse labels, combining a class-balanced classification loss with a contextual adversarial loss. Using this strategy, we demonstrate a model that learns to predict a walkability map from a single image. We evaluate our model on the Waymo and Cityscapes datasets, demonstrating superior performance compared to baselines and state-of-the-art models.

READ FULL TEXT

page 1

page 3

page 5

page 7

page 9

page 13

page 14

05/12/2020

Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels

While neural networks trained for semantic segmentation are essential fo...
07/15/2020

Augmented Bi-path Network for Few-shot Learning

Few-shot Learning (FSL) which aims to learn from few labeled training da...
05/25/2018

What Face and Body Shapes Can Tell About Height

Recovering a person's height from a single image is important for virtua...
04/01/2022

SIMBAR: Single Image-Based Scene Relighting For Effective Data Augmentation For Automated Driving Vision Tasks

Real-world autonomous driving datasets comprise of images aggregated fro...
03/14/2019

Learning to Reconstruct People in Clothing from a Single RGB Camera

We present a learning-based model to infer the personalized 3D shape of ...
03/04/2020

DefogGAN: Predicting Hidden Information in the StarCraft Fog of War with Generative Adversarial Nets

We propose DefogGAN, a generative approach to the problem of inferring s...