TrackletMapper: Ground Surface Segmentation and Mapping from Traffic Participant Trajectories

by   Jannik Zürn, et al.

Robustly classifying ground infrastructure such as roads and street crossings is an essential task for mobile robots operating alongside pedestrians. While many semantic segmentation datasets are available for autonomous vehicles, models trained on such datasets exhibit a large domain gap when deployed on robots operating in pedestrian spaces. Manually annotating images recorded from pedestrian viewpoints is both expensive and time-consuming. To overcome this challenge, we propose TrackletMapper, a framework for annotating ground surface types such as sidewalks, roads, and street crossings from object tracklets without requiring human-annotated data. To this end, we project the robot ego-trajectory and the paths of other traffic participants into the ego-view camera images, creating sparse semantic annotations for multiple types of ground surfaces from which a ground segmentation model can be trained. We further show that the model can be self-distilled for additional performance benefits by aggregating a ground surface map and projecting it into the camera images, creating a denser set of training annotations compared to the sparse tracklet annotations. We qualitatively and quantitatively attest our findings on a novel large-scale dataset for mobile robots operating in pedestrian areas. Code and dataset will be made available at


page 1

page 7

page 8

page 14

page 15

page 17

page 18

page 19


Forecasting Pedestrian Trajectory with Machine-Annotated Training Data

Reliable anticipation of pedestrian trajectory is imperative for the ope...

I see you: A Vehicle-Pedestrian Interaction Dataset from Traffic Surveillance Cameras

The development of autonomous vehicles arises new challenges in urban tr...

APE: An Open and Shared Annotated Dataset for Learning Urban Pedestrian Path Networks

Inferring the full transportation network, including sidewalks and cycle...

WiderPerson: A Diverse Dataset for Dense Pedestrian Detection in the Wild

Pedestrian detection has achieved significant progress with the availabi...

Efficient Volumetric Fusion of Airborne and Street-Side Data for Urban Reconstruction

Airborne acquisition and on-road mobile mapping provide complementary 3D...

Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation

Lacking the ability to sense ambient environments effectively, blind and...

Please sign up or login with your details

Forgot password? Click here to reset