Spatial-Temporal Deep Embedding for Vehicle Trajectory Reconstruction from High-Angle Video

09/17/2022
by   Tianya T. Zhang Ph. D., et al.
17

Spatial-temporal Map (STMap)-based methods have shown great potential to process high-angle videos for vehicle trajectory reconstruction, which can meet the needs of various data-driven modeling and imitation learning applications. In this paper, we developed Spatial-Temporal Deep Embedding (STDE) model that imposes parity constraints at both pixel and instance levels to generate instance-aware embeddings for vehicle stripe segmentation on STMap. At pixel level, each pixel was encoded with its 8-neighbor pixels at different ranges, and this encoding is subsequently used to guide a neural network to learn the embedding mechanism. At the instance level, a discriminative loss function is designed to pull pixels belonging to the same instance closer and separate the mean value of different instances far apart in the embedding space. The output of the spatial-temporal affinity is then optimized by the mutex-watershed algorithm to obtain final clustering results. Based on segmentation metrics, our model outperformed five other baselines that have been used for STMap processing and shows robustness under the influence of shadows, static noises, and overlapping. The designed model is applied to process all public NGSIM US-101 videos to generate complete vehicle trajectories, indicating a good scalability and adaptability. Last but not least, the strengths of the scanline method with STDE and future directions were discussed. Code, STMap dataset and video trajectory are made publicly available in the online repository. GitHub Link: shorturl.at/jklT0.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

page 8

page 10

research
01/13/2022

Spatial-Temporal Map Vehicle Trajectory Detection Using Dynamic Mode Decomposition and Res-UNet+ Neural Networks

This paper presents a machine-learning-enhanced longitudinal scanline me...
research
01/03/2018

Instance Embedding Transfer to Unsupervised Video Object Segmentation

We propose a method for unsupervised video object segmentation by transf...
research
09/14/2017

Learning to Segment Instances in Videos with Spatial Propagation Network

We propose a deep learning-based framework for instance-level object seg...
research
11/27/2018

Affinity Derivation and Graph Merge for Instance Segmentation

We present an instance segmentation scheme based on pixel affinity infor...
research
09/03/2023

Spatial-temporal Vehicle Re-identification

Vehicle re-identification (ReID) in a large-scale camera network is impo...
research
11/19/2020

Towards Spatio-Temporal Video Scene Text Detection via Temporal Clustering

With only bounding-box annotations in the spatial domain, existing video...
research
04/14/2020

Divergence-Based Adaptive Extreme Video Completion

Extreme image or video completion, where, for instance, we only retain 1...

Please sign up or login with your details

Forgot password? Click here to reset