Geometric Cross-Modal Comparison of Heterogeneous Sensor Data

11/23/2017
by   Christopher J. Tralie, et al.
0

In this work, we address the problem of cross-modal comparison of aerial data streams. A variety of simulated automobile trajectories are sensed using two different modalities: full-motion video, and radio-frequency (RF) signals received by detectors at various locations. The information represented by the two modalities is compared using self-similarity matrices (SSMs) corresponding to time-ordered point clouds in feature spaces of each of these data sources; we note that these feature spaces can be of entirely different scale and dimensionality. Several metrics for comparing SSMs are explored, including a cutting-edge time-warping technique that can simultaneously handle local time warping and partial matches, while also controlling for the change in geometry between feature spaces of the two modalities. We note that this technique is quite general, and does not depend on the choice of modalities. In this particular setting, we demonstrate that the cross-modal distance between SSMs corresponding to the same trajectory type is smaller than the cross-modal distance between SSMs corresponding to distinct trajectory types, and we formalize this observation via precision-recall metrics in experiments. Finally, we comment on promising implications of these ideas for future integration into multiple-hypothesis tracking systems.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 8

research
10/13/2018

Multi-scale Geometric Summaries for Similarity-based Sensor Fusion

In this work, we address fusion of heterogeneous sensor data using wavel...
research
11/20/2017

Self-Similarity Based Time Warping

In this work, we explore the problem of aligning two time-ordered point ...
research
07/02/2023

Deep Cross-Modal Steganography Using Neural Representations

Steganography is the process of embedding secret data into another messa...
research
02/02/2019

Collaborative Quantization for Cross-Modal Similarity Search

Cross-modal similarity search is a problem about designing a search syst...
research
10/27/2022

3D Shape Knowledge Graph for Cross-domain and Cross-modal 3D Shape Retrieval

With the development of 3D modeling and fabrication, 3D shape retrieval ...
research
12/19/2016

Cross-Modal Manifold Learning for Cross-modal Retrieval

This paper presents a new scalable algorithm for cross-modal similarity ...
research
09/16/2019

Learning Controls Using Cross-Modal Representations: Bridging Simulation and Reality for Drone Racing

Machines are a long way from robustly solving open-world perception-cont...

Please sign up or login with your details

Forgot password? Click here to reset