Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors

12/01/2020
by   Michelle A. Lee, et al.
0

Using sensor data from multiple modalities presents an opportunity to encode redundant and complementary features that can be useful when one modality is corrupted or noisy. Humans do this everyday, relying on touch and proprioceptive feedback in visually-challenging environments. However, robots might not always know when their sensors are corrupted, as even broken sensors can return valid values. In this work, we introduce the Crossmodal Compensation Model (CCM), which can detect corrupted sensor modalities and compensate for them. CMM is a representation model learned with self-supervision that leverages unimodal reconstruction loss for corruption detection. CCM then discards the corrupted modality and compensates for it with information from the remaining sensors. We show that CCM learns rich state representations that can be used for contact-rich manipulation policies, even when input modalities are corrupted in ways not seen during training time.

READ FULL TEXT

page 1

page 4

page 6

research
02/16/2022

More to Less (M2L): Enhanced Health Recognition in the Wild with Reduced Modality of Wearable Sensors

Accurately recognizing health-related conditions from wearable data is c...
research
02/18/2023

RecNet: Early Attention Guided Feature Recovery

Uncertainty in sensors results in corrupted input streams and hinders th...
research
01/09/2021

Investigating the Effect of Sensor Modalities in Multi-Sensor Detection-Prediction Models

Detection of surrounding objects and their motion prediction are critica...
research
02/17/2022

Multi-Modal Fusion in Contact-Rich Precise Tasks via Hierarchical Policy Learning

Combined visual and force feedback play an essential role in contact-ric...
research
08/23/2022

In-Air Imaging Sonar Sensor Network with Real-Time Processing Using GPUs

For autonomous navigation and robotic applications, sensing the environm...
research
07/13/2020

Low to High Dimensional Modality Hallucination using Aggregated Fields of View

Real-world robotics systems deal with data from a multitude of modalitie...
research
06/04/2007

Virtual Sensor Based Fault Detection and Classification on a Plasma Etch Reactor

The SEMATECH sponsored J-88-E project teaming Texas Instruments with Neu...

Please sign up or login with your details

Forgot password? Click here to reset