DeepAI AI Chat
Log In Sign Up

A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder

by   Daehyung Park, et al.
Georgia Institute of Technology

The detection of anomalous executions is valuable for reducing potential hazards in assistive manipulation. Multimodal sensory signals can be helpful for detecting a wide range of anomalies. However, the fusion of high-dimensional and heterogeneous modalities is a challenging problem. We introduce a long short-term memory based variational autoencoder (LSTM-VAE) that fuses signals and reconstructs their expected distribution. We also introduce an LSTM-VAE-based detector using a reconstruction-based anomaly score and a state-based threshold. For evaluations with 1,555 robot-assisted feeding executions including 12 representative types of anomalies, our detector had a higher area under the receiver operating characteristic curve (AUC) of 0.8710 than 5 other baseline detectors from the literature. We also show the multimodal fusion through the LSTM-VAE is effective by comparing our detector with 17 raw sensory signals versus 4 hand-engineered features.


page 1

page 3

page 5

page 6


Fusing multimodal neuroimaging data with a variational autoencoder

Neuroimaging studies often involve the collection of multiple data modal...

Smart Metering System Capable of Anomaly Detection by Bi-directional LSTM Autoencoder

Anomaly detection is concerned with a wide range of applications such as...

GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection

This paper looks into the problem of detecting network anomalies by anal...

Deep Learning for the Analysis of Disruption Precursors based on Plasma Tomography

The JET baseline scenario is being developed to achieve high fusion perf...

Automated Antenna Testing Using Encoder-Decoder-based Anomaly Detection

We propose a new method for testing antenna arrays that records the radi...

Multi-Modal Anomaly Detection for Unstructured and Uncertain Environments

To achieve high-levels of autonomy, modern robots require the ability to...

What went wrong?: Identification of Everyday Object Manipulation Anomalies

Extending the abilities of service robots is important for expanding wha...