An Incremental Clustering Method for Anomaly Detection in Flight Data

05/20/2020
by   Weizun Zhao, et al.
8

Safety is a top priority for civil aviation. Data mining in digital Flight Data Recorder (FDR) or Quick Access Recorder (QAR) data, commonly referred as black box data on aircraft, has gained interest from researchers, airlines, and aviation regulation agencies for safety management. New anomaly detection methods based on supervised or unsupervised learning have been developed to monitor pilot operations and detect any risks from onboard digital flight data recorder data. However, all existing anomaly detection methods are offline learning - the models are trained once using historical data and used for all future predictions. In practice, new QAR data are generated by every flight and collected by airlines whenever a datalink is available. Offline methods cannot respond to new data in time. Though these offline models can be updated by being re-trained after adding new data to the original training set, it is time-consuming and computational costly to train a new model every time new data come in. To address this problem, we propose a novel incremental anomaly detection method to identify common patterns and detect outliers in flight operations from FDR data. The proposed method is based on Gaussian Mixture Model (GMM). An initial GMM cluster model is trained on historical offline data. Then, it continuously adapts to new incoming data points via an expectation-maximization (EM) algorithm. To track changes in flight operation patterns, only model parameters need to be saved, not the raw flight data. The proposed method was tested on two sets of simulation data. Comparable results were found from the proposed online method and a classic offline model. A real-world application of the proposed method is demonstrated using FDR data from daily operations of an airline. Results are presented and future challenges of using online learning scheme for flight data analytics are discussed.

READ FULL TEXT

page 24

page 27

page 31

page 34

research
01/31/2023

Real-Time Outlier Detection with Dynamic Process Limits

Anomaly detection methods are part of the systems where rare events may ...
research
05/25/2023

Towards Total Online Unsupervised Anomaly Detection and Localization in Industrial Vision

Although existing image anomaly detection methods yield impressive resul...
research
02/21/2016

Semi-Markov Switching Vector Autoregressive Model-based Anomaly Detection in Aviation Systems

In this work we consider the problem of anomaly detection in heterogeneo...
research
07/01/2019

Automatic Real-time Anomaly Detection for Autonomous Aerial Vehicles

The recent increase in the use of aerial vehicles raises concerns about ...
research
04/05/2020

Any-Shot Sequential Anomaly Detection in Surveillance Videos

Anomaly detection in surveillance videos has been recently gaining atten...
research
01/25/2016

Expected Similarity Estimation for Large-Scale Batch and Streaming Anomaly Detection

We present a novel algorithm for anomaly detection on very large dataset...
research
09/25/2020

Deep Autoencoding GMM-based Unsupervised Anomaly Detection in Acoustic Signals and its Hyper-parameter Optimization

Failures or breakdowns in factory machinery can be costly to companies, ...

Please sign up or login with your details

Forgot password? Click here to reset