Log In Sign Up

Detecting Anomalies Through Contrast in Heterogeneous Data

by   Debanjan Datta, et al.

Detecting anomalies has been a fundamental approach in detecting potentially fraudulent activities. Tasked with detection of illegal timber trade that threatens ecosystems and economies and association with other illegal activities, we formulate our problem as one of anomaly detection. Among other challenges annotations are unavailable for our large-scale trade data with heterogeneous features (categorical and continuous), that can assist in building automated systems to detect fraudulent transactions. Modelling the task as unsupervised anomaly detection, we propose a novel model Contrastive Learning based Heterogeneous Anomaly Detector to address shortcomings of prior models. Our model uses an asymmetric autoencoder that can effectively handle large arity categorical variables, but avoids assumptions about structure of data in low-dimensional latent space and is robust to changes to hyper-parameters. The likelihood of data is approximated through an estimator network, which is jointly trained with the autoencoder,using negative sampling. Further the details and intuition for an effective negative sample generation approach for heterogeneous data are outlined. We provide a qualitative study to showcase the effectiveness of our model in detecting anomalies in timber trade.


Scrutinizing Shipment Records To Thwart Illegal Timber Trade

Timber and forest products made from wood, like furniture, are valuable ...

Anomaly Detection and Localisation using Mixed Graphical Models

We propose a method that performs anomaly detection and localisation wit...

Categorical anomaly detection in heterogeneous data using minimum description length clustering

Fast and effective unsupervised anomaly detection algorithms have been p...

CSCAD: Correlation Structure-based Collective Anomaly Detection in Complex System

Detecting anomalies in large complex systems is a critical and challengi...

Anomaly Detection and Interpretation using Multimodal Autoencoder and Sparse Optimization

Automated anomaly detection is essential for managing information and co...

Detecting anomalies in heterogeneous population-scale VAT networks

Anomaly detection in network science is the method to determine aberrant...