Detecting Anomalies Through Contrast in Heterogeneous Data

04/02/2021
by   Debanjan Datta, et al.
27

Detecting anomalies has been a fundamental approach in detecting potentially fraudulent activities. Tasked with detection of illegal timber trade that threatens ecosystems and economies and association with other illegal activities, we formulate our problem as one of anomaly detection. Among other challenges annotations are unavailable for our large-scale trade data with heterogeneous features (categorical and continuous), that can assist in building automated systems to detect fraudulent transactions. Modelling the task as unsupervised anomaly detection, we propose a novel model Contrastive Learning based Heterogeneous Anomaly Detector to address shortcomings of prior models. Our model uses an asymmetric autoencoder that can effectively handle large arity categorical variables, but avoids assumptions about structure of data in low-dimensional latent space and is robust to changes to hyper-parameters. The likelihood of data is approximated through an estimator network, which is jointly trained with the autoencoder,using negative sampling. Further the details and intuition for an effective negative sample generation approach for heterogeneous data are outlined. We provide a qualitative study to showcase the effectiveness of our model in detecting anomalies in timber trade.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

07/20/2016

Anomaly Detection and Localisation using Mixed Graphical Models

We propose a method that performs anomaly detection and localisation wit...
06/14/2020

Categorical anomaly detection in heterogeneous data using minimum description length clustering

Fast and effective unsupervised anomaly detection algorithms have been p...
01/18/2019

Robust Anomaly Detection in Images using Adversarial Autoencoders

Reliably detecting anomalies in a given set of images is a task of high ...
05/30/2021

CSCAD: Correlation Structure-based Collective Anomaly Detection in Complex System

Detecting anomalies in large complex systems is a critical and challengi...
06/26/2021

Detecting anomalies in heterogeneous population-scale VAT networks

Anomaly detection in network science is the method to determine aberrant...
12/18/2018

Anomaly Detection and Interpretation using Multimodal Autoencoder and Sparse Optimization

Automated anomaly detection is essential for managing information and co...
06/29/2018

Unsupervised Detection and Explanation of Latent-class Contextual Anomalies

Detecting and explaining anomalies is a challenging effort. This holds e...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.