Log In Sign Up

Collaborative Anomaly Detection

by   Ke Bai, et al.

In recommendation systems, items are likely to be exposed to various users and we would like to learn about the familiarity of a new user with an existing item. This can be formulated as an anomaly detection (AD) problem distinguishing between "common users" (nominal) and "fresh users" (anomalous). Considering the sheer volume of items and the sparsity of user-item paired data, independently applying conventional single-task detection methods on each item quickly becomes difficult, while correlations between items are ignored. To address this multi-task anomaly detection problem, we propose collaborative anomaly detection (CAD) to jointly learn all tasks with an embedding encoding correlations among tasks. We explore CAD with conditional density estimation and conditional likelihood ratio estimation. We found that: i) estimating a likelihood ratio enjoys more efficient learning and yields better results than density estimation. ii) It is beneficial to select a small number of tasks in advance to learn a task embedding model, and then use it to warm-start all task embeddings. Consequently, these embeddings can capture correlations between tasks and generalize to new correlated tasks.


page 1

page 2

page 3

page 4


Supervised Anomaly Detection based on Deep Autoregressive Density Estimators

We propose a supervised anomaly detection method based on neural density...

Anomaly Detection with Density Estimation

We leverage recent breakthroughs in neural density estimation to propose...

AD-DMKDE: Anomaly Detection through Density Matrices and Fourier Features

This paper presents a novel density estimation method for anomaly detect...

Early Anomaly Detection by Learning and Forecasting Behavior

Graph anomaly detection systems aim at identifying suspicious accounts o...

Multi-Source Anomaly Detection in Distributed IT Systems

The multi-source data generated by distributed systems, provide a holist...

Anomaly Detection in Trajectory Data with Normalizing Flows

The task of detecting anomalous data patterns is as important in practic...

Simulation-Assisted Decorrelation for Resonant Anomaly Detection

A growing number of weak- and unsupervised machine learning approaches t...