DeepAI AI Chat
Log In Sign Up

Deep Clustering based Fair Outlier Detection

06/09/2021
by   Hanyu Song, et al.
0

In this paper, we focus on the fairness issues regarding unsupervised outlier detection. Traditional algorithms, without a specific design for algorithmic fairness, could implicitly encode and propagate statistical bias in data and raise societal concerns. To correct such unfairness and deliver a fair set of potential outlier candidates, we propose Deep Clustering based Fair Outlier Detection (DCFOD) that learns a good representation for utility maximization while enforcing the learnable representation to be subgroup-invariant on the sensitive attribute. Considering the coupled and reciprocal nature between clustering and outlier detection, we leverage deep clustering to discover the intrinsic cluster structure and out-of-structure instances. Meanwhile, an adversarial training erases the sensitive pattern for instances for fairness adaptation. Technically, we propose an instance-level weighted representation learning strategy to enhance the joint deep clustering and outlier detection, where the dynamic weight module re-emphasizes contributions of likely-inliers while mitigating the negative impact from outliers. Demonstrated by experiments on eight datasets comparing to 17 outlier detection algorithms, our DCFOD method consistently achieves superior performance on both the outlier detection validity and two types of fairness notions in outlier detection.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/20/2020

Fair Outlier Detection

An outlier detection method may be considered fair over specified sensit...
12/21/2021

Understanding and Mitigating the Effect of Outliers in Fair Ranking

Traditional ranking systems are expected to sort items in the order of t...
03/17/2021

Fairness-aware Outlier Ensemble

Outlier ensemble methods have shown outstanding performance on the disco...
12/01/2019

XGBOD: Improving Supervised Outlier Detection with Unsupervised Representation Learning

A new semi-supervised ensemble algorithm called XGBOD (Extreme Gradient ...
05/11/2021

Towards a Model for LSH

As data volumes continue to grow, clustering and outlier detection algor...
02/22/2023

Cluster Purging: Efficient Outlier Detection based on Rate-Distortion Theory

Rate-distortion theory-based outlier detection builds upon the rationale...
05/10/2018

A Proposal for Outlier and Noise Detection in Public Officials' Affidavits

Outlier and noise detection processes are highly useful in the quality a...