Robust Chauvenet Rejection: Powerful, but Easy to Use Outlier Detection for Heavily Contaminated Data Sets

01/19/2023
by   Nicholas Konz, et al.
0

In Maples et al. (2018) we introduced Robust Chauvenet Outlier Rejection, or RCR, a novel outlier rejection technique that evolves Chauvenet's Criterion by sequentially applying different measures of central tendency and empirically determining the rejective sigma value. RCR is especially powerful for cleaning heavily-contaminated samples, and unlike other methods such as sigma clipping, it manages to be both accurate and precise when characterizing the underlying uncontaminated distributions of data sets, by using decreasingly robust but increasingly precise statistics in sequence. For this work, we present RCR from a software standpoint, newly implemented as a Python package while maintaining the speed of the C++ original. RCR has been well-tested, calibrated and simulated, and it can be used for both one-dimensional outlier rejection and n-dimensional model-fitting, with or without weighted data. RCR is free to use for academic and non-commercial purposes, and the code, documentation and accompanying web calculator can be found and easily used online at https://github.com/nickk124/RCR

READ FULL TEXT

page 4

page 5

page 7

page 8

research
01/06/2019

PyOD: A Python Toolbox for Scalable Outlier Detection

PyOD is an open-source Python toolbox for performing scalable outlier de...
research
06/26/2019

Quantum Entropy Scoring for Fast Robust Mean Estimation and Improved Outlier Detection

We study two problems in high-dimensional robust statistics: robust mean...
research
10/07/2019

PyODDS: An End-to-End Outlier Detection System

PyODDS is an end-to end Python system for outlier detection with databas...
research
03/30/2023

OpenMix: Exploring Outlier Samples for Misclassification Detection

Reliable confidence estimation for deep neural classifiers is a challeng...
research
06/18/2018

Kernel-based Outlier Detection using the Inverse Christoffel Function

Outlier detection methods have become increasingly relevant in recent ye...
research
05/09/2023

Spatially smoothed robust covariance estimation for local outlier detection

Most multivariate outlier detection procedures ignore the spatial depend...
research
05/22/2022

Robust Flow-based Conformal Inference (FCI) with Statistical Guarantee

Conformal prediction aims to determine precise levels of confidence in p...

Please sign up or login with your details

Forgot password? Click here to reset