Conservation of the t-digest Scale Invariant

03/24/2019
by   Ted Dunning, et al.
0

A t-digest is a compact data structure that allows estimates of quantiles which increased accuracy near q = 0 or q=1. This is done by clustering samples from R subject to a constraint that the number of points associated with any particular centroid is constrained so that the so-called k-size of the centroid is always < 1. The k-size is defined using a scale function that maps quantile q to index k. Since the centroids are real numbers, they can be ordered and thus the quantile range of a centroid can be mapped into an interval in k whose size is the k-size of that centroid. The accuracy of quantile estimates made using a t-digest depends on the invariance of this constraint even as new data is added or t-digests are merged. This paper provides proofs of this invariance for four practically important scale functions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/24/2019

The Size of a t-Digest

A t-digest is a compact data structure that allows estimates of quantile...
research
09/12/2019

Fast Algorithms for the Quantile Regression Process

The widespread use of quantile regression methods depends crucially on t...
research
02/02/2018

On the Predictive Risk in Misspecified Quantile Regression

In the present paper we investigate the predictive risk of possibly miss...
research
02/14/2023

Private Statistical Estimation of Many Quantiles

This work studies the estimation of many statistical quantiles under dif...
research
12/28/2020

Generalized Quantile Loss for Deep Neural Networks

This note presents a simple way to add a count (or quantile) constraint ...
research
08/15/2023

On Subsampled Quantile Randomized Kaczmarz

When solving noisy linear systems Ax = b + c, the theoretical and empiri...
research
02/11/2019

Computing Extremely Accurate Quantiles Using t-Digests

We present on-line algorithms for computing approximations of rank-based...

Please sign up or login with your details

Forgot password? Click here to reset