DeepAI AI Chat
Log In Sign Up

Independence Tests Without Ground Truth for Noisy Learners

by   Andrés Corrada-Emmanuel, et al.

Exact ground truth invariant polynomial systems can be written for arbitrarily correlated binary classifiers. Their solutions give estimates for sample statistics that require knowledge of the ground truth of the correct labels in the sample. Of these polynomial systems, only a few have been solved in closed form. Here we discuss the exact solution for independent binary classifiers - resolving an outstanding problem that has been presented at this conference and others. Its practical applicability is hampered by its sole remaining assumption - the classifiers need to be independent in their sample errors. We discuss how to use the closed form solution to create a self-consistent test that can validate the independence assumption itself absent the correct labels ground truth. It can be cast as an algebraic geometry conjecture for binary classifiers that remains unsolved. A similar conjecture for the ground truth invariant algebraic system for scalar regressors is solvable, and we present the solution here. We also discuss experiments on the Penn ML Benchmark classification tasks that provide further evidence that the conjecture may be true for the polynomial system of binary classifiers.


page 1

page 2

page 3

page 4


Algebraic Ground Truth Inference: Non-Parametric Estimation of Sample Errors by AI Algorithms

Binary classification is widely used in ML production systems. Monitorin...

Error Correcting Algorithms for Sparsely Correlated Regressors

Autonomy and adaptation of machines requires that they be able to measur...

Perfect Is the Enemy of Test Oracle

Automation of test oracles is one of the most challenging facets of soft...

Refuting Feder, Kinne and Rafiey

I give an example showing that the recent claimed solution by Feder, Kin...

Condorcet's Jury Theorem for Consensus Clustering and its Implications for Diversity

Condorcet's Jury Theorem has been invoked for ensemble classifiers to in...

Virtual Ground Truth, and Pre-selection of 3D Interest Points for Improved Repeatability Evaluation of 2D Detectors

In Computer Vision, finding simple features is performed using classifie...

Sinusoidal Sensitivity Calculation for Line Segment Geometries

Purpose: Provide a closed-form solution to the sinusoidal coil sensitivi...