Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions

05/11/2020
by   Bojan Karlas, et al.
9

Machine learning (ML) applications have been thriving recently, largely attributed to the increasing availability of data. However, inconsistency and incomplete information are ubiquitous in real-world datasets, and their impact on ML applications remains elusive. In this paper, we present a formal study of this impact by extending the notion of Certain Answers for Codd tables, which has been explored by the database research community for decades, into the field of machine learning. Specifically, we focus on classification problems and propose the notion of "Certain Predictions" (CP) – a test data example can be certainly predicted (CP'ed) if all possible classifiers trained on top of all possible worlds induced by the incompleteness of data would yield the same prediction. We study two fundamental CP queries: (Q1) checking query that determines whether a data example can be CP'ed; and (Q2) counting query that computes the number of classifiers that support a particular prediction (i.e., label). Given that general solutions to CP queries are, not surprisingly, hard without assumption over the type of classifier, we further present a case study in the context of nearest neighbor (NN) classifiers, where efficient solutions to CP queries can be developed – we show that it is possible to answer both queries in linear or polynomial time over exponentially many possible worlds. We demonstrate one example use case of CP in the important application of "data cleaning for machine learning (DC for ML)." We show that our proposed CPClean approach built based on CP can often significantly outperform existing techniques in terms of classification accuracy with mild manual cleaning effort.

READ FULL TEXT

page 1

page 2

page 3

page 4

01/13/2022

Certifiable Robustness for Nearest Neighbor Classifiers

ML models are typically trained using large datasets of high quality. Ho...
02/02/2022

Approximating Full Conformal Prediction at Scale via Influence Functions

Conformal prediction (CP) is a wrapper around traditional machine learni...
09/22/2021

Improved Multi-label Classification with Frequent Label-set Mining and Association

Multi-label (ML) data deals with multiple classes associated with indivi...
07/06/2020

Relaxed Conformal Prediction Cascades for Efficient Inference Over Many Labels

Providing a small set of promising candidates in place of a single predi...
02/05/2021

Exact Optimization of Conformal Predictors via Incremental and Decremental Learning

Conformal Predictors (CP) are wrappers around ML methods, providing erro...
10/18/2021

Learning Optimal Conformal Classifiers

Modern deep learning based classifiers show very high accuracy on test d...
06/19/2013

Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifiers

Machine Learning (ML) algorithms are used to train computers to perform ...

Code Repositories

certain-prediction-framework

Queries which determine if a KNN model can make certain predictions when training data is incomplete


view repo