Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithms

08/22/2019
by   Ruoxi Jia, et al.
11

Given a data set D containing millions of data points and a data consumer who is willing to pay for X to train a machine learning (ML) model over D, how should we distribute this X to each data point to reflect its "value"? In this paper, we define the "relative value of data" via the Shapley value, as it uniquely possesses properties with appealing real-world interpretations, such as fairness, rationality and decentralizability. For general, bounded utility functions, the Shapley value is known to be challenging to compute: to get Shapley values for all N data points, it requires O(2^N) model evaluations for exact computation and O(N N) for (ϵ, δ)-approximation. In this paper, we focus on one popular family of ML models relying on K-nearest neighbors (KNN). The most surprising result is that for unweighted KNN classifiers and regressors, the Shapley value of all N data points can be computed, exactly, in O(N N) time -- an exponential improvement on computational complexity! Moreover, for (ϵ, δ)-approximation, we are able to develop an algorithm based on Locality Sensitive Hashing (LSH) with only sublinear complexity O(N^h(ϵ,K) N) when ϵ is not too small and K is not too large. We empirically evaluate our algorithms on up to 10 million data points and even our exact algorithm is up to three orders of magnitude faster than the baseline approximation algorithm. The LSH-based approximation algorithm can accelerate the value calculation process even further. We then extend our algorithms to other scenarios such as (1) weighed KNN classifiers, (2) different data points are clustered by different data curators, and (3) there are data analysts providing computation who also requires proper valuation.

READ FULL TEXT
11/17/2019

An Empirical and Comparative Analysis of Data Valuation with Scalable Algorithms

This paper focuses on valuating training data for supervised learning ta...
05/25/2020

On the Problem of p_1^-1 in Locality-Sensitive Hashing

A Locality-Sensitive Hash (LSH) function is called (r,cr,p_1,p_2)-sensit...
01/15/2013

Why Size Matters: Feature Coding as Nystrom Sampling

Recently, the computer vision and machine learning community has been in...
02/26/2018

All nearest neighbor calculation based on Delaunay graphs

When we have two data sets and want to find the nearest neighbour of eac...
02/27/2020

A Distributional Framework for Data Valuation

Shapley value is a classic notion from game theory, historically used to...
12/30/2008

A New Clustering Algorithm Based Upon Flocking On Complex Network

We have proposed a model based upon flocking on a complex network, and t...
07/02/2020

Efficient computation and analysis of distributional Shapley values

Distributional data Shapley value (DShapley) has been recently proposed ...