DeepAI AI Chat
Log In Sign Up

Training Set Debugging Using Trusted Items

by   Xuezhou Zhang, et al.
University of Wisconsin-Madison

Training set bugs are flaws in the data that adversely affect machine learning. The training set is usually too large for man- ual inspection, but one may have the resources to verify a few trusted items. The set of trusted items may not by itself be adequate for learning, so we propose an algorithm that uses these items to identify bugs in the training set and thus im- proves learning. Specifically, our approach seeks the smallest set of changes to the training set labels such that the model learned from this corrected training set predicts labels of the trusted items correctly. We flag the items whose labels are changed as potential bugs, whose labels can be checked for veracity by human experts. To find the bugs in this way is a challenging combinatorial bilevel optimization problem, but it can be relaxed into a continuous optimization problem. Ex- periments on toy and real data demonstrate that our approach can identify training set bugs effectively and suggest appro- priate changes to the labels. Our algorithm is a step toward trustworthy machine learning.


Robust Federated Training via Collaborative Machine Teaching using Trusted Instances

Federated learning performs distributed model training using local data ...

Training Set Camouflage

We introduce a form of steganography in the domain of machine learning w...

Label differential privacy via clustering

We present new mechanisms for label differential privacy, a relaxation o...

Generalization of Machine Learning for Problem Reduction: A Case Study on Travelling Salesman Problems

Combinatorial optimization plays an important role in real-world problem...

Overcoming Conflicting Data for Model Updates

In this paper, we explore how to use a small amount of new data to updat...

A SMART Stochastic Algorithm for Nonconvex Optimization with Applications to Robust Machine Learning

In this paper, we show how to transform any optimization problem that ar...

Modelling the development of counting with memory-augmented neural networks

Learning to count is an important example of the broader human capacity ...