Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints

by   Andrew Cotter, et al.

Classifiers can be trained with data-dependent constraints to satisfy fairness goals, reduce churn, achieve a targeted false positive rate, or other policy goals. We study the generalization performance for such constrained optimization problems, in terms of how well the constraints are satisfied at evaluation time, given that they are satisfied at training time. To improve generalization performance, we frame the problem as a two-player game where one player optimizes the model parameters on a training dataset, and the other player enforces the constraints on an independent validation dataset. We build on recent work in two-player constrained optimization to show that if one uses this two-dataset approach, then constraint generalization can be significantly improved. As we illustrate experimentally, this approach works not only in theory, but also in practice.


page 1

page 2

page 3

page 4


Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals

We show that many machine learning goals, such as improved fairness metr...

Calibrated Data-Dependent Constraints with Exact Satisfaction Guarantees

We consider the task of training machine learning models with data-depen...

Optimizing Generalized Rate Metrics through Game Equilibrium

We present a general framework for solving a large class of learning pro...

Two-Player Games for Efficient Non-Convex Constrained Optimization

In recent years, constrained optimization has become increasingly releva...

Sufficient reasons for classifier decisions in the presence of constraints

Recent work has unveiled a theory for reasoning about the decisions made...

Optimal Training of Fair Predictive Models

Recently there has been sustained interest in modifying prediction algor...

FairNeuron: Improving Deep Neural Network Fairness with Adversary Games on Selective Neurons

With Deep Neural Network (DNN) being integrated into a growing number of...

Please sign up or login with your details

Forgot password? Click here to reset