Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals

by   Andrew Cotter, et al.
cornell university

We show that many machine learning goals, such as improved fairness metrics, can be expressed as constraints on the model's predictions, which we call rate constraints. We study the problem of training non-convex models subject to these rate constraints (or any non-convex and non-differentiable constraints). In the non-convex setting, the standard approach of Lagrange multipliers may fail. Furthermore, if the constraints are non-differentiable, then one cannot optimize the Lagrangian with gradient-based methods. To solve these issues, we introduce the proxy-Lagrangian formulation. This new formulation leads to an algorithm that produces a stochastic classifier by playing a two-player non-zero-sum game solving for what we call a semi-coarse correlated equilibrium, which in turn corresponds to an approximately optimal and feasible solution to the constrained optimization problem. We then give a procedure which shrinks the randomized solution down to one that is a mixture of at most m+1 deterministic solutions, given m constraints. This culminates in algorithms that can solve non-convex constrained optimization problems with possibly non-differentiable and non-convex constraints with theoretical guarantees. We provide extensive experimental results enforcing a wide range of policy goals including different fairness metrics, and other goals on accuracy, coverage, recall, and churn.


page 1

page 2

page 3

page 4


Two-Player Games for Efficient Non-Convex Constrained Optimization

In recent years, constrained optimization has become increasingly releva...

Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints

Classifiers can be trained with data-dependent constraints to satisfy fa...

Optimizing Generalized Rate Metrics through Game Equilibrium

We present a general framework for solving a large class of learning pro...

Integrated Conditional Estimation-Optimization

Many real-world optimization problems involve uncertain parameters with ...

Achieving Fairness with a Simple Ridge Penalty

Estimating a fair linear regression model subject to a user-defined leve...

Consistent Multiclass Algorithms for Complex Metrics and Constraints

We present consistent algorithms for multiclass learning with complex pe...

Rate Allocation and Content Placement in Cache Networks

We introduce the problem of optimal congestion control in cache networks...

Please sign up or login with your details

Forgot password? Click here to reset