Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals

09/11/2018
by   Andrew Cotter, et al.
Google
cornell university
0

We show that many machine learning goals, such as improved fairness metrics, can be expressed as constraints on the model's predictions, which we call rate constraints. We study the problem of training non-convex models subject to these rate constraints (or any non-convex and non-differentiable constraints). In the non-convex setting, the standard approach of Lagrange multipliers may fail. Furthermore, if the constraints are non-differentiable, then one cannot optimize the Lagrangian with gradient-based methods. To solve these issues, we introduce the proxy-Lagrangian formulation. This new formulation leads to an algorithm that produces a stochastic classifier by playing a two-player non-zero-sum game solving for what we call a semi-coarse correlated equilibrium, which in turn corresponds to an approximately optimal and feasible solution to the constrained optimization problem. We then give a procedure which shrinks the randomized solution down to one that is a mixture of at most m+1 deterministic solutions, given m constraints. This culminates in algorithms that can solve non-convex constrained optimization problems with possibly non-differentiable and non-convex constraints with theoretical guarantees. We provide extensive experimental results enforcing a wide range of policy goals including different fairness metrics, and other goals on accuracy, coverage, recall, and churn.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/17/2018

Two-Player Games for Efficient Non-Convex Constrained Optimization

In recent years, constrained optimization has become increasingly releva...
06/29/2018

Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints

Classifiers can be trained with data-dependent constraints to satisfy fa...
09/06/2019

Optimizing Generalized Rate Metrics through Game Equilibrium

We present a general framework for solving a large class of learning pro...
10/24/2021

Integrated Conditional Estimation-Optimization

Many real-world optimization problems involve uncertain parameters with ...
05/18/2021

Achieving Fairness with a Simple Ridge Penalty

Estimating a fair linear regression model subject to a user-defined leve...
10/18/2022

Consistent Multiclass Algorithms for Complex Metrics and Constraints

We present consistent algorithms for multiclass learning with complex pe...
01/09/2021

Rate Allocation and Content Placement in Cache Networks

We introduce the problem of optimal congestion control in cache networks...

Please sign up or login with your details

Forgot password? Click here to reset