Probably Approximately Correct Constrained Learning

06/09/2020
by   Luiz F. O. Chamon, et al.
0

As learning solutions reach critical applications in social, industrial, and medical domains, the need to curtail their behavior becomes paramount. There is now ample evidence that without explicit tailoring, learning can lead to biased, unsafe, and prejudiced solutions. To tackle these problems, we develop a generalization theory of constrained learning based on the probably approximately correct (PAC) learning framework. In particular, we show that imposing requirements does not make a learning problem harder in the sense that any PAC learnable class is also PAC constrained learnable using a constrained counterpart of the empirical risk minimization (ERM) rule. For typical parametrized models, however, this learner involves solving a non-convex optimization program for which even obtaining a feasible solution may be hard. To overcome this issue, we prove that under mild conditions the empirical dual problem of constrained learning is also a PAC constrained learner that now leads to a practical constrained learning algorithm. We analyze the generalization properties of this solution and use it to illustrate how constrained learning can address problems in fair and robust classification.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/08/2021

Constrained Learning with Non-Convex Losses

Though learning has become a core technology of modern information proce...
research
07/27/2018

Learnable: Theory vs Applications

Two different views on machine learning problem: Applied learning (machi...
research
03/28/2023

Learnability, Sample Complexity, and Hypothesis Class Complexity for Regression Models

The goal of a learning algorithm is to receive a training data set as in...
research
09/19/2013

Predictive PAC Learning and Process Decompositions

We informally call a stochastic process learnable if it admits a general...
research
01/29/2019

Learning Schatten--Von Neumann Operators

We study the learnability of a class of compact operators known as Schat...
research
08/22/2022

On the non-efficient PAC learnability of acyclic conjunctive queries

This note serves three purposes: (i) we provide a self-contained exposit...
research
02/12/2020

The empirical duality gap of constrained statistical learning

This paper is concerned with the study of constrained statistical learni...

Please sign up or login with your details

Forgot password? Click here to reset