Log In Sign Up

Fair Representation Learning through Implicit Path Alignment

by   Changjian Shui, et al.

We consider a fair representation learning perspective, where optimal predictors, on top of the data representation, are ensured to be invariant with respect to different sub-groups. Specifically, we formulate this intuition as a bi-level optimization, where the representation is learned in the outer-loop, and invariant optimal group predictors are updated in the inner-loop. Moreover, the proposed bi-level objective is demonstrated to fulfill the sufficiency rule, which is desirable in various practical scenarios but was not commonly studied in the fair learning. Besides, to avoid the high computational and memory cost of differentiating in the inner-loop of bi-level objective, we propose an implicit path alignment algorithm, which only relies on the solution of inner optimization and the implicit differentiation rather than the exact optimization path. We further analyze the error gap of the implicit approach and empirically validate the proposed method in both classification and regression settings. Experimental results show the consistently better trade-off in prediction performance and fairness measurement.


page 1

page 2

page 3

page 4


Meta-Learning with Implicit Gradients

A core capability of intelligent systems is the ability to quickly learn...

On Learning Fairness and Accuracy on Multiple Subgroups

We propose an analysis in fair learning that preserves the utility of th...

Debiasing a First-order Heuristic for Approximate Bi-level Optimization

Approximate bi-level optimization (ABLO) consists of (outer-level) optim...

BiOpt: Bi-Level Optimization for Few-Shot Segmentation

Few-shot segmentation is a challenging task that aims to segment objects...

Fair Normalizing Flows

Fair representation learning is an attractive approach that promises fai...

Amortized Implicit Differentiation for Stochastic Bilevel Optimization

We study a class of algorithms for solving bilevel optimization problems...

Mixed Link Networks

Basing on the analysis by revealing the equivalence of modern networks, ...