Implicit Bilevel Optimization: Differentiating through Bilevel Optimization Programming

02/28/2023
by   Francesco Alesiani, et al.
0

Bilevel Optimization Programming is used to model complex and conflicting interactions between agents, for example in Robust AI or Privacy-preserving AI. Integrating bilevel mathematical programming within deep learning is thus an essential objective for the Machine Learning community. Previously proposed approaches only consider single-level programming. In this paper, we extend existing single-level optimization programming approaches and thus propose Differentiating through Bilevel Optimization Programming (BiGrad) for end-to-end learning of models that use Bilevel Programming as a layer. BiGrad has wide applicability and can be used in modern machine learning frameworks. BiGrad is applicable to both continuous and combinatorial Bilevel optimization problems. We describe a class of gradient estimators for the combinatorial case which reduces the requirements in terms of computation complexity; for the case of the continuous variable, the gradient computation takes advantage of the push-back approach (i.e. vector-jacobian product) for an efficient implementation. Experiments show that the BiGrad successfully extends existing single-level approaches to Bilevel Programming.

READ FULL TEXT
research
11/22/2019

Smart Predict-and-Optimize for Hard Combinatorial Optimization Problems

Combinatorial optimization assumes that all parameters of the optimizati...
research
12/31/2020

Differentiable Programming à la Moreau

The notion of a Moreau envelope is central to the analysis of first-orde...
research
08/05/2022

A Model-Oriented Approach for Lifting Symmetries in Answer Set Programming

When solving combinatorial problems, pruning symmetric solution candidat...
research
04/06/2021

Ecole: A Library for Learning Inside MILP Solvers

In this paper we describe Ecole (Extensible Combinatorial Optimization L...
research
12/04/2020

Divide and Learn: A Divide and Conquer Approach for Predict+Optimize

The predict+optimize problem combines machine learning ofproblem coeffic...
research
05/14/2021

How to effectively use machine learning models to predict the solutions for optimization problems: lessons from loss function

Using machine learning in solving constraint optimization and combinator...
research
03/17/2022

Learning Distributionally Robust Models at Scale via Composite Optimization

To train machine learning models that are robust to distribution shifts ...

Please sign up or login with your details

Forgot password? Click here to reset