Learning Variational Models with Unrolling and Bilevel Optimization

09/26/2022
by   Christoph Brauer, et al.
0

In this paper we consider the problem learning of variational models in the context of supervised learning via risk minimization. Our goal is to provide a deeper understanding of the two approaches of learning of variational models via bilevel optimization and via algorithm unrolling. The former considers the variational model as a lower level optimization problem below the risk minimization problem, while the latter replaces the lower level optimization problem by an algorithm that solves said problem approximately. Both approaches are used in practice, but, unrolling is much simpler from a computational point of view. To analyze and compare the two approaches, we consider a simple toy model, and compute all risks and the respective estimators explicitly. We show that unrolling can be better than the bilevel optimization approach, but also that the performance of unrolling can depend significantly on further parameters, sometimes in unexpected ways: While the stepsize of the unrolled algorithm matters a lot, the number of unrolled iterations only matters if the number is even or odd, and these two cases are notably different.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/10/2021

Learning from Satisfying Assignments Using Risk Minimization

In this paper we consider the problem of Learning from Satisfying Assign...
research
02/17/2023

A Near-Optimal Algorithm for Bilevel Empirical Risk Minimization

Bilevel optimization problems, which are problems where two optimization...
research
06/19/2015

Approximate Inference with the Variational Holder Bound

We introduce the Variational Holder (VH) bound as an alternative to Vari...
research
10/18/2022

Explainable bilevel optimization: an application to the Helsinki deblur challenge

In this paper we present a bilevel optimization scheme for the solution ...
research
09/08/2018

Non-Parametric Variational Inference with Graph Convolutional Networks for Gaussian Processes

Inference for GP models with non-Gaussian noises is computationally expe...
research
06/10/2020

Variational Optimization for the Submodular Maximum Coverage Problem

We examine the submodular maximum coverage problem (SMCP), which is rela...
research
06/23/2020

Inexact Derivative-Free Optimization for Bilevel Learning

Variational regularization techniques are dominant in the field of mathe...

Please sign up or login with your details

Forgot password? Click here to reset