DeepAI
Log In Sign Up

A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization

06/15/2021
by   Risheng Liu, et al.
0

Bi-level optimization model is able to capture a wide range of complex learning tasks with practical interest. Due to the witnessed efficiency in solving bi-level programs, gradient-based methods have gained popularity in the machine learning community. In this work, we propose a new gradient-based solution scheme, namely, the Bi-level Value-Function-based Interior-point Method (BVFIM). Following the main idea of the log-barrier interior-point scheme, we penalize the regularized value function of the lower level problem into the upper level objective. By further solving a sequence of differentiable unconstrained approximation problems, we consequently derive a sequential programming scheme. The numerical advantage of our scheme relies on the fact that, when gradient methods are applied to solve the approximation problem, we successfully avoid computing any expensive Hessian-vector or Jacobian-vector product. We prove the convergence without requiring any convexity assumption on either the upper level or the lower level objective. Experiments demonstrate the efficiency of the proposed BVFIM on non-convex bi-level problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/11/2021

Value-Function-based Sequential Minimization for Bi-level Optimization

Gradient-based Bi-Level Optimization (BLO) methods have been widely appl...
02/16/2021

A Generic Descent Aggregation Framework for Gradient-based Bi-level Optimization

In recent years, gradient-based methods for solving bi-level optimizatio...
02/03/2023

Efficient Gradient Approximation Method for Constrained Bilevel Optimization

Bilevel optimization has been developed for many machine learning tasks ...
06/07/2020

A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton

In recent years, a variety of gradient-based first-order methods have be...
10/01/2021

Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond

In recent years, Bi-Level Optimization (BLO) techniques have received ex...
06/13/2022

Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems

Gradient-based optimization methods for hyperparameter tuning guarantee ...
01/27/2021

Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond

Bi-Level Optimization (BLO) is originated from the area of economic game...