Towards Constituting Mathematical Structures for Learning to Optimize

05/29/2023
by   Jialin Liu, et al.
0

Learning to Optimize (L2O), a technique that utilizes machine learning to learn an optimization algorithm automatically from data, has gained arising attention in recent years. A generic L2O approach parameterizes the iterative update rule and learns the update direction as a black-box network. While the generic approach is widely applicable, the learned model can overfit and may not generalize well to out-of-distribution test sets. In this paper, we derive the basic mathematical conditions that successful update rules commonly satisfy. Consequently, we propose a novel L2O model with a mathematics-inspired structure that is broadly applicable and generalized well to out-of-distribution problems. Numerical simulations validate our theoretical findings and demonstrate the superior empirical performance of the proposed L2O model.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/09/2019

Stochastic Implicit Natural Gradient for Black-box Optimization

Black-box optimization is primarily important for many compute-intensive...
research
06/22/2017

MAGIX: Model Agnostic Globally Interpretable Explanations

Explaining the behavior of a black box machine learning model at the ins...
research
10/21/2019

Learning to Learn by Zeroth-Order Oracle

In the learning to learn (L2L) framework, we cast the design of optimiza...
research
06/26/2021

Quantum Dynamics Interpretation of Black-box Optimization

In recent decades, with the emergence of numerous novel intelligent opti...
research
01/15/2021

Machine-Learning Mathematical Structures

We review, for a general audience, a variety of recent experiments on ex...
research
03/13/2022

Symbolic Learning to Optimize: Towards Interpretability and Scalability

Recent studies on Learning to Optimize (L2O) suggest a promising path to...
research
08/25/2020

t-Soft Update of Target Network for Deep Reinforcement Learning

This paper proposes a new robust update rule of the target network for d...

Please sign up or login with your details

Forgot password? Click here to reset