Meta Subspace Optimization

10/28/2021
by   Yoni Choukroun, et al.
0

Subspace optimization methods have the attractive property of reducing large-scale optimization problems to a sequence of low-dimensional subspace optimization problems. However, existing subspace optimization frameworks adopt a fixed update policy of the subspace, and therefore, appear to be sub-optimal. In this paper we propose a new Meta Subspace Optimization (MSO) framework for large-scale optimization problems, which allows to determine the subspace matrix at each optimization iteration. In order to remain invariant to the optimization problem's dimension, we design an efficient meta optimizer based on very low-dimensional subspace optimization coefficients, inducing a rule-based agent that can significantly improve performance. Finally, we design and analyze a reinforcement learning procedure based on the subspace optimization dynamics whose learnt policies outperform existing subspace optimization methods.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

08/20/2020

Primal-Dual Sequential Subspace Optimization for Saddle-point Problems

We introduce a new sequential subspace optimization method for large-sca...
10/01/2020

Reducing Subspace Models for Large-Scale Covariance Regression

We develop an envelope model for joint mean and covariance regression in...
09/02/2016

SEBOOST - Boosting Stochastic Learning Using Subspace Optimization Techniques

We present SEBOOST, a technique for boosting the performance of existing...
03/15/2018

Fast Subspace Clustering Based on the Kronecker Product

Subspace clustering is a useful technique for many computer vision appli...
09/25/2019

Accelerating design optimization using reduced order models

Although design optimization has shown its great power of automatizing t...
11/09/2011

Scaling Up Estimation of Distribution Algorithms For Continuous Optimization

Since Estimation of Distribution Algorithms (EDA) were proposed, many at...
12/17/2018

Accelerating Multigrid Optimization via SESOP

A merger of two optimization frameworks is introduced: SEquential Subspa...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.