Second-order Conditional Gradients

02/20/2020
by   Alejandro Carderera, et al.
0

Constrained second-order convex optimization algorithms are the method of choice when a high accuracy solution to a problem is needed, due to the quadratic convergence rates these methods enjoy when close to the optimum. These algorithms require the solution of a constrained quadratic subproblem at every iteration. In the case where the feasible region can only be accessed efficiently through a linear optimization oracle, and computing first-order information about the function, although possible, is costly, the coupling of constrained second-order and conditional gradient algorithms leads to competitive algorithms with solid theoretical guarantees and good numerical performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/28/2015

Newton-Stein Method: An optimization method for GLMs via Stein's Lemma

We consider the problem of efficiently computing the maximum likelihood ...
research
08/08/2018

Random directions stochastic approximation with deterministic perturbations

We introduce deterministic perturbation schemes for the recently propose...
research
11/15/2016

Oracle Complexity of Second-Order Methods for Finite-Sum Problems

Finite-sum optimization problems are ubiquitous in machine learning, and...
research
11/03/2022

Extra-Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods

This work proposes a universal and adaptive second-order method for mini...
research
02/16/2023

FOSI: Hybrid First and Second Order Optimization

Though second-order optimization methods are highly effective, popular a...
research
09/13/2021

Nonlinear matrix recovery using optimization on the Grassmann manifold

We investigate the problem of recovering a partially observed high-rank ...
research
10/27/2019

Towards a theory of non-commutative optimization: geodesic first and second order methods for moment maps and polytopes

This paper initiates a systematic development of a theory of non-commuta...

Please sign up or login with your details

Forgot password? Click here to reset