Randomized Iterative Methods for Linear Systems: Momentum, Inexactness and Gossip

09/26/2019
by   Nicolas Loizou, et al.
0

In the era of big data, one of the key challenges is the development of novel optimization algorithms that can accommodate vast amounts of data while at the same time satisfying constraints and limitations of the problem under study. The need to solve optimization problems is ubiquitous in essentially all quantitative areas of human endeavor, including industry and science. In the last decade there has been a surge in the demand from practitioners, in fields such as machine learning, computer vision, artificial intelligence, signal processing and data science, for new methods able to cope with these new large scale problems. In this thesis we are focusing on the design, complexity analysis and efficient implementations of such algorithms. In particular, we are interested in the development of randomized iterative methods for solving large scale linear systems, stochastic quadratic optimization problems, the best approximation problem and quadratic optimization problems. A large part of the thesis is also devoted to the development of efficient methods for obtaining average consensus on large scale networks.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/26/2020

Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters

Many key problems in machine learning and data science are routinely mod...
06/30/2019

A Note On The Popularity of Stochastic Optimization Algorithms in Different Fields: A Quantitative Analysis from 2007 to 2017

Stochastic optimization algorithms are often used to solve complex large...
06/22/2021

Faster Randomized Methods for Orthogonality Constrained Problems

Recent literature has advocated the use of randomized methods for accele...
09/23/2019

Machine Learning Optimization Algorithms & Portfolio Allocation

Portfolio optimization emerged with the seminal paper of Markowitz (1952...
10/06/2011

Runtime Guarantees for Regression Problems

We study theoretical runtime guarantees for a class of optimization prob...
05/11/2021

Frank-Wolfe Methods in Probability Space

We introduce a new class of Frank-Wolfe algorithms for minimizing differ...
02/05/2018

Analytical Cost Metrics : Days of Future Past

As we move towards the exascale era, the new architectures must be capab...