On Radial Isotropic Position: Theory and Algorithms

05/11/2020
by   Shiri Artstein-Avidan, et al.
0

We review the theory of, and develop algorithms for transforming a finite point set in R^d into a set in radial isotropic position by a nonsingular linear transformation followed by rescaling each image point to the unit sphere. This problem arises in a wide spectrum of applications in computer science and mathematics. Our algorithms use gradient descent methods for a particular convex function f whose minimum defines the transformation, and our main focus is on analyzing their performance. Although the minimum can be computed exactly, by expensive symbolic algebra techniques, gradient descent only approximates the desired minimum to any desired level of accuracy. We show that computing the gradient of f amounts to computing the Singular Value Decomposition (SVD) of a certain matrix associated with the input set, making it simple to implement. We believe it to be superior to other approximate techniques (mainly the ellipsoid algorithm) used previously to find this transformation, and it should run much faster in practice. We prove that f is smooth, which yields convergence rate proportional to 1/ϵ, where ϵ is the desired approximation accuracy. To complete the analysis, we provide upper bounds on the norm of the optimal solution which depend on new parameters measuring "the degeneracy" in our input. We believe that our parameters capture degeneracy better than other, seemingly weaker, parameters used in previous works. We next analyze the strong convexity of f, and present two worst-case lower bounds on the smallest eigenvalue of its Hessian. This gives another worst-case bound on the convergence rate of another variant of gradient decent that depends only logarithmically on 1/ϵ.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/01/2020

Least-squares regressions via randomized Hessians

We consider the least-squares regression problem with a finite number of...
research
08/07/2017

Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls

We propose a rank-k variant of the classical Frank-Wolfe algorithm to so...
research
08/02/2019

Path Length Bounds for Gradient Descent and Flow

We provide path length bounds on gradient descent (GD) and flow (GF) cur...
research
05/29/2021

On Centralized and Distributed Mirror Descent: Exponential Convergence Analysis Using Quadratic Constraints

Mirror descent (MD) is a powerful first-order optimization technique tha...
research
01/01/2023

Sharper analysis of sparsely activated wide neural networks with trainable biases

This work studies training one-hidden-layer overparameterized ReLU netwo...
research
05/16/2023

The Average Rate of Convergence of the Exact Line Search Gradient Descent Method

It is very well-known that when the exact line search gradient descent m...
research
06/01/2018

SFA-GTM: Seismic Facies Analysis Based on Generative Topographic Map and RBF

Seismic facies identification plays a significant role in reservoir char...

Please sign up or login with your details

Forgot password? Click here to reset