Coordinate-wise Armijo's condition: General case

03/11/2020
by   Tuyen Trung Truong, et al.
0

Let z=(x,y) be coordinates for the product space R^m_1×R^m_2. Let f:R^m_1×R^m_2→R be a C^1 function, and ∇ f=(∂ _xf,∂ _yf) its gradient. Fix 0<α <1. For a point (x,y) ∈R^m_1×R^m_2, a number δ >0 satisfies Armijo's condition at (x,y) if the following inequality holds: f(x-δ∂ _xf,y-δ∂ _yf)-f(x,y)≤ -αδ (||∂ _xf||^2+||∂ _yf||^2). In one previous paper, we proposed the following coordinate-wise Armijo's condition. Fix again 0<α <1. A pair of positive numbers δ _1,δ _2>0 satisfies the coordinate-wise variant of Armijo's condition at (x,y) if the following inequality holds: [f(x-δ _1∂ _xf(x,y), y-δ _2∂ _y f(x,y))]-[f(x,y)]≤ -α (δ _1||∂ _xf(x,y)||^2+δ _2||∂ _yf(x,y)||^2). Previously we applied this condition for functions of the form f(x,y)=f(x)+g(y), and proved various convergent results for them. For a general function, it is crucial - for being able to do real computations - to have a systematic algorithm for obtaining δ _1 and δ _2 satisfying the coordinate-wise version of Armijo's condition, much like Backtracking for the usual Armijo's condition. In this paper we propose such an algorithm, and prove according convergent results. We then analyse and present experimental results for some functions such as f(x,y)=a|x|+y (given by Asl and Overton in connection to Wolfe's method), f(x,y)=x^3 sin (1/x) + y^3 sin(1/y) and Rosenbrock's function.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/18/2019

Coordinate-wise Armijo's condition

Let z=(x,y) be coordinates for the product space R^m_1×R^m_2. Let f:R^m_...
research
01/16/2020

Some convergent results for Backtracking Gradient Descent method on Banach spaces

Our main result concerns the following condition: Condition C. Let X ...
research
03/04/2016

Finding best possible constant for a polynomial inequality

Given a multi-variant polynomial inequality with a parameter, how to fin...
research
07/30/2023

On Approximability of Satisfiable k-CSPs: IV

We prove a stability result for general 3-wise correlations over distrib...
research
02/25/2017

Efficient coordinate-wise leading eigenvector computation

We develop and analyze efficient "coordinate-wise" methods for finding t...
research
10/20/2017

Biased halfspaces, noise sensitivity, and relative Chernoff inequalities (extended version)

In analysis of Boolean functions, a halfspace is a function f:{-1,1}^n →...
research
03/01/2019

Trajectory convergence from coordinate-wise decrease of quadratic energy functions, and applications to platoons

We consider trajectories where the sign of the derivative of each entry ...

Please sign up or login with your details

Forgot password? Click here to reset