Accelerated gradient descent method for functionals of probability measures by new convexity and smoothness based on transport maps

05/09/2023
by   Ken'ichiro Tanaka, et al.
0

We consider problems of minimizing functionals ℱ of probability measures on the Euclidean space. To propose an accelerated gradient descent algorithm for such problems, we consider gradient flow of transport maps that give push-forward measures of an initial measure. Then we propose a deterministic accelerated algorithm by extending Nesterov's acceleration technique with momentum. This algorithm do not based on the Wasserstein geometry. Furthermore, to estimate the convergence rate of the accelerated algorithm, we introduce new convexity and smoothness for ℱ based on transport maps. As a result, we can show that the accelerated algorithm converges faster than a normal gradient descent algorithm. Numerical experiments support this theoretical result.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/13/2023

Accelerated Convergence of Nesterov's Momentum for Deep Neural Networks under Partial Strong Convexity

Current state-of-the-art analyses on the convergence of gradient descent...
research
07/07/2021

Fast and Accurate Optimization of Metasurfaces with Gradient Descent and the Woodbury Matrix Identity

A fast metasurface optimization strategy for finite-size metasurfaces mo...
research
06/14/2017

Accelerated Extra-Gradient Descent: A Novel Accelerated First-Order Method

We provide a novel accelerated first-order method that achieves the asym...
research
02/26/2020

Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization

Due to the high communication cost in distributed and federated learning...
research
05/30/2019

Implicit Regularization of Accelerated Methods in Hilbert Spaces

We study learning properties of accelerated gradient descent methods for...
research
12/14/2017

Stochastic Particle Gradient Descent for Infinite Ensembles

The superior performance of ensemble methods with infinite models are we...
research
08/01/2022

A Particle-Based Algorithm for Distributional Optimization on Constrained Domains via Variational Transport and Mirror Descent

We consider the optimization problem of minimizing an objective function...

Please sign up or login with your details

Forgot password? Click here to reset