Gradient flows on graphons: existence, convergence, continuity equations

by   Sewoong Oh, et al.

Wasserstein gradient flows on probability measures have found a host of applications in various optimization problems. They typically arise as the continuum limit of exchangeable particle systems evolving by some mean-field interaction involving a gradient-type potential. However, in many problems, such as in multi-layer neural networks, the so-called particles are edge weights on large graphs whose nodes are exchangeable. Such large graphs are known to converge to continuum limits called graphons as their size grow to infinity. We show that the Euclidean gradient flow of a suitable function of the edge-weights converges to a novel continuum limit given by a curve on the space of graphons that can be appropriately described as a gradient flow or, more technically, a curve of maximal slope. Several natural functions on graphons, such as homomorphism functions and the scalar entropy, are covered by our set-up, and the examples have been worked out in detail.


page 1

page 2

page 3

page 4


Nonlocal-interaction equation on graphs: gradient flow structure and continuum limit

We consider dynamics driven by interaction energies on graphs. We introd...

Path convergence of Markov chains on large graphs

We consider two classes of natural stochastic processes on finite unlabe...

Wasserstein Mirror Gradient Flow as the limit of the Sinkhorn Algorithm

We prove that the sequence of marginals obtained from the iterations of ...

Global Convergence of Second-order Dynamics in Two-layer Neural Networks

Recent results have shown that for two-layer fully connected neural netw...

A Riemannian Mean Field Formulation for Two-layer Neural Networks with Batch Normalization

The training dynamics of two-layer neural networks with batch normalizat...

A Distributed Algorithm for Measure-valued Optimization with Additive Objective

We propose a distributed nonparametric algorithm for solving measure-val...

Stochastic optimization on matrices and a graphon McKean-Vlasov limit

We consider stochastic gradient descents on the space of large symmetric...

Please sign up or login with your details

Forgot password? Click here to reset