DeepAI AI Chat
Log In Sign Up

Differentiating through the Fréchet Mean

02/29/2020
by   Aaron Lou, et al.
23

Recent advances in deep representation learning on Riemannian manifolds extend classical deep learning operations to better capture the geometry of the manifold. One possible extension is the Fréchet mean, the generalization of the Euclidean mean; however, it has been difficult to apply because it lacks a closed form with an easily computable derivative. In this paper, we show how to differentiate through the Fréchet mean for arbitrary Riemannian manifolds. Then, focusing on hyperbolic space, we derive explicit gradient expressions and a fast, accurate, and hyperparameter-free Fréchet mean solver. This fully integrates the Fréchet mean into the hyperbolic neural network pipeline. To demonstrate this integration, we present two case studies. First, we apply our Fréchet mean to the existing Hyperbolic Graph Convolutional Network, replacing its projected aggregation to obtain state-of-the-art results on datasets with high hyperbolicity. Second, to demonstrate the Fréchet mean's capacity to generalize Euclidean neural network operations, we develop a hyperbolic batch normalization method that gives an improvement parallel to the one observed in the Euclidean setting.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/01/2020

Ultrahyperbolic Representation Learning

In machine learning, data is usually represented in a (flat) Euclidean s...
04/15/2021

Lorentzian Graph Convolutional Networks

Graph convolutional networks (GCNs) have received considerable research ...
12/01/2021

Diffusion Mean Estimation on the Diagonal of Product Manifolds

Computing sample means on Riemannian manifolds is typically computationa...
02/07/2023

FFHR: Fully and Flexible Hyperbolic Representation for Knowledge Graph Completion

Learning hyperbolic embeddings for knowledge graph (KG) has gained incre...
04/14/2021

A Hyperbolic-to-Hyperbolic Graph Convolutional Network

Hyperbolic graph convolutional networks (GCNs) demonstrate powerful repr...
12/03/2021

Nested Hyperbolic Spaces for Dimensionality Reduction and Hyperbolic NN Design

Hyperbolic neural networks have been popular in the recent past due to t...
05/24/2021

On the pathwidth of hyperbolic 3-manifolds

According to Mostow's celebrated rigidity theorem, the geometry of close...

Code Repositories

Differentiable-Frechet-Mean

[ICML 2020] Differentiating through the Fréchet Mean (https://arxiv.org/abs/2003.00335).


view repo