Geometric Optimisation on Manifolds with Applications to Deep Learning

03/09/2022
by   Mario Lezcano Casado, et al.
0

We design and implement a Python library to help the non-expert using all these powerful tools in a way that is efficient, extensible, and simple to incorporate into the workflow of the data scientist, practitioner, and applied researcher. The algorithms implemented in this library have been designed with usability and GPU efficiency in mind, and they can be added to any PyTorch model with just one extra line of code. We showcase the effectiveness of these tools on an application of optimisation on manifolds in the setting of time series analysis. In this setting, orthogonal and unitary optimisation is used to constraint and regularise recurrent models and avoid vanishing and exploding gradient problems. The algorithms designed for GeoTorch allow us to achieve state of the art results in the standard tests for this family of models. We use tools from comparison geometry to give bounds on quantities that are of interest in optimisation problems. In particular, we build on the work of (Kaul 1976) to give explicit bounds on the norm of the second derivative of the Riemannian exponential.

READ FULL TEXT
research
08/06/2020

Curvature-Dependant Global Convergence Rates for Optimization on Manifolds of Bounded Geometry

We give curvature-dependant convergence rates for the optimization of we...
research
10/28/2021

Deeptime: a Python library for machine learning dynamical models from time series data

Generation and analysis of time-series data is relevant to many quantita...
research
10/10/2022

Rieoptax: Riemannian Optimization in JAX

We present Rieoptax, an open source Python library for Riemannian optimi...
research
01/26/2021

Statistical models and probabilistic methods on Riemannian manifolds

This entry contains the core material of my habilitation thesis, soon to...
research
05/21/2018

geomstats: a Python Package for Riemannian Geometry in Machine Learning

We introduce geomstats, a python package that performs computations on m...
research
12/18/2018

Probabilistic Inference on Noisy Time Series (PINTS)

Time series models are ubiquitous in science, arising in any situation w...
research
12/23/2020

Library of efficient algorithms for phylogenetic analysis

Evolutionary relationships between species are usually inferred through ...

Please sign up or login with your details

Forgot password? Click here to reset