Normalizing Flows on Riemannian Manifolds

11/07/2016

∙

by Mevlana C. Gemici, et al.

∙

Google

∙

We consider the problem of density estimation on Riemannian manifolds. Density estimation on manifolds has many applications in fluid-mechanics, optics and plasma physics and it appears often when dealing with angular variables (such as used in protein folding, robot limbs, gene-expression) and in general directional statistics. In spite of the multitude of algorithms available for density estimation in the Euclidean spaces R^n that scale to large n (e.g. normalizing flows, kernel methods and variational approximations), most of these methods are not immediately suitable for density estimation in more general Riemannian manifolds. We revisit techniques related to homeomorphisms from differential geometry for projecting densities to sub-manifolds and use it to generalize the idea of normalizing flows to more general Riemannian manifolds. The resulting algorithm is scalable, simple to implement and suitable for use with automatic differentiation. We demonstrate concrete examples of this method on the n-sphere S^n.

READ FULL TEXT VIEW PDF

References

(1) D. J. Rezende, S. Mohamed, and D. Wierstra.

Stochastic backpropagation and approximate inference in deep generative models.

In ICML, 2014.

(2) D. P. Kingma and M. Welling. Auto-encoding variational Bayes. In ICLR, 2014.

(3) Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, and Daan Wierstra.

Draw: A recurrent neural network for image generation.

In ICML, 2015.

(4) SM Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, Koray Kavukcuoglu, and Geoffrey E Hinton. Attend, infer, repeat: Fast scene understanding with generative models. arXiv preprint arXiv:1603.08575, 2016.

(5) Danilo Jimenez Rezende, Shakir Mohamed, Ivo Danihelka, Karol Gregor, and Daan Wierstra. One-shot generalization in deep generative models. In ICML, 2016.

(6) Matthew D. Hoffman, David M. Blei, Chong Wang, and John Paisley. Stochastic variational inference.

Journal of Machine Learning Research

, 14:1303–1347, 2013.

(7) Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. arXiv preprint arXiv:1505.05770, 2015.

(8) Diederik P. Kingma, Tim Salimans, and Max Welling. Improving variational inference with inverse autoregressive flow. CoRR, abs/1606.04934, 2016.

(9) Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. 2016.

(10) Tim Salimans, Diederik P. Kingma, and Max Welling. Markov chain monte carlo and variational inference: Bridging the gap. In Francis R. Bach and David M. Blei, editors, ICML, volume 37 of JMLR Workshop and Conference Proceedings, pages 1218–1226. JMLR.org, 2015.

(11) Arindam Banerjee, Inderjit S. Dhillon, Joydeep Ghosh, and Suvrit Sra. Clustering on the unit hypersphere using von mises-fisher distributions. J. Mach. Learn. Res., 6:1345–1382, December 2005.

(12) Siddharth Gopal and Yiming Yang. Von mises-fisher clustering models. In Tony Jebara and Eric P. Xing, editors, Proceedings of the 31st International Conference on Machine Learning (ICML-14), pages 154–162. JMLR Workshop and Conference Proceedings, 2014.

(13) Marco Fraccaro, Ulrich Paquet, and Ole Winther. Indexable probabilistic matrix factorization for maximum inner product search. In

Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA.

, pages 1554–1560, 2016.

(14) Arindam Banerjee, Inderjit Dhillon, Joydeep Ghosh, and Suvrit Sra. Generative model-based clustering of directional data. In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’03, pages 19–28, New York, NY, USA, 2003. ACM.

(15) Alex Graves. Stochastic backpropagation through mixture density distributions. CoRR, abs/1607.05690, 2016.

(16) Lars Maaloe, Casper Kaae Sonderby, Soren Kaae Sonderby, and Ole Winther. Auxiliary deep generative models. CoRR, abs/1602.05473, 2016.

(17) Scott W. Linderman David M. Blei Christian A. Naesseth, Francisco J. R. Ruiz. Rejection sampling variational inference. 2016.

(18) Adi Ben-Israel. The change-of-variables formula using matrix volume. SIAM Journal on Matrix Analysis and Applications, 21(1):300–312, 1999.

(19) Adi Ben-Israel. An application of the matrix volume in probability. Linear Algebra and its Applications, 321(1):9–25, 2000.

(20) Marcel Berger and Bernard Gostiaux. Differential Geometry: Manifolds, Curves, and Surfaces: Manifolds, Curves, and Surfaces, volume 115. Springer Science & Business Media, 2012.

Normalizing Flows on Riemannian Manifolds

Authors

Strong Uniform Consistency with Rates for Kernel Density Estimators with General Kernels on Manifolds

Manifold Density Estimation via Generalized Dequantization

Implicit Riemannian Concave Potential Maps

Riemannian Continuous Normalizing Flows

Gaussians on Riemannian Manifolds for Robot Learning and Adaptive Control

Moser Flow: Divergence-based Generative Modeling on Manifolds

Simulation of Conditioned Semimartingales on Riemannian Manifolds

References

Normalizing Flows on Riemannian Manifolds

Authors

Related Research

Strong Uniform Consistency with Rates for Kernel Density Estimators with General Kernels on Manifolds

Manifold Density Estimation via Generalized Dequantization

Implicit Riemannian Concave Potential Maps

Riemannian Continuous Normalizing Flows

Gaussians on Riemannian Manifolds for Robot Learning and Adaptive Control

Moser Flow: Divergence-based Generative Modeling on Manifolds

Simulation of Conditioned Semimartingales on Riemannian Manifolds

This week in AI

References