Geometric Scattering on Manifolds

12/15/2018 ∙ by Michael Perlmutter, et al. ∙ Yale University Michigan State University 0

We present a mathematical model for geometric deep learning based upon a scattering transform defined over manifolds, which generalizes the wavelet scattering transform of Mallat. This geometric scattering transform is (locally) invariant to isometry group actions, and we conjecture that it is stable to actions of the diffeomorphism group.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Convolutional neural networks (CNNs) are revolutionizing imaging science for two and three dimensional images over Euclidean domains. However, many images, and more generally data sets, are intrinsically non-Euclidean and are better modeled through other mathematical structures, such as graphs or manifolds. This has led to the development of geometric deep learning geodeepweb ; Bronstein:geoDeepLearn2017 (and references therein), which refers to a body of research that aims to translate the principles of CNNs to these non-Euclidean structures. In the process, various challenges have arisen, including how to define such networks, how to train them efficiently, and how to analyze their mathematical properties. In this letter we focus primarily on the last question in a relatively general context, akin to several existing methods, which illustrates the fundamental properties of such networks.

Figure 1: The geometric scattering transform. Black: Equivariant intermediate layers. Blue: Invariant output coefficients at each layer.

We present a geometric version of the scattering transform (Figure 1), which is similar to the one introduced in mallat:scattering2012

, but here is defined over manifolds instead of Euclidean space. The Euclidean scattering transform can be, on the one hand, thought of as a mathematical model for standard CNNs, but has, on the other hand, obtained state of the art or near state of the art empirical results in computer vision

mallat:rotoScat2013 ; bruna:invariantScatConvNet2013 ; oyallon:scatObjectClass2014 , audio signal processing bruna:audioTextureSynth2013 ; anden:deepScatSpectrum2014 ; lostanlen:scatPitchSpiral2015 , and even quantum chemistry eickenberg:3DSolidHarmonicScat2017 ; eickenberg:scatMoleculesJCP2018 . In Section 3 we define the geometric scattering transform, and provide results showing it encodes localized isometry invariant descriptions of signal data defined on a manifold. These results generalize local translation and rotation invariance on Euclidean domains. The underlying spectral integral operators that are the foundation of the geometric scattering transform are presented in Section 2. In Section 4 we discuss the stability of the geometric scattering transform to the action of diffeomorphisms. We provide a framework in which to quantify how much a diffeomorphism differs from being an isometry, provide results proving that individual spectral integral operators are Lipschitz stable to diffeomorphism actions within this framework, and conjecture that these results can be extended to the geometric scattering transform. Finally, we provide a short conclusion in Section 5.

1.1 Notation

Let be a smooth, compact, and connected, -dimensional Riemannian manifold without boundary contained in . Let denote the geodesic distance between two points, and let be the Laplace-Beltrami operator on

The eigenfunctions and non-unique eigenvalues of

are denoted by and , respectively. Since is compact, the spectrum of is countable and we may assume that forms an orthonormal basis for . The set of unique eigenvalues of is denoted by , and for we let and

denote the corresponding multiplicities and eigenspaces. For a diffeomorphism

we let be the operator and let

2 Spectral Integral Operators

For a smooth function we define a spectral kernel by


and refer to the integral operator with kernel as a spectral integral operator. It can be verified that

Since is an orthonormal basis for , it follows that

Therefore, if , then is a nonexpansive operator on Operators of this form are analogous to convolution operators defined on since like the latter they are diagonalized in the Fourier basis. To further emphasize this connection, we note the following theorem which shows that spectral integral operators are equivariant with respect to isometries.

Theorem 2.1.

Let be a spectral integral operator. Then, if is an isometry,

for all

The proof of Theorem 2.1 can be found in Appendix B.1. We will consider frame operators that are constructed using a countable family of spectral integral operators. In particular, we assume that we have a low-pass filter , satisfying , and a family of high-pass filters with , which satisfy a Littlewood-Paley type condition


for some A frame analysis operator is then defined by

where and are the spectral integral operators corresponding to and , respectively. Figure 2 illustrates the low pass operator applied to the Stanford bunny manifold.





Figure 2: Illustration of over the Stanford bunny manifold turk2005stanford with Gaussian , three scales, and three Diracs centered at different regions of the manifold.
Proposition 2.2.

Under the Littlewood-Paley condition (2), is a bounded operator from to and for all . In particular, if , then is an isometry.

Remark 2.3.

In coifman:diffWavelets2006 , R. Coifman and M. Maggioni used the heat semigroup to construct a class filters such that the high-pass filters form a wavelet frame and a low-pass filter is chosen so that satisfies (2). This construction can be generalized to the manifold setting after making suitable adjustments to account for the multiplicities of the eigenvalues.

3 The Geometric Scattering Transform

The geometric scattering transform is a nonlinear operator constructed through an alternating cascade of spectral integral operators and nonlinearities. Let be the modulus operator, and for each we let We define an operator , called the one-step scattering propagator, by

The -step scattering propagator is constructed by iteratively applying the one-step scattering propagator. For let be the set of all paths of the form Let denote the empty set, and let denote the set of a all finite paths. For let

and for , we define as the collection of all path propagators with paths in

The scattering transform over a path is defined as the integration of against the low-pass integral operator i.e.

Analogously to , we define

The operator is referred to as the scattering transform. The following proposition shows that is nonexpansive. The proof is nearly identical to (mallat:scattering2012, , Proposition 2.5), and is thus omitted.

Proposition 3.1.

If the Littlewood-Paley condition (2) holds, then

The scattering transform is invariant to the action of the isometry group on the inputted signal up to a factor that depends upon the decay of the low-pass spectral function . If the low-pass spectral function is rapidly decaying and satisfies for some constant and (e.g., the heat kernel), then the following theorem establishes isometric invariance up to the scale .

Theorem 3.2.

Let be an isometry. If the Littlewood-Paley condition (2) holds and for some constant and , then there exists a constant such that

See Appendix B.2 for the proof.

4 Stability to Diffeomorphisms

As stated in Theorem 2.1, spectral integral operators are equivariant to the action of isometries. This fact is crucial to proving Theorem 3.2

because it allows us to estimate


instead of


In mallat:scattering2012 , it is shown that the Euclidean scattering transform is stable to the action of certain diffeomorphisms which are close to being translations. A key step in the proof is a bound on the commutator norm , which then allows the author to bound a quantity analogous to (3) instead of bounding (4) directly. This motivates us to study the commutator of spectral integral operators with for diffeomorphisms which are close to being isometries.

For technical reasons, we will assume that is two-point homogeneous, that is, for any two pairs of points, such that there exists an isometry such that and In order to quantify how far a diffeomorphism differs from being an isometry we will consider two quantities:




We let and note that if is an isometry, then . The following theorem, which is proved in Appendix B.3, bounds the operator norm of in terms of and a quantity depending upon .

Theorem 4.1.

Assume that is two-point homogeneous, and let be a spectral integral operator. Then there exists a constant such that for any diffeomorphism ,



Remark 4.2.

We conjecture that when is constructed to be a wavelet frame as in Remark 2.3, we can use (7) to prove a bound on in terms of . If true, this result would allow us to show that the scattering transform is stable to diffeomorphisms using methods analogous to the ones in mallat:scattering2012 .

5 Conclusion

We have presented a path towards understanding the mathematical properties of geometric deep networks through the notion of the geometric scattering transform. Recently, related analyses for graphs have been presented in zou:graphCNNScat2018 ; gama:diffScatGraphs2018 ; gao:graphScat2018 . We remark that the analysis proposed here applies to compact Riemannian manifolds of arbitrary dimension, thus providing a road map that goes beyond 2D surface or 3D shape matching. Looking ahead, such an approach naturally lends itself to research avenues that synthesize geometric deep learning and geometric data analysis (e.g., manifold learning roweis:lle2000 ; belkin:laplacianEigen2003 ; coifman:diffusionMaps2006 ), which in turn has the potential to bridge the graph and manifold theories for geometric deep networks.


M.H. is partially supported by Alfred P. Sloan Fellowship #FG-2016-6607, DARPA YFA #D16AP00117, and NSF grant #1620216.


  • [1] Federico Monti., Last accessed in 2018.
  • [2] Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: Going beyond Euclidean data. IEEE Signal Processing Magazine, 34(4):18–42, 2017.
  • [3] Stéphane Mallat. Group invariant scattering. Communications on Pure and Applied Mathematics, 65(10):1331–1398, October 2012.
  • [4] Laurent Sifre and Stéphane Mallat. Rotation, scaling and deformation invariant scattering for texture discrimination. In

    The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    , June 2013.
  • [5] Joan Bruna and Stéphane Mallat. Invariant scattering convolution networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1872–1886, August 2013.
  • [6] Edouard Oyallon and Stéphane Mallat. Deep roto-translation scattering for object classification. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2865–2873, 2015.
  • [7] Joan Bruna and Stéphane Mallat. Audio texture synthesis with scattering moments. arXiv:1311.0407, 2013.
  • [8] Joakim Andén and Stéphane Mallat. Deep scattering spectrum. IEEE Transactions on Signal Processing, 62(16):4114–4128, August 2014.
  • [9] Vincent Lostanlen and Stéphane Mallat. Wavelet scattering on the pitch spiral. In Proceedings of the 18th International Conference on Digital Audio Effects, pages 429–432, 2015.
  • [10] Michael Eickenberg, Georgios Exarchakis, Matthew Hirn, and Stéphane Mallat. Solid harmonic wavelet scattering: Predicting quantum molecular energy from invariant descriptors of 3D electronic densities. In Advances in Neural Information Processing Systems 30 (NIPS 2017), pages 6540–6549, 2017.
  • [11] Michael Eickenberg, Georgios Exarchakis, Matthew Hirn, Stéphane Mallat, and Louis Thiry. Solid harmonic wavelet scattering for predictions of molecule properties. Journal of Chemical Physics, 148:241732, 2018.
  • [12] Greg Turk and Marc Levoy. The Stanford bunny, 2005.
  • [13] Ronald R. Coifman and Mauro Maggioni. Diffusion wavelets. Applied and Computational Harmonic Analysis, 21(1):53–94, 2006.
  • [14] Dongmian Zou and Gilad Lerman. Graph convolutional neural networks via scattering. arXiv:1804:00099, 2018.
  • [15] Fernando Gama, Alejandro Ribeiro, and Joan Bruna. Diffusion scattering transforms on graphs. arXiv:1806.08829, 2018.
  • [16] Feng Gao, Guy Wolf, and Matthew Hirn. Graph classification with geometric scattering. arXiv:1810.03068, 2018.
  • [17] Sam T. Roweis and Lawrence K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323–2326, 2000.
  • [18] Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6):1373–1396, 2003.
  • [19] Ronald R. Coifman and Stéphane Lafon. Diffusion maps. Applied and Computational Harmonic Analysis, 21:5–30, 2006.
  • [20] P. Bérard, G. Besson, and S. Gallot. Embedding Riemannian manifolds by their heat kernel. Geometric and Functional Analysis, 4(4):373–398, 1994.
  • [21] Lars Hörmander. The spectral function of an elliptic operator. Acta Math., 121:193–218, 1968.
  • [22] Yiqian Shi and Bin Xu. Gradient estimate of an eigenfunction on a compact Riemannian manifold without boundary. Annals of Global Analysis and Geometry, 38:21–26, 2010.
  • [23] Evariste Giné M. The addition formula for the eigenfunctions of the Laplacian. Advances in Mathematics, 18(1):102–107, 1975.

Appendix A Auxiliary Results

In the following subsections, we will state and prove some auxiliary results that will be needed to prove our main theorems.

a.1 Stability of Spectral Integral Operators to Left Composition

Theorem A.1.

For every spectral integral operator there exists a constant such that for any diffeomorphism



Let be the kernel of . By the definition of and the Cauchy-Schwartz inequality,


where line (8) follows from the fact that for all ,

It follows that and so

Lemma A.3 shows

and therefore

Corollary A.2.

Let be a spectral integral operator. If for some constant and , then there exists a constant such that for any diffeomorphism


[20, Theorem 2.4] proves that for any , , and ,

Integrating both sides over yields:


Using Theorem A.1 and (9) with gives

Lemma A.3.


so that if is the kernel of a spectral integral operator defined as in (1), we may write

Then, there exists a constant such that

As a consequence,


For any such that it is a consequence of Hörmander’s local Weyl law [21] (see also [22]) that


Theorem 1 of [22] shows that


It follows that

Alternatively, Theorem 3.2 of [23] shows that

Substituting this into the above string of inequalities yields

a.2 Commutator Estimate for Radial Kernels

We will say that a kernel operator

is radial if

for some The following theorem establishes a commutator estimate for operators with radial kernels.

Theorem A.4.

Let be a kernel integral operator with a radial kernel for some . Then there exists constants and such that

Here and are defined as in (5) and (6) respectively,



We first compute

Therefore, by the Cauchy-Schwartz inequality,

We may bound the first integral by observing

To bound the second integral observe, that by the mean value theorem and the assumption that is radial, we have

Lastly, since we see that

which completes the proof. ∎

Appendix B The Proof of Theorems

In this section, we will give the proofs of Theorems, 2.1, 3.2, and 4.1. The proofs of Propositions 2.2 and 3.1 are similar to the proofs of Propositions 2.1 and 2.5 in [3] respectively.

b.1 The Proof of Theorem 2.1


Write and , and recall that



Let be an isometry and set . For

, define the vectors

and as

Since is an isometry, and are both orthonormal bases for . Therefore, there exists an unitary matrix (that does not depend upon ) such that

Using this fact, we see that

It follows from (12) that for all . Now writing we have

b.2 The Proof of Theorem 3.2


We rewrite as

Theorem 2.1 proves that spectral integral operators commute with isometries. Since the modulus operator does as well, it follows that and

Since , we see that


and since Corollary A.2 shows that

which completes the proof. ∎

b.3 The Proof of Theorem 4.1


We write and . If is two-point homogeneous and then by the definition of two-point homogeneity there exists an isometry mapping and . Therefore, we may use the proof of Theorem 2.1 to see that . It follows that is radial and so we may write for some .

Applying Theorem A.4, we see that

Lemma A.3 implies that

and since forms an orthonormal basis for it can be checked that

Therefore, the proof is complete since