Convolutional neural networks (CNNs) are revolutionizing imaging science for two and three dimensional images over Euclidean domains. However, many images, and more generally data sets, are intrinsically non-Euclidean and are better modeled through other mathematical structures, such as graphs or manifolds. This has led to the development of geometric deep learning geodeepweb ; Bronstein:geoDeepLearn2017 (and references therein), which refers to a body of research that aims to translate the principles of CNNs to these non-Euclidean structures. In the process, various challenges have arisen, including how to define such networks, how to train them efficiently, and how to analyze their mathematical properties. In this letter we focus primarily on the last question in a relatively general context, akin to several existing methods, which illustrates the fundamental properties of such networks.
, but here is defined over manifolds instead of Euclidean space. The Euclidean scattering transform can be, on the one hand, thought of as a mathematical model for standard CNNs, but has, on the other hand, obtained state of the art or near state of the art empirical results in computer visionmallat:rotoScat2013 ; bruna:invariantScatConvNet2013 ; oyallon:scatObjectClass2014 , audio signal processing bruna:audioTextureSynth2013 ; anden:deepScatSpectrum2014 ; lostanlen:scatPitchSpiral2015 , and even quantum chemistry eickenberg:3DSolidHarmonicScat2017 ; eickenberg:scatMoleculesJCP2018 . In Section 3 we define the geometric scattering transform, and provide results showing it encodes localized isometry invariant descriptions of signal data defined on a manifold. These results generalize local translation and rotation invariance on Euclidean domains. The underlying spectral integral operators that are the foundation of the geometric scattering transform are presented in Section 2. In Section 4 we discuss the stability of the geometric scattering transform to the action of diffeomorphisms. We provide a framework in which to quantify how much a diffeomorphism differs from being an isometry, provide results proving that individual spectral integral operators are Lipschitz stable to diffeomorphism actions within this framework, and conjecture that these results can be extended to the geometric scattering transform. Finally, we provide a short conclusion in Section 5.
Let be a smooth, compact, and connected, -dimensional Riemannian manifold without boundary contained in . Let denote the geodesic distance between two points, and let be the Laplace-Beltrami operator onare denoted by and , respectively. Since is compact, the spectrum of is countable and we may assume that forms an orthonormal basis for . The set of unique eigenvalues of is denoted by , and for we let and
denote the corresponding multiplicities and eigenspaces. For a diffeomorphismwe let be the operator and let
2 Spectral Integral Operators
For a smooth function we define a spectral kernel by
and refer to the integral operator with kernel as a spectral integral operator. It can be verified that
Since is an orthonormal basis for , it follows that
Therefore, if , then is a nonexpansive operator on Operators of this form are analogous to convolution operators defined on since like the latter they are diagonalized in the Fourier basis. To further emphasize this connection, we note the following theorem which shows that spectral integral operators are equivariant with respect to isometries.
Let be a spectral integral operator. Then, if is an isometry,
The proof of Theorem 2.1 can be found in Appendix B.1. We will consider frame operators that are constructed using a countable family of spectral integral operators. In particular, we assume that we have a low-pass filter , satisfying , and a family of high-pass filters with , which satisfy a Littlewood-Paley type condition
for some A frame analysis operator is then defined by
where and are the spectral integral operators corresponding to and , respectively. Figure 2 illustrates the low pass operator applied to the Stanford bunny manifold.
Under the Littlewood-Paley condition (2), is a bounded operator from to and for all . In particular, if , then is an isometry.
In coifman:diffWavelets2006 , R. Coifman and M. Maggioni used the heat semigroup to construct a class filters such that the high-pass filters form a wavelet frame and a low-pass filter is chosen so that satisfies (2). This construction can be generalized to the manifold setting after making suitable adjustments to account for the multiplicities of the eigenvalues.
3 The Geometric Scattering Transform
The geometric scattering transform is a nonlinear operator constructed through an alternating cascade of spectral integral operators and nonlinearities. Let be the modulus operator, and for each we let We define an operator , called the one-step scattering propagator, by
The -step scattering propagator is constructed by iteratively applying the one-step scattering propagator. For let be the set of all paths of the form Let denote the empty set, and let denote the set of a all finite paths. For let
and for , we define as the collection of all path propagators with paths in
The scattering transform over a path is defined as the integration of against the low-pass integral operator i.e.
Analogously to , we define
The operator is referred to as the scattering transform. The following proposition shows that is nonexpansive. The proof is nearly identical to (mallat:scattering2012, , Proposition 2.5), and is thus omitted.
If the Littlewood-Paley condition (2) holds, then
The scattering transform is invariant to the action of the isometry group on the inputted signal up to a factor that depends upon the decay of the low-pass spectral function . If the low-pass spectral function is rapidly decaying and satisfies for some constant and (e.g., the heat kernel), then the following theorem establishes isometric invariance up to the scale .
Let be an isometry. If the Littlewood-Paley condition (2) holds and for some constant and , then there exists a constant such that
See Appendix B.2 for the proof.
4 Stability to Diffeomorphisms
because it allows us to estimate
In mallat:scattering2012 , it is shown that the Euclidean scattering transform is stable to the action of certain diffeomorphisms which are close to being translations. A key step in the proof is a bound on the commutator norm , which then allows the author to bound a quantity analogous to (3) instead of bounding (4) directly. This motivates us to study the commutator of spectral integral operators with for diffeomorphisms which are close to being isometries.
For technical reasons, we will assume that is two-point homogeneous, that is, for any two pairs of points, such that there exists an isometry such that and In order to quantify how far a diffeomorphism differs from being an isometry we will consider two quantities:
We let and note that if is an isometry, then . The following theorem, which is proved in Appendix B.3, bounds the operator norm of in terms of and a quantity depending upon .
Assume that is two-point homogeneous, and let be a spectral integral operator. Then there exists a constant such that for any diffeomorphism ,
We have presented a path towards understanding the mathematical properties of geometric deep networks through the notion of the geometric scattering transform. Recently, related analyses for graphs have been presented in zou:graphCNNScat2018 ; gama:diffScatGraphs2018 ; gao:graphScat2018 . We remark that the analysis proposed here applies to compact Riemannian manifolds of arbitrary dimension, thus providing a road map that goes beyond 2D surface or 3D shape matching. Looking ahead, such an approach naturally lends itself to research avenues that synthesize geometric deep learning and geometric data analysis (e.g., manifold learning roweis:lle2000 ; belkin:laplacianEigen2003 ; coifman:diffusionMaps2006 ), which in turn has the potential to bridge the graph and manifold theories for geometric deep networks.
M.H. is partially supported by Alfred P. Sloan Fellowship #FG-2016-6607, DARPA YFA #D16AP00117, and NSF grant #1620216.
-  Federico Monti. http://geometricdeeplearning.com/, Last accessed in 2018.
-  Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: Going beyond Euclidean data. IEEE Signal Processing Magazine, 34(4):18–42, 2017.
-  Stéphane Mallat. Group invariant scattering. Communications on Pure and Applied Mathematics, 65(10):1331–1398, October 2012.
Laurent Sifre and Stéphane Mallat.
Rotation, scaling and deformation invariant scattering for texture
The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013.
-  Joan Bruna and Stéphane Mallat. Invariant scattering convolution networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1872–1886, August 2013.
-  Edouard Oyallon and Stéphane Mallat. Deep roto-translation scattering for object classification. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2865–2873, 2015.
-  Joan Bruna and Stéphane Mallat. Audio texture synthesis with scattering moments. arXiv:1311.0407, 2013.
-  Joakim Andén and Stéphane Mallat. Deep scattering spectrum. IEEE Transactions on Signal Processing, 62(16):4114–4128, August 2014.
-  Vincent Lostanlen and Stéphane Mallat. Wavelet scattering on the pitch spiral. In Proceedings of the 18th International Conference on Digital Audio Effects, pages 429–432, 2015.
-  Michael Eickenberg, Georgios Exarchakis, Matthew Hirn, and Stéphane Mallat. Solid harmonic wavelet scattering: Predicting quantum molecular energy from invariant descriptors of 3D electronic densities. In Advances in Neural Information Processing Systems 30 (NIPS 2017), pages 6540–6549, 2017.
-  Michael Eickenberg, Georgios Exarchakis, Matthew Hirn, Stéphane Mallat, and Louis Thiry. Solid harmonic wavelet scattering for predictions of molecule properties. Journal of Chemical Physics, 148:241732, 2018.
-  Greg Turk and Marc Levoy. The Stanford bunny, 2005.
-  Ronald R. Coifman and Mauro Maggioni. Diffusion wavelets. Applied and Computational Harmonic Analysis, 21(1):53–94, 2006.
-  Dongmian Zou and Gilad Lerman. Graph convolutional neural networks via scattering. arXiv:1804:00099, 2018.
-  Fernando Gama, Alejandro Ribeiro, and Joan Bruna. Diffusion scattering transforms on graphs. arXiv:1806.08829, 2018.
-  Feng Gao, Guy Wolf, and Matthew Hirn. Graph classification with geometric scattering. arXiv:1810.03068, 2018.
-  Sam T. Roweis and Lawrence K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323–2326, 2000.
-  Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6):1373–1396, 2003.
-  Ronald R. Coifman and Stéphane Lafon. Diffusion maps. Applied and Computational Harmonic Analysis, 21:5–30, 2006.
-  P. Bérard, G. Besson, and S. Gallot. Embedding Riemannian manifolds by their heat kernel. Geometric and Functional Analysis, 4(4):373–398, 1994.
-  Lars Hörmander. The spectral function of an elliptic operator. Acta Math., 121:193–218, 1968.
-  Yiqian Shi and Bin Xu. Gradient estimate of an eigenfunction on a compact Riemannian manifold without boundary. Annals of Global Analysis and Geometry, 38:21–26, 2010.
-  Evariste Giné M. The addition formula for the eigenfunctions of the Laplacian. Advances in Mathematics, 18(1):102–107, 1975.
Appendix A Auxiliary Results
In the following subsections, we will state and prove some auxiliary results that will be needed to prove our main theorems.
a.1 Stability of Spectral Integral Operators to Left Composition
For every spectral integral operator there exists a constant such that for any diffeomorphism
Let be a spectral integral operator. If for some constant and , then there exists a constant such that for any diffeomorphism
so that if is the kernel of a spectral integral operator defined as in (1), we may write
Then, there exists a constant such that
As a consequence,
a.2 Commutator Estimate for Radial Kernels
We will say that a kernel operator
is radial if
for some The following theorem establishes a commutator estimate for operators with radial kernels.
We first compute
Therefore, by the Cauchy-Schwartz inequality,
We may bound the first integral by observing
To bound the second integral observe, that by the mean value theorem and the assumption that is radial, we have
Lastly, since we see that
which completes the proof. ∎
Appendix B The Proof of Theorems
b.1 The Proof of Theorem 2.1
Write and , and recall that
Let be an isometry and set . For
, define the vectorsand as
Since is an isometry, and are both orthonormal bases for . Therefore, there exists an unitary matrix (that does not depend upon ) such that
Using this fact, we see that
It follows from (12) that for all . Now writing we have
b.2 The Proof of Theorem 3.2
b.3 The Proof of Theorem 4.1
We write and . If is two-point homogeneous and then by the definition of two-point homogeneity there exists an isometry mapping and . Therefore, we may use the proof of Theorem 2.1 to see that . It follows that is radial and so we may write for some .