Rational Neural Networks for Approximating Jump Discontinuities of Graph Convolution Operator

08/30/2018
by   Zhiqian Chen, et al.
4

For node level graph encoding, a recent important state-of-art method is the graph convolutional networks (GCN), which nicely integrate local vertex features and graph topology in the spectral domain. However, current studies suffer from several drawbacks: (1) graph CNNs relies on Chebyshev polynomial approximation which results in oscillatory approximation at jump discontinuities; (2) Increasing the order of Chebyshev polynomial can reduce the oscillations issue, but also incurs unaffordable computational cost; (3) Chebyshev polynomials require degree Ω(poly(1/ϵ)) to approximate a jump signal such as |x|, while rational function only needs O(poly log(1/ϵ))liang2016deep,telgarsky2017neural. However, it's non-trivial to apply rational approximation without increasing computational complexity due to the denominator. In this paper, the superiority of rational approximation is exploited for graph signal recovering. RatioanlNet is proposed to integrate rational function and neural networks. We show that rational function of eigenvalues can be rewritten as a function of graph Laplacian, which can avoid multiplication by the eigenvector matrix. Focusing on the analysis of approximation on graph convolution operation, a graph signal regression task is formulated. Under graph signal regression task, its time complexity can be significantly reduced by graph Fourier transform. To overcome the local minimum problem of neural networks model, a relaxed Remez algorithm is utilized to initialize the weight parameters. Convergence rate of RatioanlNet and polynomial based methods on jump signal is analyzed for a theoretical guarantee. The extensive experimental results demonstrated that our approach could effectively characterize the jump discontinuities, outperforming competing methods by a substantial margin on both synthetic and real-world graphs.

READ FULL TEXT

page 1

page 9

research
02/04/2022

Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited

Designing spectral convolutional networks is a challenging problem in gr...
research
09/09/2020

1-Dimensional polynomial neural networks for audio signal related problems

In addition to being extremely non-linear, modern problems require milli...
research
11/08/2020

Fourier-based and Rational Graph Filters for Spectral Processing

Data are represented as graphs in a wide range of applications, such as ...
research
06/11/2017

Neural networks and rational functions

Neural networks and rational functions efficiently approximate each othe...
research
07/02/2019

Modeling and Computation of Kubo Conductivity for 2D Incommensurate Bilayers

This paper presents a unified approach to the modeling and computation o...
research
06/29/2021

From ESPRIT to ESPIRA: Estimation of Signal Parameters by Iterative Rational Approximation

We introduce a new method for Estimation of Signal Parameters based on I...
research
02/28/2020

Generalized Rational Variable Projection With Application in ECG Compression

In this paper we develop an adaptive transform-domain technique based on...

Please sign up or login with your details

Forgot password? Click here to reset