Relating graph auto-encoders to linear models

11/03/2022
by   Solveig Klepper, et al.
0

Graph auto-encoders are widely used to construct graph representations in Euclidean vector spaces. However, it has already been pointed out empirically that linear models on many tasks can outperform graph auto-encoders. In our work, we prove that the solution space induced by graph auto-encoders is a subset of the solution space of a linear map. This demonstrates that linear embedding models have at least the representational power of graph auto-encoders based on graph convolutional networks. So why are we still using nonlinear graph auto-encoders? One reason could be that actively restricting the linear solution space might introduce an inductive bias that helps improve learning and generalization. While many researchers believe that the nonlinearity of the encoder is the critical ingredient towards this end, we instead identify the node features of the graph as a more powerful inductive bias. We give theoretical insights by introducing a corresponding bias in a linear model and analyzing the change in the solution space. Our experiments show that the linear encoder can outperform the nonlinear encoder when using feature information.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/26/2019

Graph Attention Auto-Encoders

Auto-encoders have emerged as a successful framework for unsupervised le...
research
01/21/2020

Simple and Effective Graph Autoencoders with One-Hop Linear Models

Graph autoencoders (AE) and variational autoencoders (VAE) recently emer...
research
02/14/2018

Similarity measures for vocal-based drum sample retrieval using deep convolutional auto-encoders

The expressive nature of the voice provides a powerful medium for commun...
research
02/25/2022

Directed Graph Auto-Encoders

We introduce a new class of auto-encoders for directed graphs, motivated...
research
09/30/2021

Latent Network Embedding via Adversarial Auto-encoders

Graph auto-encoders have proved to be useful in network embedding task. ...
research
07/07/2019

Deep Exponential-Family Auto-Encoders

We consider the problem of learning recurring convolutional patterns fro...
research
12/20/2014

Scoring and Classifying with Gated Auto-encoders

Auto-encoders are perhaps the best-known non-probabilistic methods for r...

Please sign up or login with your details

Forgot password? Click here to reset