Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guarantees

03/02/2020
by   Sen Na, et al.
13

Graph representation learning is a ubiquitous task in machine learning where the goal is to embed each vertex into a low-dimensional vector space. We consider the bipartite graph and formalize its representation learning problem as a statistical estimation problem of parameters in a semiparametric exponential family distribution. The bipartite graph is assumed to be generated by a semiparametric exponential family distribution, whose parametric component is given by the proximity of outputs of two one-layer neural networks, while nonparametric (nuisance) component is the base measure. Neural networks take high-dimensional features as inputs and output embedding vectors. In this setting, the representation learning problem is equivalent to recovering the weight matrices. The main challenges of estimation arise from the nonlinearity of activation functions and the nonparametric nuisance component of the distribution. To overcome these challenges, we propose a pseudo-likelihood objective based on the rank-order decomposition technique and focus on its local geometry. We show that the proposed objective is strongly convex in a neighborhood around the ground truth, so that a gradient descent-based method achieves linear convergence rate. Moreover, we prove that the sample complexity of the problem is linear in dimensions (up to logarithmic factors), which is consistent with parametric Gaussian models. However, our estimator is robust to any model misspecification within the exponential family, which is validated in extensive experiments.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/10/2017

Recovery Guarantees for One-hidden-layer Neural Networks

In this paper, we consider regression problems with one-hidden-layer neu...
research
02/21/2020

Few-Shot Learning via Learning the Representation, Provably

This paper studies few-shot learning via representation learning, where ...
research
11/22/2017

GraphGAN: Graph Representation Learning with Generative Adversarial Nets

The goal of graph representation learning is to embed each vertex in a g...
research
10/25/2018

Provable Gaussian Embedding with One Observation

The success of machine learning methods heavily relies on having an appr...
research
04/03/2019

Defeats GAN: A Simpler Model Outperforms in Knowledge Representation Learning

The goal of knowledge representation learning is to embed entities and r...
research
08/17/2020

Learning Two-Layer Residual Networks with Nonparametric Function Estimation by Convex Programming

We focus on learning a two-layer residual neural network with preactivat...
research
11/06/2020

Massively Parallel Graph Drawing and Representation Learning

To fully exploit the performance potential of modern multi-core processo...

Please sign up or login with your details

Forgot password? Click here to reset