DeepAI AI Chat
Log In Sign Up

Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guarantees

by   Sen Na, et al.

Graph representation learning is a ubiquitous task in machine learning where the goal is to embed each vertex into a low-dimensional vector space. We consider the bipartite graph and formalize its representation learning problem as a statistical estimation problem of parameters in a semiparametric exponential family distribution. The bipartite graph is assumed to be generated by a semiparametric exponential family distribution, whose parametric component is given by the proximity of outputs of two one-layer neural networks, while nonparametric (nuisance) component is the base measure. Neural networks take high-dimensional features as inputs and output embedding vectors. In this setting, the representation learning problem is equivalent to recovering the weight matrices. The main challenges of estimation arise from the nonlinearity of activation functions and the nonparametric nuisance component of the distribution. To overcome these challenges, we propose a pseudo-likelihood objective based on the rank-order decomposition technique and focus on its local geometry. We show that the proposed objective is strongly convex in a neighborhood around the ground truth, so that a gradient descent-based method achieves linear convergence rate. Moreover, we prove that the sample complexity of the problem is linear in dimensions (up to logarithmic factors), which is consistent with parametric Gaussian models. However, our estimator is robust to any model misspecification within the exponential family, which is validated in extensive experiments.


page 1

page 2

page 3

page 4


Recovery Guarantees for One-hidden-layer Neural Networks

In this paper, we consider regression problems with one-hidden-layer neu...

Few-Shot Learning via Learning the Representation, Provably

This paper studies few-shot learning via representation learning, where ...

GraphGAN: Graph Representation Learning with Generative Adversarial Nets

The goal of graph representation learning is to embed each vertex in a g...

Provable Gaussian Embedding with One Observation

The success of machine learning methods heavily relies on having an appr...

Defeats GAN: A Simpler Model Outperforms in Knowledge Representation Learning

The goal of knowledge representation learning is to embed entities and r...

Learning Two-Layer Residual Networks with Nonparametric Function Estimation by Convex Programming

We focus on learning a two-layer residual neural network with preactivat...

Learning binary undirected graph in low dimensional regime

Given a random sample extracted from a Multivariate Bernoulli Variable (...