Neural Variational Inference For Estimating Uncertainty in Knowledge Graph Embeddings

Recent advances in Neural Variational Inference allowed for a renaissance in latent variable models in a variety of domains involving high-dimensional data. While traditional variational methods derive an analytical approximation for the intractable distribution over the latent variables, here we construct an inference network conditioned on the symbolic representation of entities and relation types in the Knowledge Graph, to provide the variational distributions. The new framework results in a highly-scalable method. Under a Bernoulli sampling framework, we provide an alternative justification for commonly used techniques in large-scale stochastic variational inference, which drastically reduce training time at a cost of an additional approximation to the variational lower bound. We introduce two models from this highly scalable probabilistic framework, namely the Latent Information and Latent Fact models, for reasoning over knowledge graph-based representations. Our Latent Information and Latent Fact models improve upon baseline performance under certain conditions. We use the learnt embedding variance to estimate predictive uncertainty during link prediction, and discuss the quality of these learnt uncertainty estimates. Our source code and datasets are publicly available online at https://github.com/alexanderimanicowenrivers/Neural-Variational-Knowledge-Graphs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/19/2015

Neural Variational Inference for Text Processing

Recent advances in neural variational inference have spawned a renaissan...
research
03/17/2018

Variational Knowledge Graph Reasoning

Inferring missing links in knowledge graphs (KG) has attracted a lot of ...
research
06/21/2023

Deep Language Networks: Joint Prompt Training of Stacked LLMs using Variational Inference

We view large language models (LLMs) as stochastic language layers in a ...
research
10/12/2018

Predictive Uncertainty through Quantization

High-risk domains require reliable confidence estimates from predictive ...
research
02/11/2020

Large Scale Tensor Regression using Kernels and Variational Inference

We outline an inherent weakness of tensor factorization models when late...
research
05/16/2017

Learning Hard Alignments with Variational Inference

There has recently been significant interest in hard attention models fo...
research
05/25/2023

Mortality Forecasting using Variational Inference

This paper considers the problem of forecasting mortality rates. A large...

Please sign up or login with your details

Forgot password? Click here to reset