DeepAI AI Chat
Log In Sign Up

Masking schemes for universal marginalisers

by   Divya Gautam, et al.
babylon health

We consider the effect of structure-agnostic and structure-dependent masking schemes when training a universal marginaliser (arXiv:1711.00695) in order to learn conditional distributions of the form P(x_i |x_b), where x_i is a given random variable and x_b is some arbitrary subset of all random variables of the generative model of interest. In other words, we mimic the self-supervised training of a denoising autoencoder, where a dataset of unlabelled data is used as partially observed input and the neural approximator is optimised to minimise reconstruction loss. We focus on studying the underlying process of the partially observed data—how good is the neural approximator at learning all conditional distributions when the observation process at prediction time differs from the masking process during training? We compare networks trained with different masking schemes in terms of their predictive performance and generalisation properties.


page 1

page 2

page 3

page 4


Learning about an exponential amount of conditional distributions

We introduce the Neural Conditioner (NC), a self-supervised machine able...

An Improved Training Procedure for Neural Autoregressive Data Completion

Neural autoregressive models are explicit density estimators that achiev...

Learning Functional Causal Models with Generative Neural Networks

We introduce a new approach to functional causal modeling from observati...

Analyzing the Sample Complexity of Self-Supervised Image Reconstruction Methods

Supervised training of deep neural networks on pairs of clean image and ...

Attentive Neural Processes

Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by le...

Universal Supervised Learning for Individual Data

Universal supervised learning is considered from an information theoreti...

Learning Conditional Random Fields with Augmented Observations for Partially Observed Action Recognition

This paper aims at recognizing partially observed human actions in video...