Attentive Neural Processes

01/17/2019
by   Hyunjik Kim, et al.
10

Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data efficiently with linear complexity in the number of context input-output pairs, and can learn a wide family of conditional distributions; they learn predictive distributions conditioned on context sets of arbitrary size. Nonetheless, we show that NPs suffer a fundamental drawback of underfitting, giving inaccurate predictions at the inputs of the observed data they condition on. We address this issue by incorporating attention into NPs, allowing each input location to attend to the relevant context points for the prediction. We show that this greatly improves the accuracy of predictions, results in noticeably faster training, and expands the range of functions that can be modelled.

READ FULL TEXT

page 2

page 6

page 7

page 13

page 15

page 16

page 17

page 18

research
10/17/2019

Recurrent Attentive Neural Process for Sequential Data

Neural processes (NPs) learn stochastic processes and predict the distri...
research
10/21/2022

Blind Polynomial Regression

Fitting a polynomial to observed data is an ubiquitous task in many sign...
research
06/09/2021

GP-ConvCNP: Better Generalization for Convolutional Conditional Neural Processes on Time Series Data

Neural Processes (NPs) are a family of conditional generative models tha...
research
11/03/2020

Generating Unobserved Alternatives

We consider problems where multiple predictions can be considered correc...
research
08/03/2017

Detection of Abnormal Input-Output Associations

We study a novel outlier detection problem that aims to identify abnorma...
research
12/08/2020

Deep Energy-Based NARX Models

This paper is directed towards the problem of learning nonlinear ARX mod...
research
01/16/2020

Masking schemes for universal marginalisers

We consider the effect of structure-agnostic and structure-dependent mas...

Please sign up or login with your details

Forgot password? Click here to reset