Bayesian Linear Regression on Deep Representations

12/14/2019
by   John Moberg, et al.
0

A simple approach to obtaining uncertainty-aware neural networks for regression is to do Bayesian linear regression (BLR) on the representation from the last hidden layer. Recent work [Riquelme et al., 2018, Azizzadenesheli et al., 2018] indicates that the method is promising, though it has been limited to homoscedastic noise. In this paper, we propose a novel variation that enables the method to flexibly model heteroscedastic noise. The method is benchmarked against two prominent alternative methods on a set of standard datasets, and finally evaluated as an uncertainty-aware model in model-based reinforcement learning. Our experiments indicate that the method is competitive with standard ensembling, and ensembles of BLR outperforms the methods we compared to.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/16/2021

Mesospheric nitric oxide model from SCIAMACHY data

We present an empirical model for nitric oxide NO in the mesosphere (≈60...
research
12/18/2019

Benchmarking the Neural Linear Model for Regression

The neural linear model is a simple adaptive Bayesian linear regression ...
research
07/10/2019

Sparse Unit-Sum Regression

This paper considers sparsity in linear regression under the restriction...
research
06/21/2020

Learned Uncertainty-Aware (LUNA) Bases for Bayesian Regression using Multi-Headed Auxiliary Networks

Neural Linear Models (NLM) are deep models that produce predictive uncer...
research
12/29/2020

Correlation Across Environments Encoded by Hippocampal Place Cells

The hippocampus is often attributed to episodic memory formation and sto...
research
09/25/2019

Regularising Deep Networks with DGMs

Here we develop a new method for regularising neural networks where we l...
research
02/17/2023

Optimal Training of Mean Variance Estimation Neural Networks

This paper focusses on the optimal implementation of a Mean Variance Est...

Please sign up or login with your details

Forgot password? Click here to reset