Expressive Priors in Bayesian Neural Networks: Kernel Combinations and Periodic Functions

05/15/2019
by   Tim Pearce, et al.
0

A simple, flexible approach to creating expressive priors in Gaussian process (GP) models makes new kernels from a combination of basic kernels, e.g. summing a periodic and linear kernel can capture seasonal variation with a long term trend. Despite a well-studied link between GPs and Bayesian neural networks (BNNs), the BNN analogue of this has not yet been explored. This paper derives BNN architectures mirroring such kernel combinations. Furthermore, it shows how BNNs can produce periodic kernels, which are often useful in this context. These ideas provide a principled approach to designing BNNs that incorporate prior knowledge about a function. We showcase the practical value of these ideas with illustrative experiments in supervised and reinforcement learning settings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/26/2021

Periodic Activation Functions Induce Stationarity

Neural network models are known to reinforce hidden data biases, making ...
research
06/13/2022

Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Process

In typical scenarios where the Federated Learning (FL) framework applies...
research
04/10/2020

Reinforcement Learning via Gaussian Processes with Neural Network Dual Kernels

While deep neural networks (DNNs) and Gaussian Processes (GPs) are both ...
research
02/13/2019

On the Expressive Power of Kernel Methods and the Efficiency of Kernel Learning by Association Schemes

We study the expressive power of kernel methods and the algorithmic feas...
research
05/30/2021

Periodic-GP: Learning Periodic World with Gaussian Process Bandits

We consider the sequential decision optimization on the periodic environ...
research
10/29/2019

Function-Space Distributions over Kernels

Gaussian processes are flexible function approximators, with inductive b...
research
05/15/2022

Incorporating Prior Knowledge into Neural Networks through an Implicit Composite Kernel

It is challenging to guide neural network (NN) learning with prior knowl...

Please sign up or login with your details

Forgot password? Click here to reset