Approximate leave-future-out cross-validation for Bayesian time series models

02/17/2019
by   Paul-Christian Bürkner, et al.
0

One of the common goals of time series analysis is to use the observed series to inform predictions for future observations. In the absence of any actual new data to predict, cross-validation can be used to estimate a model's future predictive accuracy, for instance, for the purpose of model comparison or selection. As exact cross-validation for Bayesian models is often computationally expensive, approximate cross-validation methods have been developed; most notably methods for leave-one-out cross-validation (LOO-CV). If the actual prediction task is to predict the future given the past, LOO-CV provides an overly optimistic estimate as the information from future observations is available to influence predictions of the past. To tackle the prediction task properly and account for the time series structure, we can use leave-future-out cross-validation (LFO-CV). Like exact LOO-CV, exact LFO-CV requires refitting the model many times to different subsets of the data. Using Pareto smoothed importance sampling, we propose a method for approximating exact LFO-CV that drastically reduces the computational costs while also providing informative diagnostics about the quality of the approximation.

READ FULL TEXT
research
02/17/2019

Approximate leave-future-out cross-validation for time series models

One of the common goals of time series analysis is to use the observed s...
research
10/24/2018

Leave-one-out cross-validation for non-factorizable normal models

Cross-validation can be used to measure a model's predictive accuracy fo...
research
09/19/2022

Robust leave-one-out cross-validation for high-dimensional Bayesian models

Leave-one-out cross-validation (LOO-CV) is a popular method for estimati...
research
09/05/2022

Using leave-one-out cross-validation (LOO) in a multilevel regression and poststratification (MRP) workflow: A cautionary tale

In recent decades, multilevel regression and poststratification (MRP) ha...
research
10/21/2019

Generalised learning of time-series: Ornstein-Uhlenbeck processes

In machine learning, statistics, econometrics and statistical physics, k...
research
04/24/2019

Bayesian leave-one-out cross-validation for large data

Model inference, such as model comparison, model checking, and model sel...

Please sign up or login with your details

Forgot password? Click here to reset