Optimal probabilistic forecasts: When do they work?

09/21/2020
by   Gael M. Martin, et al.
0

Proper scoring rules are used to assess the out-of-sample accuracy of probabilistic forecasts, with different scoring rules rewarding distinct aspects of forecast performance. Herein, we re-investigate the practice of using proper scoring rules to produce probabilistic forecasts that are `optimal' according to a given score, and assess when their out-of-sample accuracy is superior to alternative forecasts, according to that score. Particular attention is paid to relative predictive performance under misspecification of the predictive model. Using numerical illustrations, we document several novel findings within this paradigm that highlight the important interplay between the true data generating process, the assumed predictive model and the scoring rule. Notably, we show that only when a predictive model is sufficiently compatible with the true process to allow a particular score criterion to reward what it is designed to reward, will this approach to forecasting reap benefits. Subject to this compatibility however, the superiority of the optimal forecast will be greater, the greater is the degree of misspecification. We explore these issues under a range of different scenarios, and using both artificially simulated and empirical data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/12/2023

Weighted scoringRules: Emphasising Particular Outcomes when Evaluating Probabilistic Forecasts

When predicting future events, it is common to issue forecasts that are ...
research
03/03/2023

Optimal probabilistic forecasts for risk management

This paper explores the implications of producing forecast distributions...
research
06/06/2022

The Impact of Sampling Variability on Estimated Combinations of Distributional Forecasts

We investigate the performance and sampling variability of estimated for...
research
05/25/2021

Ranking earthquake forecasts using proper scoring rules: Binary events in a low probability environment

Operational earthquake forecasting for risk management and communication...
research
12/20/2017

Approximate Bayesian Forecasting

Approximate Bayesian Computation (ABC) has become increasingly prominent...
research
06/10/2018

Forecast evaluation with imperfect observations and imperfect models

The field of statistics has become one of the mathematical foundations i...
research
08/29/2020

Wisdom of the crowds forecasting the 2018 FIFA Men's World Cup

The FIFA Men's World Cup Tournament (WCT) is the most important football...

Please sign up or login with your details

Forgot password? Click here to reset