The future of forecasting competitions: Design attributes and principles

02/09/2021
by   Spyros Makridakis, et al.
0

Forecasting competitions are the equivalent of laboratory experimentation widely used in physical and life sciences. They provide useful, objective information to improve the theory and practice of forecasting, advancing the field, expanding its usage and enhancing its value to decision and policymakers. We describe ten design attributes to be considered when organizing forecasting competitions, taking into account trade-offs between optimal choices and practical concerns like costs, as well as the time and effort required to participate in them. Consequently, we map all major past competitions in respect to their design attributes, identifying similarities and differences between them, as well as design gaps, and making suggestions about the principles to be included in future competitions, putting a particular emphasis on learning as much as possible from their implementation in order to help improve forecasting accuracy and uncertainty. We discuss that the task of forecasting often presents a multitude of challenges that can be difficult to be captured in a single forecasting contest. To assess the quality of a forecaster, we, therefore, propose that organizers of future competitions consider a multi-contest approach. We suggest the idea of a forecasting pentathlon, where different challenges of varying characteristics take place.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/30/2021

Forecasting and its Beneficiaries

This chapter addresses the question of who benefits from forecasting, us...
research
02/23/2021

Fast and frugal time series forecasting

Over the years, families of forecasting models, such as the exponential ...
research
01/18/2023

Quality Attributes Optimization of Software Architecture: Research Challenges and Directions

The estimation and improvement of quality attributes in software archite...
research
02/23/2023

Online Calibrated Regression for Adversarially Robust Forecasting

Accurately estimating uncertainty is a crucial component of decision-mak...
research
12/07/2022

Criteria for Classifying Forecasting Methods

Classifying forecasting methods as being either of a "machine learning" ...
research
04/04/2020

ForecastTB – An R Package as a Test-bench for Forecasting Methods Comparison

This paper introduces the R package ForecastTB that can be used to compa...
research
01/28/2019

It could be worse, it could be raining: reliable automatic meteorological forecasting

Meteorological forecasting provides reliable prediction about the future...

Please sign up or login with your details

Forgot password? Click here to reset