DeepAI AI Chat
Log In Sign Up

Model Selection for Time Series Forecasting: Empirical Analysis of Different Estimators

by   Vítor Cerqueira, et al.

Evaluating predictive models is a crucial task in predictive analytics. This process is especially challenging with time series data where the observations show temporal dependencies. Several studies have analysed how different performance estimation methods compare with each other for approximating the true loss incurred by a given forecasting model. However, these studies do not address how the estimators behave for model selection: the ability to select the best solution among a set of alternatives. We address this issue and compare a set of estimation methods for model selection in time series forecasting tasks. We attempt to answer two main questions: (i) how often is the best possible model selected by the estimators; and (ii) what is the performance loss when it does not. We empirically found that the accuracy of the estimators for selecting the best solution is low, and the overall forecasting performance loss associated with the model selection process ranges from 1.2 size, are important in the relative performance of the estimators.


page 1

page 2

page 3

page 4


Hierarchy-guided Model Selection for Time Series Forecasting

Generalizability of time series forecasting models depends on the qualit...

Multi-Objective Model Selection for Time Series Forecasting

Research on time series forecasting has predominantly focused on develop...

Empirical Quantitative Analysis of COVID-19 Forecasting Models

COVID-19 has been a public health emergency of international concern sin...

fETSmcs: Feature-based ETS model component selection

The well-developed ETS (ExponenTial Smoothing or Error, Trend, Seasonali...

DFSeer: A Visual Analytics Approach to Facilitate Model Selection for Demand Forecasting

Selecting an appropriate model to forecast product demand is critical to...

Adaptive Learning on Time Series: Method and Financial Applications

We formally introduce a time series statistical learning method, called ...

Assessing Prediction Error at Interpolation and Extrapolation Points

Common model selection criteria, such as AIC and its variants, are based...