Hierarchical Forecasting of Dengue Incidence in Sri Lanka

12/02/2021
by   L. S. Madushani, et al.
0

The recurrent thread of dengue incidence in Sri Lanka is still abundant and it creates a huge burden to the country. Hence, the National Dengue Control Unit of Sri Lanka propose a national action plan to prevent and control the dengue incidence. To implement the necessary actions for short and long terms the proposed plan operates under three levels:country-level, province-level, and district levels. In order to optimize resource allocation, the health officers require the forecasts for country, province and district levels, which preserves the aggregate consistency associated with the district, province, and country levels as well as time correlations. Hence, the objective of this study is to forecast the dengue incidence in Sri Lanka using a hierarchical time series forecasting approach based on spatial and temporal hierarchical structures. Hierarchical forecasting involves two steps such as generating base forecasts and reconciliation of these base forecasts. In this study, Exponential smoothing(ETS) and Autoregressive Integrated Moving Average (ARIMA), NAIVE, Seasonal NAIVEapproaches and average method are used to generate base forecasts. Accuracy of forecasts is evaluated using Mean Absolute Scale Error (MASE). We compare the accuracy of hierarchical time series forecasts with other benchmark approaches. The forecast accuracy reveals that the best forecasting approaches for country, provinces and districts are not limited to a single approach. Hence, we investigate reasons for the variations of performance in different forecasting approaches based on a time series feature-based visualization approach.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 10

page 11

page 13

page 14

page 18

10/21/2020

Model selection in reconciling hierarchical time series

Model selection has been proven an effective strategy for improving accu...
11/05/2021

Long Range Probabilistic Forecasting in Time-Series using High Order Statistics

Long range forecasts are the starting point of many decision support sys...
09/30/2019

Multiscale Influenza Forecasting

Influenza forecasting in the United States (US) is complex and challengi...
11/10/2020

Forecasting and Analyzing the Military Expenditure of India Using Box-Jenkins ARIMA Model

The advancement in the field of statistical methodologies to economic da...
06/07/2019

Reconciling Hierarchical Forecasts via Bayes' Rule

When time series are organized into hierarchies, the forecasts have to s...
12/28/2018

Cardiology Admissions from Catheterization Laboratory: Time Series Forecasting

Emergent and unscheduled cardiology admissions from cardiac catheterizat...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Dengue is the most rapidly growing viral infection in the world (world2014dengue)

. The causative agent of dengue infection is classified into four serotypes: DENV-1, DENV-2, DENV-3, and DENV-4 whereas infection of one serotype does not result in lifelong immunity against other serotypes

(sanyaolu2017global). According to the World Health Organization, 50 to 100 million new dengue cases occur annually including 20,000 deaths in more than 100 countries of the world (world2012global). A large number of dengue cases were reported in Central and South America, South-East Asia and Western Pacific regions (teixeira2009diagnosis). Asia represents approximately 70 of the global burden of the disease (bhatt2013global).

Dengue incidence of Sri Lanka shows a considerable rise during the last two decades, causing economic and social burdens. The first serological confirmed dengue case was reported in 1962 and the first largest dengue epidemic outbreak occurred in 2009 with 35,008 suspected cases and 346 deaths in Sri Lanka (withanage2018forecasting). The largest dengue outbreak was reported in 2017, comprising 186,101 suspected cases and over 320 deaths according to the Epidemiological Unit of the Ministry of Health, Sri Lanka (chandrakantha2019risk). This unprecedented outbreak is due to the variation of the usual circulating DENV-2 (ali2018unprecedented; tissera2020severe). More than 30,000 cases have been reported each year since 2012 and the highest number of dengue cases is reported in the Western province of Sri Lanka (withanage2018forecasting). At present, the spread of dengue incidence in districts outside the Western province has indicated a dramatic change.

To date, there is no specific treatment methodology to cure the infection (wijegunawardana2019evaluation). However, simple fluid replacement and case management assist to reduce the fatality rate from 20 to 1 (chandrakantha2019risk). Therefore, forecasting dengue incidence is important to implement curative and control programs. In recent, several researchers have made attempts to forecast dengue incidence in Sri Lanka. For example, time series regression models were developed using different climate variables for various districts (withanage2018forecasting; goto2013analysis). Distributed lag nonlinear modeling approach was used to identify the relationship between climate factors and dengue incidence in Colombo district of Sri Lanka (talagala2015distributed). talagala2015wavelet performed a wavelet analysis to identify the pattern of dengue incidence in 25 districts of Sri Lanka . More recently, chandrakantha2019risk

developed a model to predict the likelihood of having dengue incidence in Sri Lanka based on climate factors. Furthermore, artificial neural network approach was applied to forecast dengue incidence in Kandy district of Sri Lanka for the period 2010 to 2012 by considering rainfall, humidity, temperature factors, and previous dengue cases

(nishanthi2014prediction). attanayake2020exponential used exponential smoothing approach to forecast dengue counts in Colombo. All these studies have been conducted to forecast dengue incidence of the whole country, particular provinces, or districts independently. None of these approaches has focused to cater the needs that are proposed by the national action plan to implement dengue prevention and control programs (unit2019ministry). This plan has assigned officers for national, provincial, and district levels for achieving the proposed objectives of dengue control and prevention activities. Hence, this study mainly focuses on dengue incidence forecasting for districts, provinces, and the whole country to assist the decision-making process of assigned officers and plan and implement the programs at the district, province, and country level.

The hierarchical time series forecasting has the ability to deal with time series that can be aggregated at different levels based on geography. This approach provides forecasts that are coherent across the levels of hierarchy. That is, forecasts at the aggregate level are equal to the sum of the relevant forecast at the desegregated level (wickramasuriya2019optimal). Moreover, it allows to capture of the correlation and interaction between the series at each level of hierarchy and it has the ability to capture different information based on each series. In addition, the most significant aspect of this approach is that it ensures aligned decision-making across the entire hierarchical structure (athanasopoulos2020hierarchical). Hence, this study considers hierarchical time series forecasting approach to obtain the forecasts of districts, provinces, and country levels of Sri Lanka. Furthermore, forecasts are needed for short-term and long-term periods. Hence, this study gives attention towards the time correlation as well.

2 Methodology and Data

2.1 Forecasting hierarchical time series

Figure 1: Two-level hierarchical tree diagram

A simple hierarchical structure is illustrated in Figure 1. In Figure 1, each series are observed at time and forecasts are generated for each series at time . The series at the bottom level of the hierarchy, level 2, are denoted as and . Series at level 1, and can be obtained by aggregating the corresponding bottom level series. These series are further aggregated to obtain the total series at level 0, the ”Total”. Furthermore, denotes the series corresponds to node and denotes number of series at level . The total number of series in hierarchical structure is denoted by . Hierarchical structure of Figure 1 can be denoted by matrix notation as

In general, hierarchical time series can be denoted by the matrix notation , where denotes all series in the hierarchy and

is the vector of all bottom level series in the hierarchy at time

. is the summing matrix of order where is the number of series in the most disaggregate level of the hierarchical structure.

Several methods have been proposed to reconcile the time series forecasts in a way that respects the hierarchy. However, all existing hierarchical time series forecasting approaches can be illustrated as

where is -step-ahead reconciled forecasts, is -step-ahead base forecasts, and is an matrix. The role of is different according to the hierarchical forecasting approach.

Top-down and bottom-up approaches are the common approaches used in hierarchical time series forecasting. The bottom-up approach involves forecasting each of the bottom level series and sum these forecasts to obtain other levels of the hierarchy (hyndman2011optimal). However, bottom level data are noisy and therefore it is difficult to model bottom level series (zotteri2014forecasting). In the top-down approach, forecasts are generated for the most aggregated series and then forecasts of the total series are disaggregated based on the proportions. Here, the proportions can be obtained based on average or historical proportions (athanasopoulos2009hierarchical; hyndman2011optimal).

Another approach named as optimal combination approach was proposed by (hyndman2011optimal)

which performs better than top-down and bottom-up approaches. This approach obtains the forecasts of all series of the hierarchical structure and then optimally reconcile based on a regression model. It requires an estimator for the covariance matrix of errors that occur due to incoherence. However, it is impossible to estimate. Hence, minimum trace optimal reconciliation approach (MinT) was introduced to incorporate the information from a full covariance of forecast errors by minimizing the mean squared error of the coherence forecasts under the assumption of unbiasedness and the resulting reconciled forecasts are in the form

In practice, estimation of is difficult (hyndman2018forecasting). Hence, (wickramasuriya2019optimal) proposed alternative estimators for .

The ordinary least square estimator (OLS) can be obtained with the substitution

where and

is an identity matrix. The drawback of this substitution is that the scale differences between the levels of the structure are not considered. Another estimation is

where and . In this case,

is in-sample residuals of the base forecasts stacked in the same order as the data. Here, the base forecasts are scaled using the variance of residuals. Therefore, this approach is referred to as MinT approach based on a weighted least square (WLS) estimator using variance scaling. Another criterion considers that each of the bottom-level base forecasts are uncorrelated between nodes and they have errors with equal variance

. Furthermore, each element of the diagonal matrix denotes the number of forecast error variances that contribute to that aggregation level. This estimator focuses only on the structure of the hierarchy as

where and with 1 being a unit vector of dimension . Next,

where is considered as the estimation. In this estimation, is full one-step covariance matrix. However, this estimator is not an appropriate estimator when the number of bottom level series is larger than the length of the series . Hence, shrinkage estimator is provided based on the sample covariance to a diagonal matrix as

where . In this method, is a shrinkage estimator defined as

where is a diagonal matrix that consists of diagonal entries of . Therefore, this estimator allows us to capture strong interrelations between time series in the hierarchy. schafer2005shrinkage proposed the shrinkage intensity parameter as , where is the element of which is the 1-step-ahead sample correlation matrix.

2.2 Base forecasting approaches

In hierarchical forecasting, forecasts are generated for each series at every node of the hierarchy individually using different forecasting modelling approaches. These individual forecasts are referred to as base forecasts. In this study we consider Naive method, Seasonal naive method (SNAIVE), average method, AutoRegressive Integrated Moving Average (ARIMA) approach, and Exponential Smoothing method (ETS) to generate base forecasts.

In Naive approach, forecasts are equal to the last observed value in the series.

SNAIVE approach considers that the forecasts are equal to the last observed value from the same season of the year. That means, forecasts for time is

where is the seasonal period, is the integer part of . In the average method, the forecasts of all future values are equal to the average (or mean) of the available data. If denotes the available data, then

Auto-correlations in the data are described by ARIMA models. ARIMA model is a collection of auto-regressive and moving average components which captures the effect of past observations and error terms. ARIMA model comprises with 3 terms: where is the order of AR term, is the order of MA term, and

is the number of differences required to make the time series into stationary. AR term considers the linear regression model which depends only on lagged term as predictors

MA term in ARIMA model describes the effect of lagged error terms

Exponential Smoothing models are constructed by considering the weighted averages of past observations in which high weights are assigned to the most recent observation. It was proposed by holt2004forecasting and winters1960forecasting. Statistical frame work of exponential smoothing methods was proposed by hyndman2002state are illustrated by Table 1.

Seasonal Component
Trend Component N(None) A(Additive) M(Multiplicative)
N (None) (N,N) (N,A) (N,M)
A (Additive) (A,N) (A,A) (A,M)
Ad (Additive damped) (Ad,N) (Ad,A) (Ad,M)
Multiplicative (M) (M,N) (M,A) (M,M)
Md (Multiplicative damped) (Md,N) (Md,A) (Md,M)
Table 1: Exponential Smoothing methods (continued)

For example, cell (N, N) corresponds to simple exponential smoothing method. Two possible state space models (additive errors or multiplicative errors) are available for each of the methods in Table 1.

2.3 Forecast performance evaluation

We evaluate the performance of base forecasting approaches against hierarchical forecasting approaches. Many accuracy measures have been proposed to check the forecast accuracy (hyndman2006another). However, many of these proposed accuracy measures are not applicable due to the reasons such as scale dependency, producing undefined or infinite value and giving misleading results. Therefore, in this study we use Mean Absolute Scale Error (MASE) to obtain the accuracy of forecasting methods. For -step-ahead forecasts,

is the mean absolute error of the forecast method , and are the actual and forecast values at period respectively. Then,

where is the scaling factor where is the sampling frequency per year (athanasopoulos2017forecasting).

2.4 Data and modelling strategies

Figure 2: Sri Lanka map

Weekly notified dengue cases in 26 districts of Sri Lanka were obtained from Epidemiological Unit, Ministry of Health, Sri Lanka for the period of week of 2006 to week of 2020. This study considers the three levels of geographic divisions in Sri Lanka. In the first level, Sri Lanka is divided into 9 provinces such as Central (CN), Eastern (ET), North Central (NC), Nothern (NT), North Western (NW), Sabaragamuwa (SB), Southern (ST), UVA, and Western (WT) provinces. In the second level, these 9 provinces are further classified into 26 districts. In addition, level 0 is the most aggregate level or dengue counts of the whole country. The geographic structure is illustrated as a map in Figure 2 and the corresponding spatial hierarchical structure is illustrated by Figure 3.

Figure 3: Spatial hierarchical structure

As shown in Figure 4, the series of each level of the spatial hierarchical structure shown in Figure 3 reveal different time series patterns, respectively.

Figure 4: Time series of Dengue counts in Sri Lanka for different aggregated levels of the spatial hierarchical structure

According to Figure 4, time series are different in scales and time series features. In Figure 4, the panel corresponds to level 0 demonstrates dengue counts of Sri Lanka and it indicates two significant spikes during each year due to the monsoon season of Sri Lanka. A notable upward trend pattern is also visible. When we focus on level 1 and level 2 of hierarchy, these pattern variations are less prominent. Bottom level series are much noisier than other levels. Hence, it is more challenging to model the series of bottom level. Generating coherent accurate forecasts for the series at each levels are extremely important to successfully plan the curative and preventive programs.

In addition, dengue curative and preventive programs require different strategies for different time period. Hence, this study focuses on the temporal structure of data as demonstrated by Figure 5.

Figure 5: Temporal hierarchical structure

Figure 6 represents the time series dynamic of dengue incidence for different aggregated levels in Sri Lanka. According to Figure 6, the weekly time series depicts the highest seasonal variation. Then, bi-weekly series have been omitted the negligible seasonality variations. Next, monthly series have demonstrated the important seasonality component and a small trend component. Finally, the yearly time series depicts upward trend of the series.

Figure 6: Time series of Dengue counts in Sri Lanka according to temporal hierarchical structure

Before fitting the model, we separate the data set into training set and test sets. This study considers four different training sets such as 2006-2019, 2016-2019, 2006-2018, 2016-2018 and test sets are 2020, 2019 respectively. Furthermore, weekly, monthly, quarterly, and yearly frequencies are taken into the account in all these training and test sets. All statistical analysis related to spatial hierarchical forecasting was performed by fabletools package (hyndman2018forecasting) whereas temporal hierarchical forecasting was implemented by thief package in R (hyndman2018thief).

In the case of average, NAIVE, SNAIVE approaches, adjusting base forecasts are not necessary to obtain reconciled forecasts due to the fact that forecasts already follow the aggregate constraints associated with spatial hierarchical structure. In temporal hierarchical structure, forecasts of average and SNAIVE approach only adhere to constraints associated with temporal generalities. Hence, forecast reconciliation is necessary for ARIMA, ETS, and NAIVE methods in temporal hierarchical forecasting whereas forecast reconciliation is needed for ARIMA and ETS method in spatial hierarchical forecasting.

3 Results

3.1 Forecasting results

We compare the forecasts accuracy for all series in the spatial and temporal hierarchical structure. We compare the accuracy of forecasts generated based of hierarchical forecasting approach against Average method (AVG), NAIVE, SNAIVE, ETS and ARIMA. The computed MASE values over the test sets of all series in the spatial hierarchical structure are summarized from Table 3 to Table 16. We then identified the best forecasting method for all series at each level of the of the hierarchy. The corresponding results are shown in Figure 7 - Figure 8. According to the results it can be seen that when generating forecasts for 2019, for most of the geographical regions hierarchical forecasts give the best forecast. However, when generating forecasts for the year 2020, the average method gives the best forecasts for Colombo district, Kalutara district (level 2) and Western province (level 1). This could be due to ongoing COVID-19 pandemic. Sometimes healthcare providers faced challenges to distinguish COVID-19 from dengue. Hence, there were some unusual pattern in dengue cases in 2020 specially for population density area. For the country-level series, the lowest MASE was given by the forecasts generated from average method based on the training sets 2006-2019 (TS1) and 2016-2019 (TS2) whereas the best forecasting approach for the training sets 2006-2018 (TS3) and 2016-2018 (TS4) is hierarchical time series forecasting (HS) approach. Similarly, we generated weekly, monthly and quarterly forecasts based on temporal hierarchical structure and compared the accuracy of results with other benchmark approaches. The results are are summarized in Table 2. Furthermore, Figure 9 and 10 show the best forecasting method for weekly, monthly and quarterly for districts and provinces respectively. For most of the geographical regions forecasts generated based on the hierarchical approach gives the best forecasts.

Figure 7: The best forecasting method of level 2 according to training set
Figure 8: The best forecasting method of level 1 according to training set
Training set
2006-2018 2016-2018 2006-2019 2016-2019
Series Method MASE Method MASE Method MASE Method MASE
Annual HS 0.26 HS 0.02 AVG 0.20 HS 0.09
Semi-annual HS 1.28 HS 0.29 AVG 0.61 HS 0.14
Quarterly HS 1.17 HS 0.29 AVG 0.97 HS 0.21
Monthly HS 1.17 HS 0.31 AVG 0.96 HS 0.23
Bi-Weekly HS 1.18 HS 0.31 AVG 0.97 HS 0.24
Weekly HS 1.17 HS 0.32 AVG 0.96 HS 0.24
Table 2: The best forecasting approaches for temporal granualities according to training set
Figure 9: District with the best forecasting method for 2020
Figure 10: Provinces with the best forecasting method for 2020

Based on the above results the best forecasting approaches in each series of the spatial hierarchical structure and temporal hierarchical structure are not limited to single approach. According to No-Free-Lunch theorem, there is no single method that fits all situations (kang2017visualising). Hence, this study attempts to identify the reasons for the best forecasting approach variations based on time series features.

3.2 Visualization of model performance

In Section 3.1, we identified the best forecasting approaches corresponding to weekly, monthly, quarterly, and yearly series for the years 2019 and 2020 under four different training sets based on the spatial or temporal hierarchical structure. Hence, this section attempts to identify the reasons for the forecasting model performance variations based on feature-based visualization. In order to obtain feature-based visualization, first features of time series are extracted from the training set corresponding to the best forecasting method training set. The features are computed based on the feasts package in R (feasts). We computed 48 features from each series. They are strength of seasonality, strength of trend, number of flat spots, statistic based on Lagrange Multiplier test, optimal value of Box-cox transformation, seasonal peak year, seasonal trough year, spikiness, linearity, curvature, first and tenth autocorrelation coefficient of the remainder component, first and tenth autocorrelation coefficient from the original data, longest flat spot length, average interval between non-zero observations, squared coefficient of variation of non-zero observations, proportion of data which starts with zero and ends with zero, autocorrelation function at lag 1 and lag 10 from the first differenced data, autocorrelation function at lag 1 and lag 10 from the second differenced data, autocorrelation function at lag 1 from first seasonal lag data, hurst coefficient, spectral entropy, Box-Pierce statistic, p-value from Box-Pierce statistic, Ljung-Box statistic, p-value from Ljung-Box statistic, partial autocorrelation function at lag 5, partial autocorrelation function at lag 5 from first differenced data, partial autocorrelation function at lag 5 from second differenced data, partial autocorrelation at the first seasonal lag, KPSS statistic, p-value of KPSS test, Phillips-Perron statistic, p-value from Phillips-Perron test, required number of differences, required number of seasonal differences, stability, lumpiness, the largest mean shift between two consecutive sliding windows, index at which the largest mean shift occurs, the largest variance shift between two consecutive sliding windows, index at which the largest variance shift occurs, the largest distributional shift, index at which the largest distributional shift occurs, and number of times time series crosses the median. A detailed description of the features is provided in talagala2018meta. Next, the principal component analysis (PCA) was performed to reduce the dimension of features. Furthermore, PCA was conducted separately for district-wise or province-wise weekly, monthly, and quarterly series. Principal component 1 versus principal component 2 is plotted to visualize feature variation according to the modeling approach and the corresponding visualizations are presented from Figures 11 to 16. The points in the resulting scatter plots are coloured according to four features: i) strength of trend, ii) strength of seasonality, iii) stability and iv) lumpiness. The two features stability and lumpiness are calculated based on non-overlapping windows of the time series. Then stability is measures based on the variance of the means, while lumpiness is measured based on the variance of the variances. The reason for selecting these features are they are very easy to interpret. In addition to that the points are coloured according to their province/ district name, training set corresponds to best forecasting method and best forecasting method (forecasting method that gives the lowest MASE).

Figure 11: Model performance visualization for weekly province-wise series of spatial hierarchical structure
Figure 12: Model performance visualization for monthly province-wise series of spatial hierarchical structure
Figure 13: Model performance visualization for quarterly province-wise series of spatial hierarchical structure
Figure 14: Model performance visualization for weekly district-wise series of spatial hierarchical structure
Figure 15: Model performance visualization for monthly district-wise series of spatial hierarchical structure
Figure 16: Model performance visualization for quarterly district-wise series of spatial hierarchical structure

According to Figure 11, only for Nothern province the best forecasts were given by SNAIVE and average (AVG) method based on the training sets TS4 and TS2 respectively. According to the feature visualization we can see the strength of trend corresponds to the Nothern province is very low compared to other series. Furthermore, the stability corresponds to the Nothern province is considerably higher. According to Figure 12, Eastern province and Sabaragamuwa province are located far away from the other provinces due to differences in the features. The best forecasting method for Eastern province is NAIVE approach. This could be due to differences in trend, seasonality and lumpiness compared to other series. In Figure 11 to 16, for all series that generates best forecasts with NAIVE or SNAIVE have low value for lumpiness, high value for stability, strength of trend and strength of seasonality. In addition to that in we see outlying stay points that are isolated from other points tends to give best forecasts based on NAIVE, SNAIVE or Average method.

.

4 Conclusion

In this study, we have applied hierarchical time series forecasting approaches to forecast dengue incidence in Sri Lanka. Hierarchical time series forecasting approaches are extremely important to obtain optimal decisions which nicely match with the decisions across the levels of hierarchical structure. In hierarchical forecasting, first, we need to obtain forecasts of each series without considering the hierarchical structure. This study obtain the forecasts by ARIMA, ETS, NAIVE, SNAIVE, and average approaches which are referred to as base forecasts. Then, these forecasts are adjusted in order to preserve the coherency of forecasts. However, hierarchical time series forecasting approaches do not provide the lowest MASE values in all situations of spatial hierarchical series. Furthermore, the number of observations in the training set does not impact the accuracy improvement. According to the feature-based visualization of spatial hierarchical structure, SNAIVE and NAIVE approaches are not the most accurate method if a series contains a high lumpiness value. Furthermore, the hierarchical time series forecasting approach indicates the best performance in all ranges of seasonality, trend, stability, and lumpiness feature values. In the case of the high lumpiness value, the average method also provides accurate performance. According to the results of temporal hierarchical forecasting, the hierarchical time series forecasting approach is the best approach for the years 2019 and 2020. According to feature-based visualization we observe that the choice of the best forecasting method depends of the features of the time series. We leave the study of cross-temporal hierarchical forecasting as a future research. We have developed a shiny app to disseminate the results of the analysis with the general public. The app is available at https://github.com/SamudraMadushani/DengueSriLankaApp

. All data are shared through an open source R package called DISC (DISease Counts) which is available at

https://github.com/SMART-Research/DISC. The maps are generated using ceylon R package available at https://github.com/thiyangt/ceylon.


APPENDIX

We present the results corresponds to base and reconciled forecasts for four different training and test sets in spatial hierarchical forecasting from Table 3 to 14. Base forecasting approaches such as ARIMA, ETS, NAIVE, SNAIVE, and Average are denoted by arm, snv, nve, ets, avg respectively. Hierarchical time series forecasting approaches such as bottom up, top down, ordinary least square estimator using variance scaling, structural scaling, one-step covariance matrix when base forecasting approach is ARIMA are denoted by bup, top, ols, mit, var, stc, and cov.

Base Forecasting HTS based on ARIMA HTS based on ETS
series arm snv nve ets avg bup top ols mit var stc cov ebup etop eols emit evar estc ecov
SL 3.97 2.14 4.07 4.49 0.90 3.96 3.97 3.95 3.85 3.92 3.93 3.10 3.90 4.49 4.36 3.51 3.75 3.91 1.31
Province
   CN 2.16 1.97 3.79 0.54 0.62 3.47 2.24 2.55 2.87 2.91 2.83 2.35 1.87 0.76 1.58 1.28 1.28 1.40 1.67
   ET 7.70 2.18 7.70 7.70 1.93 7.68 8.01 7.84 7.70 7.70 7.75 7.40 8.53 10.19 8.97 8.23 8.19 8.57 8.97
   NC 2.83 1.28 2.87 2.84 0.84 2.65 2.91 3.10 2.53 2.72 2.80 0.48 2.84 3.81 6.12 2.44 2.87 3.65 1.56
   NW 2.27 0.99 2.41 2.28 0.68 2.24 2.33 2.33 2.23 2.26 2.27 2.14 2.34 3.02 3.07 2.04 2.35 2.50 0.62
   NT 9.64 3.13 9.64 9.64 1.53 9.37 9.90 9.74 9.46 9.50 9.56 8.48 10.09 12.71 11.49 9.75 9.93 10.74 6.61
   SB 1.27 1.06 0.99 1.10 0.53 1.03 1.32 1.26 1.07 1.13 1.16 0.77 0.89 1.59 1.84 0.67 1.05 1.19 3.27
   ST 3.84 2.90 3.58 3.46 0.89 3.43 3.95 3.82 3.50 3.59 3.65 2.87 3.37 4.58 4.42 3.04 3.44 3.74 1.60
   UVA 1.58 1.21 2.56 2.67 0.73 2.65 1.61 2.12 2.08 2.17 2.15 1.28 2.65 3.51 4.94 2.40 2.69 3.22 0.40
   WT 3.23 2.15 3.27 2.53 0.86 3.24 3.32 3.25 3.17 3.24 3.24 2.24 3.27 3.39 2.96 2.92 3.10 2.98 0.33
Districts
   KN 2.85 1.95 3.05 0.65 0.64 2.85 1.79 2.45 2.18 2.23 2.57 1.59 0.65 0.51 0.60 0.51 0.52 0.57 1.25
   MTL 5.11 2.05 5.73 6.05 0.80 5.11 3.51 3.76 4.78 4.80 4.17 4.75 6.05 3.25 5.62 5.47 5.75 5.35 2.89
   NE 3.02 1.40 3.49 3.06 0.85 3.02 2.08 2.15 2.69 2.87 0.76 1.66 3.06 1.57 1.58 2.39 2.90 0.70 1.65
   AM 2.58 1.96 2.08 2.54 1.42 2.58 2.68 3.22 2.34 2.58 2.86 1.38 2.54 2.91 4.01 1.99 2.45 2.72 1.18
   BAT 5.42 1.46 5.53 5.42 1.99 5.42 5.65 5.50 5.45 5.43 5.46 5.41 5.42 6.58 5.65 5.27 5.20 5.44 7.75
   KAL 4.03 1.32 4.10 4.10 1.08 4.03 4.18 4.15 4.12 4.05 4.08 4.37 4.10 4.86 4.44 3.86 3.69 4.13 5.16
   TRC 12.56 3.56 12.56 15.31 2.53 12.56 13.04 12.69 12.57 12.58 12.62 11.87 15.31 18.15 15.67 14.96 14.97 15.34 12.63
   APU 2.63 1.17 2.80 2.84 0.78 2.63 2.88 2.93 2.51 2.71 2.74 0.31 2.84 3.78 5.17 2.38 2.86 3.41 1.90
   POL 2.25 1.29 2.49 2.36 0.82 2.25 2.48 2.92 2.17 2.30 2.47 0.83 2.36 3.14 6.71 2.15 2.37 3.40 0.75
   KUR 2.13 0.90 2.26 2.16 0.71 2.13 2.22 2.20 2.16 2.15 2.15 2.63 2.16 2.78 2.74 1.87 2.17 2.29 0.83
   PUT 2.32 1.09 2.54 2.52 0.62 2.32 2.40 2.41 2.22 2.32 2.34 1.31 2.52 3.23 3.41 2.22 2.53 2.72 0.32
   JAF 11.80 3.66 11.52 11.81 1.64 11.80 12.46 11.91 11.94 11.97 11.86 10.86 11.81 14.86 12.20 11.46 11.61 11.99 9.02
   KIL 11.16 2.90 10.22 11.16 1.53 11.16 11.80 12.80 11.15 11.18 12.03 11.28 11.16 14.06 17.20 10.49 11.14 13.98 3.89
   MAN 2.18 1.00 1.17 2.05 0.85 2.18 2.32 2.92 2.19 2.22 2.58 1.68 2.05 2.52 4.24 1.92 2.02 3.07 0.92
   MUL 4.59 1.48 3.40 4.58 0.89 4.59 4.86 6.35 4.47 4.61 5.52 3.32 4.58 5.72 10.66 4.01 4.56 7.42 0.68
   VAV 0.77 1.50 4.04 4.04 0.86 0.77 0.81 1.19 0.79 0.80 0.99 0.41 4.04 5.08 5.41 3.96 3.99 4.68 0.48
   KEG 1.04 0.81 1.27 0.82 0.46 1.04 1.30 1.25 1.09 1.11 1.17 1.56 0.82 1.40 1.70 0.66 0.94 1.11 1.80
   RAT 1.07 1.27 0.88 1.03 0.68 1.07 1.31 1.26 1.10 1.17 1.16 0.59 1.03 1.68 1.83 0.82 1.17 1.24 4.60
   GAL 5.94 3.75 6.16 5.43 0.96 5.94 6.85 6.26 6.01 6.12 6.13 4.82 5.43 7.35 6.25 5.17 5.50 5.72 3.94
   HAM 1.93 1.90 2.09 1.93 0.80 1.93 2.24 2.53 1.94 2.01 2.28 1.64 1.93 2.64 3.50 1.64 1.96 2.47 0.64
   MAT 1.22 2.24 1.31 1.59 0.76 1.22 1.41 1.53 1.32 1.38 1.40 1.15 1.59 2.16 2.41 1.23 1.67 1.88 0.56
   BAD 4.32 1.84 4.18 4.32 0.79 4.32 2.63 3.88 3.58 3.73 3.91 2.42 4.32 5.72 6.18 4.05 4.36 4.79 2.57
   MON 0.00 0.48 0.00 0.00 0.74 0.00 0.00 0.57 0.24 0.24 0.53 0.44 0.00 0.00 2.42 0.18 0.02 0.60 2.90
   COL 3.57 2.19 3.56 3.59 1.01 3.57 3.66 3.58 3.50 3.58 3.57 2.38 3.59 3.72 3.37 3.26 3.43 3.39 0.45
   GAM 2.25 1.75 2.46 2.46 0.66 2.25 2.31 2.26 2.20 2.26 2.25 1.58 2.46 2.55 2.21 2.14 2.28 2.23 0.26
   KLU 3.86 2.45 3.51 3.39 0.71 3.86 3.95 3.89 3.77 3.86 3.86 2.95 3.39 3.52 2.68 3.02 3.31 2.73 0.37
Table 3: MASE from training set: 2006-2019 for weekly data
Base Forecasting HTS based on ARIMA HTS based on ETS
series arm snv nve ets avg bup top ols mit var stc cov ebup etop eols emit evar estc ecov
SL 2.25 2.18 4.35 4.11 0.85 2.50 2.25 2.40 4.51 2.94 2.79 4.14 4.53 4.11 4.24 5.73 4.81 4.63 3.66
Province
   CN 4.92 2.05 4.59 5.22 0.60 4.80 3.06 3.94 5.72 4.81 4.60 3.30 4.64 4.06 4.28 5.97 4.85 4.69 3.58
   ET 7.28 2.17 6.39 13.83 1.66 1.96 4.62 4.96 4.87 4.42 4.20 8.12 6.33 10.53 11.11 10.28 10.12 9.59 11.17
   NC 2.67 1.36 3.15 3.26 0.85 1.74 1.64 1.56 3.70 2.08 1.44 5.91 2.53 2.55 1.47 4.40 2.87 2.15 5.07
   NW 2.32 1.01 2.26 2.36 0.67 2.19 1.48 1.40 3.14 2.21 2.07 3.07 2.49 1.88 1.67 3.13 2.42 2.26 2.45
   NT 12.76 3.23 13.61 12.39 1.32 3.41 8.35 9.17 7.18 7.29 7.23 5.25 13.32 9.78 10.85 12.97 12.84 12.08 12.46
   SB 1.00 1.10 1.36 1.94 0.51 0.84 0.58 0.31 2.94 0.86 0.75 2.71 2.52 1.46 1.33 2.76 2.24 2.05 4.91
   ST 1.74 3.01 2.96 3.54 0.88 2.76 1.11 0.92 3.45 2.41 1.94 2.09 3.49 2.80 2.57 4.03 3.48 3.22 1.92
   UVA 2.48 1.28 2.77 2.47 0.74 1.11 1.59 0.64 3.20 1.60 1.25 2.21 2.17 1.92 0.65 2.98 2.26 1.81 2.88
   WT 2.09 2.23 3.56 3.66 0.84 2.08 1.28 1.81 4.06 1.97 2.01 3.79 3.64 2.94 3.42 4.70 3.50 3.58 3.50
Districts
   KN 3.96 2.03 3.63 3.86 0.62 3.96 2.44 3.57 4.87 3.98 3.87 2.06 3.86 3.28 3.70 5.15 4.12 3.89 2.73
   MTL 7.98 2.12 7.74 7.22 0.74 7.98 5.39 6.70 8.62 7.99 7.69 7.46 7.22 6.64 6.68 8.68 7.31 7.29 7.65
   NE 1.38 1.67 3.00 2.84 0.94 1.38 0.92 4.25 3.19 1.39 0.47 2.35 2.84 2.52 5.23 3.68 2.86 4.77 2.22
   AM 2.76 1.99 2.86 2.57 1.44 2.76 5.81 14.77 4.24 3.16 11.63 4.23 2.57 3.99 22.83 4.22 3.08 15.97 2.79
   BAT 1.65 1.37 4.20 4.20 1.82 1.65 4.09 3.11 2.90 2.77 2.66 4.40 4.20 7.47 6.99 7.19 6.48 6.10 8.35
   KAL 1.53 1.29 3.41 3.32 0.99 1.53 3.19 3.83 5.84 4.89 3.22 12.16 3.32 5.40 7.12 8.56 8.50 5.91 13.21
   TRC 2.56 3.65 10.98 10.97 2.32 2.56 5.00 5.01 5.07 4.99 4.39 7.21 10.97 17.80 14.87 13.98 14.70 13.63 11.09
   APU 1.41 1.23 3.30 2.60 0.79 1.41 1.32 0.93 3.86 1.81 1.18 7.05 2.60 2.62 1.02 4.43 3.00 2.33 4.81
   POL 2.22 1.47 2.46 2.09 0.89 2.22 2.14 2.69 2.91 2.41 1.85 2.80 2.09 2.18 3.06 3.80 2.26 1.69 5.03
   KUR 2.05 0.92 2.04 2.38 0.70 2.05 1.37 1.40 3.05 2.08 1.95 4.05 2.38 1.84 1.74 3.04 2.32 2.19 3.71
   PUT 2.33 1.11 2.52 2.58 0.61 2.33 1.60 1.36 3.19 2.35 2.19 1.51 2.58 1.88 1.56 3.17 2.51 2.29 1.50
   JAF 4.23 3.81 16.09 16.09 1.42 4.23 16.11 5.40 8.30 8.78 4.87 5.12 16.09 11.82 15.38 15.39 15.47 15.73 14.90
   KIL 10.77 3.38 13.54 10.73 1.43 10.77 131.98 40.97 13.27 11.24 30.92 16.47 10.73 7.73 4.70 12.94 10.69 4.29 13.84
   MAN 0.70 0.94 3.65 2.43 0.75 0.70 8.88 10.51 1.78 1.35 7.18 4.02 2.43 1.73 2.19 2.96 2.35 0.62 6.14
   MUL 2.05 1.60 6.12 5.63 0.96 2.05 26.20 34.57 3.62 2.25 23.74 4.53 5.63 4.12 9.40 6.50 5.61 3.47 3.82
   VAV 0.97 1.44 6.13 6.13 0.79 0.97 11.84 6.85 2.80 2.56 4.89 2.50 6.13 4.52 3.64 5.90 5.94 4.88 3.44
   KEG 0.60 0.84 1.35 1.56 0.49 0.60 0.39 0.28 2.35 0.61 0.51 3.92 1.56 1.01 0.50 2.33 1.44 1.14 1.05
   RAT 1.10 1.36 1.43 3.46 0.68 1.10 0.78 0.60 3.42 1.13 1.01 2.26 3.46 1.88 2.23 3.05 3.02 2.96 9.99
   GAL 4.00 3.97 4.17 4.42 0.95 4.00 1.63 2.52 4.18 3.72 3.33 2.39 4.42 3.53 3.68 4.85 4.41 4.20 2.53
   HAM 1.67 1.98 1.80 2.65 0.80 1.67 0.65 1.35 2.25 1.58 0.47 1.42 2.65 2.18 1.71 2.46 2.65 2.31 1.96
   MAT 1.81 2.27 2.06 2.63 0.77 1.81 0.71 0.29 2.98 1.28 1.13 1.93 2.63 2.09 1.89 3.61 2.61 2.42 2.48
   BAD 1.92 1.97 4.62 3.61 0.79 1.92 2.75 0.50 4.55 2.61 2.04 3.70 3.61 3.20 1.92 4.35 3.74 3.31 3.71
   MON 0.09 0.46 0.00 0.00 0.78 0.09 0.13 1.96 1.01 0.07 0.09 0.70 0.00 0.00 2.39 0.79 0.03 0.49 2.17
   COL 2.24 2.31 3.98 3.85 1.01 2.24 1.37 2.05 3.77 2.14 2.19 2.86 3.85 3.24 3.69 4.22 3.77 3.81 2.57
   GAM 1.64 1.80 2.98 3.06 0.64 1.64 1.05 1.42 4.03 1.50 1.58 4.66 3.06 2.38 2.88 4.72 2.84 3.01 4.63
   KLU 2.28 2.58 2.90 3.63 0.72 2.28 1.34 1.64 3.97 2.24 2.11 3.33 3.63 2.75 3.11 4.97 3.57 3.48 2.75
Table 4: MASE from training set: 2006-2019 for monthly data
Base Forecasting HTS based on ARIMA HTS based on ETS
series arm snv nve ets avg bup top ols mit var stc ebup etop eols emit evar estc
SL 2.65 2.17 4.38 4.64 0.69 2.82 2.65 2.68 2.87 2.85 2.81 3.76 4.64 4.57 3.63 3.95 4.18
Province
   CN 1.84 1.94 5.41 5.55 0.31 1.99 1.72 1.67 1.99 1.92 1.84 5.07 6.19 5.89 4.81 5.31 5.53
   ET 1.80 2.19 3.44 2.30 2.03 1.82 1.45 1.43 1.80 1.78 1.53 2.08 2.42 2.53 2.23 2.16 2.33
   NC 2.68 1.44 2.68 2.83 0.81 2.43 2.36 1.67 2.52 2.53 2.31 2.62 3.17 4.74 2.34 2.71 3.41
   NW 1.83 0.99 2.24 2.47 0.51 1.45 1.71 1.51 1.63 1.61 1.59 2.58 2.75 2.94 2.33 2.56 2.68
   NT 2.66 3.36 11.15 6.25 1.18 3.34 2.27 2.29 3.13 3.06 2.74 3.06 6.91 6.74 3.93 4.00 5.33
   SB 0.99 0.97 1.86 2.20 0.24 0.72 0.85 0.69 0.83 0.80 0.80 1.72 2.50 2.51 2.08 1.93 2.12
   ST 2.68 3.00 3.66 4.45 0.73 2.20 2.45 2.30 2.32 2.34 2.35 4.32 4.95 4.97 4.17 4.39 4.64
   UVA 1.81 1.22 2.78 3.01 0.59 1.30 1.64 1.22 1.46 1.48 1.39 2.74 3.34 4.21 2.55 2.86 3.32
   WT 4.26 2.20 4.26 4.49 0.65 4.26 3.81 4.20 4.22 4.22 4.24 4.41 4.98 4.61 4.07 4.53 4.51
Districts
   KN 1.72 1.87 5.07 5.43 0.31 1.72 1.58 1.57 1.71 1.64 1.65 5.43 6.64 5.79 5.26 5.71 5.63
   MTL 2.94 2.18 6.74 3.89 0.61 2.94 2.70 2.45 3.00 2.92 2.72 3.89 4.67 5.14 3.41 4.00 4.58
   NE 1.25 1.84 3.21 3.61 0.93 1.25 0.98 2.27 1.21 1.25 0.78 3.61 4.44 9.13 2.94 3.64 6.65
   AM 2.60 2.07 2.60 2.60 1.44 2.60 2.15 3.06 2.45 2.60 2.14 2.60 3.05 5.81 2.59 2.61 4.10
   BAT 1.47 1.28 2.36 2.20 2.20 1.47 1.39 1.39 1.46 1.48 1.45 2.20 2.25 2.21 2.36 2.20 2.18
   KAL 1.33 1.18 1.92 1.35 0.90 1.33 1.10 0.79 1.27 1.29 1.07 1.35 1.58 1.70 1.55 1.43 1.55
   TRC 2.47 3.79 5.48 2.91 2.46 2.47 1.92 1.90 2.44 2.43 2.20 2.91 3.21 3.28 3.00 3.00 3.11
   APU 2.75 1.32 2.75 2.99 0.74 2.75 2.66 2.20 2.87 2.87 2.66 2.99 3.58 4.52 2.69 3.09 3.56
   POL 1.55 1.55 2.26 1.64 0.86 1.55 1.54 2.25 1.58 1.58 1.38 1.64 2.04 4.72 1.65 1.68 2.79
   KUR 0.90 0.89 1.82 2.11 0.54 0.90 1.08 0.94 1.14 1.10 1.01 2.11 2.24 2.40 1.63 2.08 2.18
   PUT 2.26 1.13 2.85 3.26 0.46 2.26 2.62 2.33 2.33 2.34 2.43 3.26 3.46 3.70 3.32 3.25 3.38
   JAF 4.24 4.27 14.50 3.89 1.37 4.24 2.55 3.91 3.93 3.90 4.05 3.89 8.72 4.95 5.10 5.25 4.53
   KIL 4.37 3.57 9.94 4.09 1.49 4.37 4.65 13.04 4.41 4.35 8.99 4.09 9.80 24.82 4.42 4.16 17.07
   MAN 0.39 0.80 1.93 0.56 0.56 0.39 0.58 3.53 0.38 0.40 2.30 0.56 0.71 6.38 0.61 0.57 4.02
   MUL 5.89 1.40 5.03 1.69 0.73 5.89 6.20 14.13 5.85 5.88 10.02 1.69 3.98 22.65 2.00 1.72 14.81
   VAV 0.74 1.35 4.02 1.19 0.70 0.74 0.83 2.46 0.72 0.68 1.73 1.19 2.83 4.89 1.47 1.49 3.51
   KEG 0.52 0.84 1.63 2.15 0.44 0.52 0.62 0.49 0.67 0.58 0.59 2.15 3.00 2.85 2.09 2.33 2.51
   RAT 0.95 1.19 2.09 1.29 0.36 0.95 1.11 0.91 1.00 1.04 1.04 1.29 1.80 2.01 1.99 1.43 1.58
   GAL 2.21 4.11 4.52 5.36 0.84 2.21 2.51 2.29 2.29 2.31 2.34 5.36 6.17 5.90 5.28 5.41 5.63
   HAM 2.38 1.99 2.65 3.59 0.65 2.38 2.59 2.53 2.40 2.43 2.61 3.59 4.08 4.60 3.95 3.64 4.09
   MAT 1.96 2.33 3.10 3.39 0.61 1.96 2.15 2.04 2.15 2.17 2.08 3.39 3.90 3.92 2.93 3.49 3.65
   BAD 1.70 1.97 4.60 4.53 0.68 1.70 2.21 1.48 1.92 1.96 1.78 4.53 5.53 5.77 4.21 4.69 5.02
   MON 0.63 0.51 0.12 0.12 0.80 0.63 0.71 1.16 0.70 0.69 0.73 0.12 0.14 1.70 0.12 0.17 0.75
   COL 4.73 2.31 4.73 4.72 0.81 4.73 4.23 4.69 4.70 4.70 4.72 4.72 5.38 4.86 4.14 4.84 4.80
   GAM 3.37 1.80 3.37 3.93 0.49 3.37 3.02 3.32 3.32 3.32 3.35 3.93 4.37 4.09 3.88 4.07 4.01
   KLU 4.52 2.66 4.52 3.98 0.50 4.52 4.02 4.37 4.50 4.51 4.47 3.98 4.57 4.43 3.66 4.03 4.22
Table 5: MASE from training set: 2006-2019 for quarterly data
Base Forecasting HTS based on ARIMA HTS based on ETS
series arm nve ets avg bup top ols mit var stc ebup etop eols emit evar estc
SL 0.24 2.16 1.29 0.24 0.75 0.24 0.34 0.84 0.80 0.68 1.50 1.29 1.32 1.54 1.49 1.43
Province
   CN 0.10 1.91 0.92 0.10 0.10 0.52 0.68 0.02 0.12 0.30 0.87 0.74 0.72 0.89 0.88 0.83
   ET 1.68 0.52 0.18 2.12 1.97 2.37 2.79 1.84 1.92 2.28 0.28 0.47 0.55 0.19 0.26 0.38
   NC 0.33 1.14 0.72 0.33 0.33 0.12 2.23 0.40 0.32 0.33 1.40 0.58 0.11 1.23 1.20 0.84
   NW 0.58 1.09 0.67 0.58 0.58 0.18 0.01 0.62 0.56 0.43 1.56 0.57 0.78 1.35 1.32 1.07
   NT 3.12 4.06 3.11 0.32 2.46 1.18 1.48 2.74 2.69 2.00 3.12 2.67 2.61 3.17 3.11 2.85
   SB 0.21 1.03 0.24 0.21 0.21 0.18 0.35 0.25 0.19 0.06 1.13 0.15 0.36 0.92 0.90 0.64
   ST 0.42 3.16 2.60 0.42 1.29 0.01 0.03 1.14 1.08 0.62 2.38 2.32 2.33 2.46 2.43 2.41
   UVA 0.58 1.34 1.34 0.58 0.58 0.20 0.95 0.64 0.57 0.18 1.47 1.19 0.88 1.43 1.42 1.27
   WT 2.00 2.53 1.84 0.61 1.20 0.97 1.63 1.35 1.37 1.54 1.38 1.62 1.67 1.55 1.48 1.59
Districts
   KN 0.21 1.89 0.99 0.21 0.21 0.62 0.47 0.12 0.24 0.30 0.99 0.85 0.92 1.01 1.01 0.97
   MTL 0.21 2.04 0.21 0.21 0.21 0.19 0.72 0.28 0.20 0.11 0.21 0.14 0.03 0.24 0.21 0.14
   NE 0.34 1.36 1.75 0.34 0.34 0.22 3.88 0.41 0.34 1.12 1.75 1.57 0.66 1.74 1.75 1.45
   AM 0.90 0.25 0.15 0.90 0.90 1.15 3.52 0.81 0.89 1.88 0.15 0.24 1.00 0.09 0.14 0.48
   BAT 2.46 1.45 0.61 2.83 2.46 2.96 2.98 2.34 2.41 2.65 0.61 0.83 0.78 0.53 0.59 0.68
   KAL 0.52 0.20 0.72 0.52 0.52 0.79 1.15 0.40 0.47 0.76 0.72 0.59 0.52 0.81 0.74 0.65
   TRC 2.10 0.12 0.73 2.10 2.10 2.36 2.66 1.98 2.06 2.31 0.73 0.87 0.91 0.65 0.71 0.80
   APU 0.40 1.21 1.58 0.40 0.40 0.06 1.57 0.47 0.40 0.11 1.58 0.72 0.59 1.38 1.36 1.15
   POL 0.19 0.97 1.04 0.19 0.19 0.24 3.34 0.25 0.19 0.73 1.04 0.32 0.74 0.94 0.88 0.26
   KUR 0.57 0.95 1.34 0.57 0.57 0.17 0.11 0.61 0.56 0.45 1.34 0.45 0.72 1.13 1.10 0.95
   PUT 0.59 1.33 1.94 0.59 0.59 0.19 0.21 0.65 0.57 0.38 1.94 0.78 0.88 1.73 1.69 1.27
   JAF 3.11 4.81 3.92 0.47 3.11 1.60 2.84 3.43 3.39 2.98 3.92 3.38 3.78 3.97 3.91 3.84
   KIL 3.79 4.90 4.83 1.54 3.79 1.59 3.56 3.81 3.84 0.31 4.83 4.06 1.02 4.92 4.82 2.81
   MAN 0.03 0.37 0.03 0.03 0.03 0.24 1.72 0.12 0.07 0.80 0.03 0.05 0.87 0.05 0.03 0.45
   MUL 0.87 1.78 2.00 0.33 0.87 0.28 5.11 0.82 0.90 1.96 2.00 1.72 1.09 2.06 2.00 0.36
   VAV 0.30 1.68 0.30 0.30 0.30 0.04 0.79 0.48 0.41 0.21 0.30 0.19 0.26 0.31 0.30 0.00
   KEG 0.43 0.89 1.65 0.43 0.43 0.10 0.05 0.46 0.42 0.31 1.65 0.60 0.97 1.39 1.38 1.22
   RAT 0.09 1.13 0.41 0.09 0.09 0.52 0.70 0.02 0.10 0.25 0.41 0.41 0.43 0.29 0.24 0.12
   GAL 2.31 3.96 2.81 0.11 2.31 0.15 1.20 2.09 2.13 1.75 2.81 2.73 2.76 2.91 2.86 2.84
   HAM 0.54 2.39 1.44 0.54 0.54 0.22 1.63 0.51 0.40 0.56 1.44 1.40 1.36 1.47 1.46 1.50
   MAT 0.62 2.59 2.30 0.62 0.62 0.02 0.42 0.48 0.37 0.10 2.30 2.24 2.26 2.38 2.36 2.32
   BAD 0.30 1.79 1.78 0.30 0.30 0.08 0.97 0.37 0.30 0.03 1.78 1.42 1.30 1.72 1.73 1.62
   MON 0.85 0.58 0.85 0.85 0.85 0.53 0.79 0.89 0.85 0.43 0.85 0.73 0.23 0.85 0.82 0.64
   COL 1.62 2.63 1.91 0.76 1.62 1.33 1.92 1.68 1.78 1.85 1.91 2.22 2.12 2.02 1.98 2.06
   GAM 0.52 2.11 0.61 0.52 0.52 0.40 0.86 0.80 0.71 0.79 0.61 0.73 0.84 0.84 0.74 0.77
   KLU 1.58 2.92 1.60 0.28 1.58 1.27 2.58 1.57 1.64 2.37 1.60 1.90 2.28 1.72 1.64 2.09
Table 6: MASE from training set: 2006-2019 for yearly data
Base Forecasting HTS based on ARIMA HTS based on ETS
series arm snv nve ets avg bup top ols mit var stc cov ebup etop eols emit evar estc ecov
SL 0.66 0.76 1.45 1.86 0.64 0.85 0.66 0.68 0.89 0.83 0.79 1.36 1.68 1.86 1.85 1.81 1.73 1.77 1.83
Province
   CN 0.45 0.77 1.49 2.12 0.50 0.45 0.32 0.29 0.50 0.44 0.38 1.04 1.90 2.24 2.17 2.08 2.00 2.06 2.34
   ET 1.33 0.75 2.64 3.29 0.85 0.86 1.05 1.03 1.04 1.04 0.99 1.01 2.94 3.48 3.35 3.07 3.09 3.19 2.76
   NC 0.58 0.53 1.19 1.17 0.58 0.59 0.46 0.19 0.68 0.58 0.39 1.90 1.17 1.24 1.63 1.23 1.18 1.32 2.03
   NW 0.56 0.37 0.90 0.90 0.53 0.60 0.45 0.42 0.61 0.57 0.54 0.94 1.12 0.95 1.07 1.08 1.03 1.04 1.18
   NT 4.00 1.47 4.52 5.76 1.15 4.04 3.18 3.55 4.00 4.00 3.76 3.87 5.85 6.07 6.05 5.80 5.82 5.99 4.06
   SB 0.43 0.41 0.38 0.85 0.45 0.42 0.32 0.28 0.49 0.40 0.38 1.45 0.25 0.91 0.75 0.65 0.50 0.57 0.45
   ST 0.69 1.03 1.27 1.62 0.63 1.26 0.54 0.64 1.10 1.04 0.90 1.72 1.23 1.71 1.64 1.43 1.38 1.47 1.40
   UVA 0.65 0.47 0.99 1.70 0.60 0.35 0.52 0.09 0.55 0.46 0.36 1.40 1.40 1.80 1.90 1.64 1.54 1.65 1.37
   WT 0.70 0.77 1.17 1.20 0.65 0.69 0.54 0.65 0.72 0.66 0.68 1.12 1.35 1.27 1.27 1.43 1.33 1.29 1.72
Districts
   KN 0.41 0.73 1.14 1.67 0.45 0.41 0.31 0.35 0.47 0.41 0.39 1.11 1.67 1.98 1.79 1.86 1.79 1.74 2.00
   MTL 0.63 0.97 2.70 2.75 0.64 0.63 0.52 0.37 0.65 0.63 0.52 0.78 2.75 3.18 3.23 2.85 2.78 3.02 3.79
   NE 0.76 0.77 1.92 1.72 0.74 0.76 0.61 0.73 0.82 0.76 0.29 1.45 1.72 2.01 3.79 1.82 1.74 2.91 1.33
   AM 0.73 0.79 0.84 1.01 0.75 0.73 0.81 1.47 0.82 0.77 1.27 1.77 1.01 1.18 2.86 1.12 1.07 2.13 1.16
   BAT 1.12 0.58 2.21 2.21 0.89 1.12 1.45 1.25 1.25 1.25 1.21 1.31 2.21 2.65 2.47 2.27 2.30 2.36 2.18
   KAL 0.54 0.42 1.30 1.30 0.58 0.54 0.66 0.68 0.76 0.80 0.65 0.22 1.30 1.53 1.61 1.46 1.50 1.48 0.80
   TRC 0.77 1.14 4.01 4.87 1.02 0.77 0.90 0.90 0.95 0.93 0.87 1.15 4.87 5.70 5.18 5.00 5.01 5.05 4.84
   APU 0.57 0.51 1.21 1.22 0.57 0.57 0.44 0.09 0.67 0.56 0.41 1.79 1.22 1.29 1.56 1.29 1.23 1.34 2.41
   POL 0.63 0.57 1.09 1.04 0.58 0.63 0.50 0.51 0.72 0.63 0.35 2.05 1.04 1.09 1.68 1.07 1.04 1.25 1.19
   KUR 0.62 0.37 0.92 1.32 0.56 0.62 0.46 0.46 0.64 0.59 0.57 1.02 1.32 1.11 1.27 1.27 1.20 1.25 1.30
   PUT 0.58 0.37 0.87 0.86 0.49 0.58 0.43 0.38 0.59 0.56 0.51 0.84 0.86 0.74 0.81 0.84 0.81 0.78 1.03
   JAF 4.65 1.55 4.88 6.30 1.14 4.65 3.67 4.52 4.60 4.60 4.58 4.42 6.30 6.54 6.35 6.23 6.26 6.33 4.46
   KIL 3.76 0.98 3.45 3.76 0.82 3.76 2.98 2.25 3.73 3.76 2.90 3.41 3.76 3.90 4.43 3.82 3.76 4.22 3.61
   MAN 0.67 0.61 0.72 1.46 0.67 0.67 0.58 0.82 0.61 0.67 0.38 0.41 1.46 1.52 1.99 1.43 1.45 1.81 0.53
   MUL 0.71 0.64 1.47 1.97 0.72 0.71 0.58 1.40 0.70 0.70 0.59 0.96 1.97 2.05 2.87 2.00 1.97 2.58 1.06
   VAV 0.92 1.09 2.93 3.58 0.90 0.92 0.74 0.28 0.93 0.91 0.53 1.38 3.58 3.70 3.88 3.56 3.57 3.79 2.34
   KEG 0.43 0.30 0.47 0.30 0.46 0.43 0.33 0.29 0.48 0.42 0.39 1.15 0.30 0.95 0.75 0.53 0.44 0.60 0.16
   RAT 0.48 0.56 0.39 0.30 0.48 0.48 0.40 0.36 0.55 0.46 0.44 1.81 0.30 0.83 0.74 0.81 0.61 0.56 0.98
   GAL 2.12 1.30 2.14 1.90 0.64 2.12 0.94 1.64 1.91 1.87 1.84 2.31 1.90 2.64 2.22 2.06 2.04 2.09 2.38
   HAM 0.67 0.79 0.86 0.80 0.67 0.67 0.27 0.44 0.60 0.56 0.10 0.94 0.80 1.13 1.53 0.93 0.86 1.24 1.21
   MAT 0.59 0.83 0.49 0.66 0.62 0.59 0.25 0.08 0.43 0.35 0.29 1.40 0.66 0.92 1.00 0.92 0.85 0.86 0.44
   BAD 0.62 0.77 1.75 2.49 0.62 0.62 0.92 0.37 0.89 0.77 0.63 2.06 2.49 3.19 2.93 2.84 2.69 2.71 2.33
   MON 0.00 0.17 0.00 0.00 0.58 0.00 0.00 0.27 0.10 0.06 0.03 0.48 0.00 0.00 0.47 0.08 0.05 0.24 0.12
   COL 0.83 0.90 1.47 1.91 0.80 0.83 0.66 0.80 0.86 0.81 0.82 1.35 1.91 1.78 1.84 1.95 1.88 1.86 2.26
   GAM 0.53 0.59 0.82 0.82 0.51 0.53 0.42 0.49 0.54 0.49 0.51 0.83 0.82 0.77 0.76 0.91 0.79 0.77 1.09
   KLU 0.68 0.84 1.21 1.17 0.57 0.68 0.53 0.59 0.73 0.67 0.65 1.16 1.17 1.09 0.99 1.26 1.16 1.04 1.79
Table 7: MASE from training set: 2016-2019 for weekly data
Base Forecasting HTS based on ARIMA HTS based on ETS
series arm snv nve ets avg bup top ols mit var stc cov ebup etop eols emit evar estc ecov
SL 0.70 0.77 1.54 1.41 0.63 0.63 0.70 0.70 0.73 0.66 0.67 1.13 1.79 1.41 1.43 1.56 1.59 1.57 1.86
Province
   CN 0.69 0.80 1.80 2.28 0.50 0.72 0.70 0.72 0.78 0.71 0.72 1.22 2.20 2.12 2.12 2.10 2.20 2.15 2.15
   ET 0.71 0.74 2.17 1.48 0.75 0.54 0.73 0.70 0.62 0.61 0.64 1.28 2.02 1.42 1.43 1.71 1.73 1.62 1.41
   NC 0.58 0.55 1.27 1.62 0.58 0.59 0.60 0.71 0.66 0.59 0.64 1.08 1.63 1.50 1.14 1.48 1.62 1.36 1.76
   NW 0.57 0.38 0.85 0.84 0.52 0.58 0.58 0.59 0.63 0.58 0.58 0.69 0.95 0.79 0.76 0.86 0.92 0.84 1.20
   NT 1.18 1.51 6.36 4.32 1.07 0.93 1.11 1.18 1.06 1.04 1.10 1.85 6.23 3.95 4.28 4.49 4.62 4.96 4.72
   SB 0.46 0.42 0.52 1.00 0.46 0.26 0.47 0.40 0.41 0.33 0.37 1.07 0.77 0.96 0.79 0.90 0.88 0.82 1.89
   ST 0.65 1.05 1.04 1.37 0.64 0.67 0.67 0.69 0.71 0.67 0.68 0.96 1.52 1.31 1.26 1.50 1.47 1.36 1.98
   UVA 0.65 0.49 1.05 1.11 0.62 0.34 0.66 0.60 0.53 0.46 0.52 1.27 1.18 1.05 0.79 1.15 1.14 0.98 2.00
   WT 0.71 0.79 1.26 1.18 0.65 0.71 0.72 0.72 0.80 0.72 0.71 1.05 1.52 1.12 1.23 1.26 1.28 1.33 1.50
Districts
   KN 0.59 0.76 1.35 1.70 0.45 0.59 0.57 0.59 0.65 0.58 0.59 0.97 1.70 1.64 1.67 1.62 1.71 1.69 1.58
   MTL 1.24 0.99 3.60 4.25 0.62 1.24 1.21 1.24 1.30 1.23 1.24 2.17 4.25 4.10 4.11 4.08 4.25 4.17 4.23
   NE 0.77 0.86 1.55 1.59 0.78 0.77 0.77 0.89 0.84 0.77 0.81 1.62 1.59 1.55 1.40 1.62 1.59 1.30 2.89
   AM 0.70 0.76 1.10 0.99 0.74 0.70 0.81 1.38 0.69 0.69 1.06 0.93 0.99 0.71 2.65 0.87 0.94 1.58 1.60
   BAT 0.60 0.54 1.66 2.03 0.75 0.60 0.76 0.67 0.61 0.62 0.64 0.94 2.03 1.26 1.67 1.66 1.83 1.78 1.75
   KAL 0.54 0.41 1.07 1.06 0.56 0.54 0.66 0.65 0.66 0.63 0.59 0.95 1.06 0.73 0.67 0.62 0.64 0.79 0.84
   TRC 0.61 1.14 3.41 2.94 0.93 0.61 0.80 0.75 0.70 0.69 0.69 2.31 2.94 2.35 2.66 2.91 2.78 2.68 2.96
   APU 0.59 0.52 1.40 1.79 0.59 0.59 0.60 0.68 0.68 0.59 0.63 1.22 1.79 1.67 1.35 1.57 1.77 1.59 1.84
   POL 0.60 0.59 0.99 1.31 0.60 0.60 0.62 0.80 0.62 0.60 0.67 0.86 1.31 1.18 1.23 1.32 1.31 1.00 1.66
   KUR 0.59 0.38 0.84 1.00 0.56 0.59 0.60 0.60 0.65 0.59 0.59 0.60 1.00 0.80 0.83 0.95 0.99 0.90 1.08
   PUT 0.56 0.38 0.85 0.88 0.49 0.56 0.56 0.58 0.59 0.56 0.57 0.86 0.88 0.76 0.66 0.74 0.83 0.76 1.34
   JAF 0.96 1.60 6.74 6.73 1.07 0.96 1.13 0.99 1.08 1.08 0.98 1.57 6.73 4.21 6.23 4.57 4.68 6.40 4.78
   KIL 1.02 1.08 4.33 3.45 0.86 1.02 1.24 2.06 1.11 1.03 1.61 3.07 3.45 2.09 6.28 3.14 3.42 3.50 3.55
   MAN 0.60 0.57 2.21 1.41 0.61 0.60 0.73 1.32 0.64 0.61 1.01 1.77 1.41 0.82 4.32 0.99 1.36 2.47 0.41
   MUL 0.88 0.64 2.46 2.17 0.76 0.88 1.05 2.49 0.94 0.88 1.76 1.64 2.17 1.34 9.98 1.99 2.15 6.01 2.74
   VAV 1.50 1.07 4.55 4.71 0.86 1.50 1.74 1.90 1.54 1.51 1.77 2.90 4.71 3.48 3.72 4.67 4.66 3.39 5.33
   KEG 0.09 0.30 0.49 0.50 0.47 0.09 0.14 0.21 0.19 0.11 0.17 0.63 0.50 0.61 0.51 0.54 0.60 0.54 1.05
   RAT 0.52 0.59 0.62 1.20 0.52 0.52 0.91 0.69 0.72 0.63 0.65 1.96 1.20 1.42 1.16 1.37 1.27 1.20 3.03
   GAL 0.71 1.33 1.40 1.94 0.63 0.71 0.71 0.72 0.73 0.71 0.72 1.10 1.94 1.66 1.74 1.95 1.89 1.82 2.40
   HAM 0.66 0.80 0.73 1.24 0.69 0.66 0.66 0.69 0.69 0.66 0.67 0.43 1.24 1.01 0.77 1.13 1.19 0.94 1.08
   MAT 0.63 0.83 0.76 1.13 0.63 0.63 0.62 0.63 0.68 0.62 0.63 1.01 1.13 1.01 0.91 1.13 1.07 1.00 1.86
   BAD 0.61 0.80 1.89 2.11 0.63 0.61 1.17 0.84 0.86 0.78 0.77 2.19 2.11 1.88 1.73 2.11 2.07 1.93 3.41
   MON 0.00 0.16 0.00 0.00 0.59 0.00 0.00 0.25 0.08 0.04 0.17 0.28 0.00 0.00 0.50 0.13 0.07 0.22 0.29
   COL 0.85 0.91 1.56 1.97 0.78 0.85 0.86 0.85 0.94 0.86 0.85 0.97 1.97 1.40 1.74 1.58 1.69 1.82 1.25
   GAM 0.57 0.60 1.00 1.02 0.53 0.57 0.58 0.58 0.67 0.58 0.57 1.29 1.02 0.74 0.80 0.81 0.74 0.88 1.58
   KLU 0.61 0.86 0.97 1.49 0.57 0.61 0.62 0.62 0.68 0.61 0.61 0.90 1.49 1.25 0.99 1.47 1.46 1.07 1.70
Table 8: MASE from training set: 2016-2019 for monthly data
Base Forecasting HTS based on ARIMA HTS based on ETS
series arm snv nve ets avg bup top ols mit var stc ebup etop eols emit evar estc
SL 0.65 0.77 1.55 1.92 0.52 0.60 0.65 0.65 0.68 0.62 0.63 1.44 1.92 1.87 1.49 1.54 1.66
Province
   CN 0.68 0.77 2.14 2.58 0.43 0.59 0.70 0.68 0.71 0.63 0.64 2.18 3.03 2.79 2.30 2.36 2.52
   ET 0.57 0.76 1.19 0.73 0.71 0.70 0.53 0.54 0.63 0.65 0.61 0.76 0.80 0.97 0.81 0.76 0.87
   NC 0.58 0.59 1.09 1.50 0.52 0.76 0.60 0.74 0.73 0.70 0.71 1.48 1.77 2.87 1.40 1.49 1.93
   NW 0.67 0.38 0.86 1.12 0.52 0.59 0.68 0.66 0.67 0.64 0.64 1.15 1.31 1.41 1.10 1.15 1.22
   NT 0.92 1.59 5.29 2.07 0.92 0.92 0.87 0.81 0.93 0.91 0.84 2.34 2.49 2.96 2.32 2.25 2.76
   SB 0.45 0.38 0.72 0.90 0.45 0.44 0.45 0.46 0.55 0.44 0.45 0.62 1.08 1.12 0.67 0.72 0.86
   ST 0.75 1.04 1.27 1.54 0.56 0.73 0.76 0.77 0.79 0.74 0.75 1.62 1.80 1.89 1.53 1.61 1.72
   UVA 0.65 0.46 1.05 1.30 0.59 0.28 0.67 0.59 0.51 0.42 0.49 0.86 1.52 1.98 0.99 1.04 1.35
   WT 0.69 0.78 1.51 1.90 0.62 0.72 0.71 0.71 0.77 0.72 0.71 1.61 2.22 1.91 1.70 1.77 1.79
Districts
   KN 0.61 0.71 1.92 2.36 0.41 0.61 0.74 0.65 0.73 0.66 0.64 2.36 3.28 2.63 2.50 2.58 2.51
   MTL 0.47 1.02 3.15 1.36 0.47 0.47 0.54 0.63 0.61 0.49 0.57 1.36 1.91 2.46 1.50 1.42 1.97
   NE 0.66 0.91 1.58 2.15 0.66 0.66 0.66 1.31 0.73 0.66 1.04 2.15 3.01 7.30 2.01 2.18 4.98
   AM 0.69 0.82 1.02 1.30 0.76 0.69 0.48 0.86 0.64 0.69 0.60 1.30 1.44 2.91 1.27 1.30 2.03
   BAT 0.91 0.53 0.98 0.91 0.91 0.91 0.70 0.79 0.82 0.86 0.84 0.91 0.89 0.88 0.88 0.90 0.89
   KAL 0.48 0.38 0.61 0.55 0.48 0.48 0.41 0.38 0.39 0.43 0.41 0.55 0.53 0.69 0.60 0.55 0.63
   TRC 0.93 1.22 1.77 0.93 0.93 0.93 0.79 0.81 0.90 0.88 0.86 0.93 0.98 1.09 0.96 0.94 1.02
   APU 0.81 0.56 1.17 1.66 0.52 0.81 0.64 0.79 0.76 0.74 0.77 1.66 1.98 2.71 1.60 1.67 2.01
   POL 0.64 0.62 0.91 1.08 0.53 0.64 0.49 0.61 0.63 0.61 0.57 1.08 1.31 3.08 0.98 1.08 1.74
   KUR 0.54 0.37 0.76 0.97 0.54 0.54 0.61 0.61 0.61 0.60 0.58 0.97 1.11 1.21 0.89 0.97 1.04
   PUT 0.66 0.39 0.98 1.37 0.48 0.66 0.77 0.74 0.74 0.69 0.71 1.37 1.55 1.66 1.36 1.37 1.45
   JAF 1.00 1.86 6.31 2.61 1.00 1.00 0.95 0.97 1.02 0.99 0.98 2.61 2.78 2.78 2.56 2.49 2.73
   KIL 0.74 1.12 3.11 2.10 0.74 0.74 0.70 0.66 0.70 0.74 0.61 2.10 2.24 4.34 2.11 2.10 3.61
   MAN 0.44 0.48 1.16 0.45 0.44 0.44 0.42 0.37 0.44 0.44 0.34 0.45 0.47 1.59 0.48 0.45 1.15
   MUL 0.62 0.55 1.96 0.62 0.62 0.62 0.63 0.88 0.73 0.62 0.82 0.62 0.66 3.45 0.75 0.62 2.52
   VAV 0.76 1.05 3.12 2.38 0.76 0.76 0.72 0.69 0.75 0.76 0.67 2.38 2.52 3.37 2.45 2.37 3.04
   KEG 0.45 0.31 0.60 0.73 0.46 0.45 0.46 0.48 0.52 0.46 0.46 0.73 1.29 1.15 0.68 0.85 0.93
   RAT 0.42 0.51 0.91 0.66 0.42 0.42 0.43 0.45 0.59 0.42 0.43 0.66 0.77 1.07 0.65 0.63 0.75
   GAL 0.92 1.33 1.46 2.24 0.48 0.92 0.96 0.94 0.95 0.92 0.93 2.24 2.49 2.45 2.14 2.23 2.32
   HAM 0.52 0.80 1.06 1.06 0.63 0.52 0.55 0.58 0.52 0.52 0.56 1.06 1.19 1.53 1.01 1.06 1.24
   MAT 0.62 0.86 1.14 1.18 0.62 0.62 0.63 0.65 0.72 0.63 0.63 1.18 1.31 1.41 1.10 1.16 1.27
   BAD 0.50 0.77 1.81 1.65 0.50 0.50 1.19 0.77 0.85 0.73 0.69 1.65 2.92 2.64 1.86 1.90 2.09
   MON 0.01 0.18 0.04 0.15 0.60 0.01 0.02 0.29 0.06 0.02 0.20 0.15 0.27 0.95 0.12 0.08 0.33
   COL 0.86 0.90 1.85 1.66 0.74 0.86 0.84 0.84 0.90 0.85 0.85 1.66 2.30 1.90 1.84 1.85 1.80
   GAM 0.55 0.62 1.15 1.45 0.52 0.55 0.53 0.53 0.59 0.54 0.54 1.45 2.00 1.69 1.50 1.62 1.59
   KLU 0.83 0.89 1.51 1.85 0.55 0.83 0.81 0.79 0.88 0.82 0.80 1.85 2.58 2.52 1.80 1.89 2.26
Table 9: MASE for training set: 2016-2019 for quarterly data
Base Forecasting HTS based on ARIMA HTS based on ETS
series arm snv nve ets avg bup top ols mit var stc cov ebup etop eols emit evar estc ecov
SL 1.29 1.44 1.34 1.19 1.62 1.34 1.29 1.30 1.33 1.34 1.33 1.27 1.35 1.19 1.21 1.36 1.34 1.30 1.84
Province
   CN 2.44 1.82 1.90 2.59 1.83 1.97 2.42 2.18 2.11 2.15 2.15 1.82 2.30 2.59 2.11 2.41 2.40 2.29 2.73
   ET 1.43 2.36 1.39 1.41 1.33 1.36 1.53 1.51 1.38 1.39 1.42 1.35 1.42 1.62 1.67 1.43 1.42 1.52 1.48
   NC 0.76 1.03 0.78 0.77 0.77 0.75 0.75 0.81 0.75 0.75 0.74 0.77 0.77 0.80 1.38 0.78 0.76 0.81 2.41
   NW 0.66 0.84 0.65 0.65 0.65 0.64 0.64 0.62 0.64 0.64 0.64 0.63 0.66 0.66 0.67 0.66 0.65 0.65 1.02
   NT 5.45 3.15 5.66 5.62 3.79 5.48 5.86 5.62 5.47 5.47 5.54 5.51 5.56 6.61 6.05 5.58 5.60 5.83 5.57
   SB 0.84 0.86 0.85 0.81 0.77 0.87 0.78 0.76 0.82 0.85 0.83 0.62 0.82 0.67 0.60 0.88 0.79 0.75 2.40
   ST 2.88 2.76 2.56 2.51 2.72 2.55 2.81 2.63 2.65 2.66 2.65 2.49 2.48 2.24 2.12 2.59 2.47 2.34 3.54
   UVA 0.82 0.83 0.71 0.71 0.75 0.70 0.91 0.88 0.73 0.72 0.75 0.95 0.70 0.69 0.90 0.72 0.70 0.69 1.39
   WT 1.39 1.49 1.38 1.41 1.66 1.38 1.31 1.37 1.36 1.37 1.38 1.28 1.39 1.22 1.36 1.39 1.37 1.38 1.75
Districts
   KN 2.02 1.78 1.87 2.46 1.88 2.02 2.48 2.10 2.18 2.23 2.09 1.81 2.46 2.64 2.38 2.55 2.57 2.45 2.67
   MTL 1.91 2.18 1.98 1.91 1.85 1.91 2.16 2.12 1.96 1.96 2.09 1.98 1.91 2.35 1.78 1.98 1.93 1.88 2.46
   NE 1.09 1.11 1.17 1.07 0.92 1.09 1.36 2.18 1.11 1.13 2.03 0.96 1.07 1.54 0.62 1.22 1.08 0.98 2.73
   AM 1.04 1.24 1.54 0.96 0.97 1.04 0.96 0.82 1.05 1.02 0.82 1.41 0.96 0.87 1.24 1.09 0.96 0.70 2.60
   BAT 1.34 3.15 1.32 1.34 1.19 1.34 1.59 1.44 1.37 1.37 1.38 1.41 1.34 1.57 1.48 1.34 1.34 1.40 1.47
   KAL 0.82 1.44 0.86 0.86 0.78 0.82 0.93 0.93 0.85 0.86 0.87 0.78 0.86 0.99 1.04 0.89 0.86 0.93 1.40
   TRC 1.83 2.02 1.93 1.94 1.86 1.83 1.90 1.90 1.85 1.85 1.86 1.84 1.94 2.08 2.09 1.95 1.95 2.00 1.88
   APU 0.79 1.09 0.84 0.79 0.77 0.79 0.79 0.88 0.78 0.78 0.80 0.81 0.79 0.86 1.15 0.79 0.79 0.85 2.63
   POL 0.71 0.87 0.67 0.67 0.71 0.71 0.69 0.67 0.72 0.72 0.65 0.76 0.67 0.67 1.74 0.69 0.67 0.70 1.58
   KUR 0.52 0.68 0.52 0.52 0.52 0.52 0.50 0.50 0.51 0.51 0.51 0.65 0.52 0.53 0.53 0.53 0.52 0.52 0.72
   PUT 0.86 1.10 0.84 0.85 0.84 0.86 0.86 0.82 0.87 0.86 0.86 1.22 0.85 0.85 0.88 0.86 0.84 0.84 1.51
   JAF 7.34 3.75 7.41 7.32 4.81 7.34 7.86 7.38 7.33 7.32 7.36 7.52 7.32 8.73 7.47 7.36 7.37 7.41 7.46
   KIL 4.04 4.08 5.95 4.24 3.43 4.04 4.33 4.77 3.97 4.04 4.38 3.53 4.24 5.10 6.97 4.12 4.25 5.71 3.62
   MAN 0.81 1.21 0.82 0.85 0.79 0.81 0.84 1.03 0.81 0.81 0.90 0.73 0.85 0.97 1.81 0.86 0.86 1.31 0.89
   MUL 1.45 1.81 1.37 1.45 1.43 1.45 1.49 1.99 1.43 1.45 1.71 1.38 1.45 1.52 3.58 1.41 1.45 2.38 1.60
   VAV 1.22 1.47 1.34 1.34 1.22 1.22 1.23 1.28 1.22 1.22 1.25 1.22 1.34 1.47 1.75 1.35 1.35 1.53 1.25
   KEG 0.57 0.60 0.58 0.57 0.56 0.57 0.52 0.52 0.55 0.56 0.55 0.56 0.57 0.50 0.47 0.60 0.56 0.54 1.21
   RAT 1.27 1.11 1.21 1.12 1.00 1.27 1.16 1.11 1.18 1.22 1.21 0.85 1.12 0.92 0.81 1.24 1.06 1.02 3.54
   GAL 4.23 4.24 4.13 4.05 3.98 4.23 4.39 4.33 4.40 4.36 4.35 4.68 4.05 3.88 3.69 4.19 4.05 3.92 5.53
   HAM 1.15 1.40 1.29 1.15 1.54 1.15 1.39 1.20 1.18 1.18 1.23 1.14 1.15 0.97 0.87 1.20 1.14 1.02 1.25
   MAT 1.69 1.93 1.68 1.67 1.90 1.69 1.86 1.70 1.73 1.75 1.72 1.51 1.67 1.50 1.49 1.74 1.66 1.60 2.65
   BAD 1.29 1.43 1.28 1.29 1.40 1.29 1.35 1.27 1.28 1.27 1.26 1.52 1.29 1.21 1.23 1.32 1.28 1.22 1.89
   MON 0.46 0.82 0.41 0.47 0.46 0.46 0.86 1.09 0.54 0.54 0.75 0.62 0.47 0.61 1.37 0.43 0.47 0.68 0.72
   COL 1.25 1.40 1.27 1.27 1.57 1.25 1.20 1.24 1.24 1.24 1.25 1.25 1.27 1.21 1.25 1.27 1.26 1.27 2.29
   GAM 1.27 1.30 1.24 1.29 1.39 1.27 1.23 1.27 1.25 1.27 1.27 1.12 1.29 1.18 1.26 1.26 1.27 1.28 0.92
   KLU 1.85 2.21 1.79 1.80 2.32 1.85 1.75 1.81 1.83 1.85 1.84 1.69 1.80 1.54 1.72 1.84 1.79 1.78 2.71
Table 10: MASE from training set: 2006-2018 for weekly data
Base Forecasting HTS based on ARIMA HTS based on ETS
series arm snv nve ets avg bup top ols mit var stc cov ebup etop eols emit evar estc ecov
SL 1.34 1.45 1.35 1.29 1.60 1.39 1.34 1.34 1.33 1.36 1.35 1.66 1.30 1.29 1.28 1.31 1.29 1.28 1.80
Province
   CN 1.87 1.83 1.82 1.68 1.88 1.87 1.88 1.87 1.91 1.87 1.87 2.45 1.80 1.71 1.74 1.76 1.72 1.73 2.19
   ET 1.60 2.48 1.55 1.55 1.31 2.02 1.55 1.65 1.85 1.82 1.81 2.35 1.53 1.34 1.33 1.51 1.54 1.44 1.32
   NC 0.84 1.03 0.72 0.76 0.76 0.71 0.85 0.91 0.74 0.74 0.76 0.85 0.74 0.58 0.51 0.61 0.64 0.53 1.34
   NW 0.58 0.84 0.64 0.75 0.63 0.65 0.58 0.60 0.58 0.62 0.61 0.58 0.72 0.70 0.70 0.74 0.73 0.72 1.20
   NT 4.49 3.03 5.18 4.15 3.69 3.50 4.46 4.31 3.86 3.89 4.00 3.83 5.16 3.77 4.02 4.53 4.59 4.44 4.10
   SB 0.64 0.81 0.74 0.71 0.73 0.67 0.65 0.66 0.65 0.66 0.65 1.06 0.75 0.70 0.72 0.78 0.72 0.72 1.25
   ST 2.69 2.82 2.55 2.58 2.76 2.40 2.73 2.65 2.56 2.48 2.54 2.67 2.54 2.70 2.76 2.56 2.56 2.63 2.76
   UVA 0.72 0.82 0.69 0.72 0.71 0.65 0.73 0.72 0.68 0.68 0.67 0.88 0.71 0.76 1.00 0.70 0.71 0.75 1.16
   WT 1.42 1.50 1.42 1.45 1.67 1.43 1.43 1.42 1.38 1.42 1.42 2.16 1.43 1.46 1.44 1.47 1.44 1.44 1.94
Districts
   KN 1.89 1.81 1.83 1.83 1.93 1.89 1.90 1.89 1.97 1.89 1.88 2.67 1.83 1.71 1.77 1.79 1.73 1.79 2.21
   MTL 1.91 2.24 1.87 1.87 1.87 1.91 1.93 1.94 1.92 1.91 1.92 2.03 1.87 1.96 2.19 1.92 1.88 2.07 2.30
   NE 0.98 1.05 1.09 1.02 0.98 0.98 0.99 1.00 0.99 0.98 0.96 1.13 1.02 1.17 2.31 1.07 1.03 1.82 1.45
   AM 1.05 1.09 0.87 0.82 0.98 1.05 1.19 2.90 1.26 1.09 2.08 1.47 0.82 0.86 1.47 0.77 0.82 1.06 1.13
   BAT 2.28 3.38 1.38 1.38 1.19 2.28 1.66 2.04 2.16 2.16 2.14 2.58 1.38 1.19 1.25 1.40 1.39 1.33 1.49
   KAL 1.33 1.44 0.84 0.81 0.73 1.33 1.00 1.06 1.15 1.04 1.18 2.07 0.81 0.71 0.67 0.73 0.81 0.74 1.02
   TRC 2.13 2.10 2.25 2.25 1.81 2.13 1.88 1.89 1.95 1.96 1.96 2.12 2.25 2.04 2.08 2.22 2.25 2.17 1.83
   APU 0.71 1.13 0.72 0.72 0.75 0.71 0.77 0.77 0.68 0.70 0.71 1.04 0.72 0.63 0.53 0.66 0.67 0.59 1.33
   POL 0.82 0.78 0.66 0.74 0.75 0.82 1.03 1.36 0.95 0.87 1.05 0.56 0.74 0.45 0.50 0.63 0.64 0.42 1.30
   KUR 0.50 0.67 0.48 0.59 0.48 0.50 0.53 0.54 0.57 0.51 0.51 0.45 0.59 0.65 0.65 0.62 0.64 0.64 1.16
   PUT 1.05 1.10 0.92 0.94 0.85 1.05 0.92 0.98 0.96 1.02 0.99 1.00 0.94 0.86 0.88 0.96 0.95 0.94 1.21
   JAF 4.44 3.65 6.90 6.90 4.75 4.44 5.60 4.68 4.86 4.88 4.59 5.14 6.90 4.97 6.52 6.17 6.25 6.68 5.65
   KIL 3.94 4.32 3.95 3.94 4.01 3.94 5.89 9.65 3.98 4.02 7.46 4.34 3.94 3.25 9.09 3.92 3.88 5.60 4.63
   MAN 0.73 1.13 0.84 0.76 0.73 0.73 0.96 2.40 0.87 0.86 1.68 0.76 0.76 0.62 2.32 0.63 0.63 1.41 0.76
   MUL 1.91 2.04 1.65 1.65 1.65 1.91 2.81 8.30 1.92 1.96 5.30 2.53 1.65 1.55 8.50 1.63 1.63 5.32 1.77
   VAV 1.15 1.49 1.29 1.29 1.20 1.15 1.35 1.83 1.30 1.36 1.49 1.87 1.29 1.10 1.60 1.09 1.07 1.15 2.35
   KEG 0.54 0.58 0.56 0.38 0.55 0.54 0.53 0.53 0.51 0.53 0.52 0.95 0.38 0.52 0.53 0.46 0.45 0.49 1.18
   RAT 0.78 1.04 0.95 1.36 1.00 0.78 0.75 0.76 0.77 0.76 0.76 1.28 1.36 1.08 1.33 1.34 1.33 1.34 1.60
   GAL 4.15 4.58 4.34 4.46 4.27 4.15 4.48 4.47 4.36 4.24 4.34 4.21 4.46 4.57 4.67 4.48 4.48 4.55 4.56
   HAM 1.16 1.46 1.10 1.07 1.55 1.16 1.48 1.52 1.22 1.18 1.35 1.48 1.07 0.92 0.98 1.09 1.05 0.98 1.65
   MAT 1.62 1.93 1.74 1.87 1.90 1.62 1.79 1.70 1.69 1.68 1.67 1.79 1.87 1.93 1.98 1.81 1.86 1.89 2.09
   BAD 1.27 1.50 1.42 1.46 1.45 1.27 1.41 1.34 1.33 1.32 1.31 1.46 1.46 1.52 1.72 1.41 1.47 1.50 2.06
   MON 0.45 0.80 0.51 0.46 0.44 0.45 0.36 0.30 0.41 0.43 0.37 1.05 0.46 0.42 0.45 0.41 0.45 0.41 0.54
   COL 1.36 1.42 1.37 1.41 1.60 1.36 1.37 1.36 1.32 1.36 1.36 2.06 1.41 1.61 1.50 1.47 1.44 1.47 1.81
   GAM 1.22 1.24 1.22 1.23 1.37 1.22 1.23 1.22 1.15 1.22 1.22 2.15 1.23 1.22 1.23 1.26 1.23 1.23 1.80
   KLU 2.09 2.28 1.93 2.06 2.40 2.09 2.05 1.99 2.12 2.08 2.00 2.21 2.06 2.04 1.94 2.00 2.01 1.95 2.41
Table 11: MASE from training set: 2006-2018for monthly data
Base Forecasting HTS based on ARIMA HTS based on ETS
series arm snv nve ets avg bup top ols mit var stc ebup etop eols emit evar estc
SL 1.46 1.52 1.46 1.43 1.73 1.46 1.46 1.46 1.41 1.45 1.46 1.55 1.43 1.43 1.41 1.50 1.46
Province
   CN 1.93 1.90 1.93 1.85 1.93 2.26 1.94 1.96 2.06 2.07 2.02 1.85 1.87 1.85 1.80 1.85 1.84
   ET 1.97 2.72 1.42 1.72 1.42 1.88 1.92 1.93 1.94 1.91 1.93 2.40 1.69 1.75 2.24 2.18 1.96
   NC 0.79 0.90 0.80 0.68 0.78 0.71 0.82 0.88 0.70 0.74 0.76 0.66 0.71 0.84 0.75 0.67 0.63
   NW 0.58 0.84 0.68 0.65 0.68 0.64 0.59 0.61 0.60 0.61 0.61 0.75 0.64 0.63 0.68 0.71 0.67
   NT 4.21 3.38 4.44 3.07 4.15 4.01 4.14 4.14 4.12 4.10 4.11 3.83 3.09 3.02 3.65 3.66 3.26
   SB 0.76 0.72 0.83 0.79 0.76 0.62 0.80 0.68 0.57 0.64 0.65 0.70 0.82 0.76 0.62 0.67 0.70
   ST 2.73 3.00 3.00 2.82 2.96 2.90 2.79 2.83 2.82 2.85 2.82 2.64 2.90 2.95 2.77 2.70 2.81
   UVA 0.60 0.79 0.74 0.59 0.69 0.58 0.63 0.66 0.55 0.58 0.59 0.76 0.64 0.94 0.80 0.69 0.76
   WT 1.54 1.53 1.54 1.46 1.78 1.58 1.59 1.56 1.53 1.57 1.56 1.64 1.49 1.50 1.53 1.58 1.54
Districts
   KN 2.50 1.84 1.93 1.86 1.99 2.50 2.22 2.36 2.26 2.26 2.40 1.86 1.89 1.86 1.78 1.86 1.86
   MTL 1.99 2.28 2.07 1.93 2.00 1.99 2.36 2.22 1.95 1.99 2.13 1.93 1.97 1.99 2.05 1.93 1.94
   NE 1.03 1.02 1.18 0.93 1.09 1.03 1.82 2.09 0.98 1.03 1.76 0.93 0.96 1.61 0.98 0.93 1.18
   AM 1.06 0.76 0.85 0.85 1.03 1.06 0.95 0.58 0.98 1.05 0.55 0.85 1.06 3.56 0.93 0.85 2.53
   BAT 1.94 3.80 1.39 3.13 1.33 1.94 2.01 1.98 2.01 1.97 1.97 3.13 1.93 2.72 3.04 2.97 2.86
   KAL 1.26 1.58 0.77 1.28 0.77 1.26 1.29 1.30 1.33 1.30 1.30 1.28 0.91 0.81 0.99 1.05 0.97
   TRC 2.07 2.04 1.88 2.19 1.96 2.07 2.10 2.12 2.10 2.10 2.11 2.19 1.92 2.03 2.12 1.96 1.89
   APU 0.80 1.08 0.79 0.69 0.79 0.80 0.89 0.93 0.79 0.84 0.84 0.69 0.69 0.70 0.84 0.69 0.61
   POL 0.48 0.72 0.79 0.65 0.72 0.48 0.62 0.72 0.49 0.49 0.55 0.65 0.69 1.13 0.60 0.65 0.70
   KUR 0.58 0.64 0.52 0.55 0.52 0.58 0.63 0.62 0.53 0.58 0.58 0.55 0.51 0.51 0.54 0.52 0.51
   PUT 1.09 1.13 1.01 1.10 0.91 1.09 1.01 1.05 1.07 1.07 1.05 1.10 0.95 0.94 1.01 1.06 1.00
   JAF 5.56 4.29 6.21 5.18 5.75 5.56 5.73 5.60 5.69 5.67 5.59 5.18 4.20 4.85 4.89 4.92 4.96
   KIL 4.47 5.42 4.19 4.49 4.91 4.47 4.68 5.30 4.46 4.48 5.10 4.49 3.17 6.58 4.40 4.47 4.77
   MAN 0.82 1.21 0.75 0.75 0.75 0.82 0.83 0.86 0.83 0.82 0.82 0.75 0.58 1.39 0.68 0.71 0.98
   MUL 2.04 2.28 1.88 1.88 1.88 2.04 2.12 2.72 2.03 2.05 2.56 1.88 1.47 4.97 2.04 1.86 3.48
   VAV 1.31 1.57 1.30 1.44 1.30 1.31 1.31 1.30 1.31 1.30 1.30 1.44 1.06 1.06 1.36 1.31 0.96
   KEG 0.63 0.57 0.59 0.63 0.59 0.63 0.77 0.79 0.58 0.66 0.71 0.63 0.69 0.72 0.65 0.67 0.69
   RAT 0.82 0.89 1.20 1.13 1.12 0.82 1.00 0.88 0.79 0.84 0.85 1.13 0.96 0.94 0.83 1.03 0.98
   GAL 4.67 5.08 5.08 4.79 4.75 4.67 4.55 4.61 4.64 4.63 4.60 4.79 4.96 5.08 4.88 4.84 4.96
   HAM 1.52 1.58 1.29 1.67 1.68 1.52 1.40 1.42 1.46 1.50 1.40 1.67 1.36 1.57 1.46 1.64 1.55
   MAT 2.05 2.06 2.09 1.91 2.03 2.05 1.96 2.00 1.92 1.98 1.99 1.91 2.08 2.14 2.04 1.99 2.04
   BAD 1.37 1.58 1.51 1.55 1.49 1.37 1.50 1.45 1.35 1.41 1.41 1.55 1.43 1.70 1.58 1.47 1.55
   MON 0.50 0.80 0.28 0.28 0.31 0.50 0.45 0.40 0.50 0.49 0.45 0.28 0.38 0.29 0.42 0.28 0.31
   COL 1.51 1.51 1.51 1.61 1.73 1.51 1.57 1.53 1.48 1.52 1.52 1.61 1.44 1.51 1.48 1.56 1.54
   GAM 1.50 1.28 1.32 1.48 1.45 1.50 1.49 1.46 1.39 1.48 1.47 1.48 1.34 1.37 1.39 1.41 1.40
   KLU 2.18 2.51 2.36 2.28 2.64 2.18 2.20 2.14 2.14 2.18 2.15 2.28 2.45 2.65 2.29 2.31 2.52
Table 12: MASE from training set: 2006-2018 for quarterly data
Base Forecasting HTS based on ARIMA HTS based on ETS
series arm nve ets avg bup top ols mit var stc ebup etop eols emit evar estc
SL 2.15 1.55 1.24 2.15 1.31 2.15 2.12 1.51 1.55 1.84 1.48 1.24 1.27 1.44 1.44 1.38
Province
   CN 2.32 1.89 2.37 2.32 2.32 2.38 2.53 2.29 2.34 2.46 2.30 2.29 2.21 2.32 2.32 2.27
   ET 1.68 0.73 0.77 1.68 0.41 1.75 1.78 0.65 0.71 1.34 0.85 1.07 1.02 0.84 0.84 0.94
   NC 0.83 0.36 0.42 0.83 0.83 0.89 1.67 0.79 0.83 1.25 0.39 0.30 0.15 0.40 0.40 0.22
   NW 0.51 0.17 0.41 0.51 0.51 0.57 0.70 0.49 0.53 0.61 0.01 0.33 0.15 0.11 0.11 0.16
   NT 3.79 3.03 3.64 5.45 3.37 3.99 4.33 3.56 3.52 4.21 3.13 3.37 3.15 3.34 3.34 3.12
   SB 0.87 0.73 0.86 0.87 0.87 0.92 1.05 0.85 0.88 0.96 0.86 0.78 0.73 0.86 0.85 0.82
   ST 3.52 3.51 2.63 3.52 3.52 3.58 3.79 3.49 3.53 3.71 3.34 2.48 2.63 3.12 3.12 2.91
   UVA 0.79 0.56 0.47 0.79 0.79 0.85 1.30 0.76 0.80 1.05 0.07 0.37 0.00 0.16 0.16 0.16
   WT 2.20 1.82 1.35 2.20 0.65 2.27 1.87 1.10 1.13 1.47 1.63 1.18 1.38 1.52 1.51 1.47
Districts
   KN 2.42 1.92 2.43 2.42 2.42 2.48 2.52 2.40 2.45 2.49 2.43 2.42 2.39 2.45 2.45 2.41
   MTL 2.10 1.81 2.10 2.10 2.10 2.16 2.44 2.08 2.11 2.33 2.10 2.09 1.96 2.11 2.10 2.06
   NE 1.12 1.30 0.68 1.12 1.12 1.20 2.60 1.08 1.12 2.13 0.68 0.66 0.04 0.69 0.68 0.49
   AM 1.20 0.61 1.21 1.20 1.20 1.65 5.56 1.20 1.24 4.18 1.21 1.16 0.67 1.21 1.21 0.93
   BAT 2.19 3.32 3.19 1.75 2.19 0.85 1.13 1.84 1.83 1.46 3.19 3.57 3.32 3.18 3.18 3.26
   KAL 0.76 0.75 1.01 0.76 0.76 1.28 1.81 0.97 1.07 1.48 1.01 1.16 1.14 0.99 0.99 1.07
   TRC 2.23 1.55 1.20 2.23 2.23 2.69 3.22 2.41 2.47 2.90 1.20 1.09 1.08 1.21 1.21 1.14
   APU 0.82 0.17 0.64 0.82 0.82 0.88 1.45 0.78 0.82 1.14 0.64 0.56 0.23 0.65 0.65 0.51
   POL 0.82 0.70 0.09 0.82 0.82 0.88 2.01 0.78 0.82 1.42 0.09 0.21 0.86 0.08 0.08 0.33
   KUR 0.38 0.22 0.66 0.38 0.38 0.43 0.53 0.36 0.40 0.45 0.66 0.20 0.53 0.55 0.55 0.52
   PUT 0.75 0.09 1.09 0.75 0.75 0.80 1.00