Supply chain management (SCM) represents the managerial backbone of the logistics and production sector. Due to its relevance, new methodologies appear regularly in the literature. One of them is Build-To-Order Supply Chain Management (BTO-SCM). This technique has seen significant adoption. Motivated by the lack of work in the demand forecasting literature on BTO-SCM problems, this research develops and presents Diagonal Feeding, a data transformation technique, specially tailored for this setting together with a relevant and novel data set from an electronics manufacturer. Data set and implementations of all methods are available for download 111https://github.com/rodrigorivera/isnn19.
An accurate demand forecasting is essential for the global economy with stockpiled or in-transit inventories representing 17% of the world’s Gross Domestic Product (GDP), . Yet, imprecise demand planning is still pervasive with forecast errors of up to 44-53% for new products, , . As a result, retailers experience out-of-stock (OOS) events with rates amounting to 8.3% worldwide, . BTO-SCM helps address the uncertainty that arises from variability in demand.
The aims and backgrounds of this research is to present a technique for data pre-processing accessible to non-technical business experts. Traditional forecasting methods are still being used primarily by over 40% of demand planners in the industry, 
, and the use of novel machine learning methods is a promising area with little academic research and insufficient efforts to expose practitioners to them, .
The significance of developing demand forecasting methods that can be easily adopted by demand planners is evident. For discrepancies as low as 2%, it is worth improving the accuracy of a forecast, , and a 10% reduction in OOS can increase retailers’ revenue by up to 0.5%, 
. Yet, companies struggle hiring the adequate personnel to address these tasks. For example by 2020, Vietnam is expected to face a shortage of over 500,000 employees with data science and analytic skills and over 80% of the local workforce is unsuited to fill this gap,. In Europe, over 70% of surveyed businesses struggle hiring data science personnel and over 60% are resorting to internal training to upgrade the skills of existing business analysts, . This work seeks to alleviate this situation by presenting a feature engineering technique well-suited for demand forecasting in manufacturing that is both accurate as well as easy to communicate to decision-makers.
The research goal is to propose a method for time series forecast of BTO products that can be adopted by practitioners. For this purpose, the study poses the questions: 1) What is the state of the art in academic research in demand forecasting for BTO products? 2) How does Diagonal Feeding support the BTO supply chain? To achieve the research goal, two objectives have been assigned: a) To review the existing theory on time series prediction, specially on demand forecasting; b) To make a performance comparison of the proposed technique. The object of research is the balance between accessibility and precision of methods for time series in an industry setting. The subject of the research is forecasting product demand for BTO-SCM.
2 Literature Review
BTO-SCM can be defined according to  as ”the system that produces goods and services based on individual customer requirements in a timely and cost competitive manner by leveraging global outsourcing, the application of information technology and through the standardization of components and delayed product differentiation strategies”. Proponents of BTO supply chains such as  argue that they promote sales, reduce costs and increase customer satisfaction. For example,  mentions that 74% of US car buyers would purchase a customized vehicle if the delivery time is less than three weeks. As an example of costs reduction, it is claimed that Nissan, a car manufacturer, could save up to 3600 USD per vehicle, if they were to transition their supply chain completely to BTO. In the technology industry, Dell, a computer maker, has generated up to a 160% return on its invested capital by implementing a BTO supply chain for its e-commerce website, .
Demand forecasting has commanded attention from different communities due to its importance in the supply chain management, . A comprehensive treatment can be found in the works of  and . Formally, it can be stated as predicting future values , given a time series , where is the length of the window on the time series and is called the prediction horizon, . To obtain these predictions, quantitative methods such as ARIMA, exponential smoothing models and alike are often used. Yet,  argues that there have been few large scale comparison studies of machine learning models for regression or time series aimed at forecasting problems. For the electronics manufacturing industry,  introduced SVM regression to the supply chain of various producers. They concluded that it yields good results compared to other forecasting methods. Although SVM regression is a popular method for forecasting, not everyone has identified it as the most effective method. For example, 
used multivariate adaptive regression splines (MARS) to construct sales forecasting models for computer wholesalers. Similarly, proposed a Bayesian model for demand forecasting of computer parts and compared it against exponential smoothing and a judgment-based method.
The data consists of three data sets of observations from an electronics manufacturer representing a subset of their total inventory. Each of them contains the demand of 854 different items, totaling 2562 items, with up to 45 periods of data. The items can be requested for delivery either for the same month or up to three months later. In addition, two hierarchical dependencies are provided in the form of categories and parent-child relationships. These items have varying amounts of required quantities, with many of them being requested only sporadically, as seen in Figure 1, and few of them being requested in large quantities. Noticeable, a significant percentage of products lacks a continuous demand. There are no periods in all three data sets where all 854 products are requested. The demand for items both in variety as well as quantity is significantly higher for the delivery date 0 (same month). From Figure 2, it can be appreciated that quantities for all periods are consistently higher on delivery date 0 versus others. In the case of delivery date 3, the requested quantities for all periods are significantly smaller. Similarly, the amounts required in the third data set are much higher and they peak out, whereas the first and second data sets have an increasing demand pattern.
4 Diagonal Feeding
This work introduces the practitioner to Diagonal Feeding, a data transformation technique well-suited for multi-step structured forecasting from anticipatory data. As its main benefit, it levers the anticipatory nature of pre-orders’ time series data and makes forecasting the pre-order structure more streamlined. As a result, the accuracy of a regression model is improved. This is made possible due to the data set containing information not only about the current demand, but also on the volumes of pre-orders made in advance. Advance pre-orders are expectation-driven, naturally forward-looking and known beforehand, as they reflect planning and some anticipation of the market at the end of the period, when the order is to be fulfilled. This information is leveraged for predicting the pre-order structure in the future by also taking into account the cross-correlations between the pre-orders.
Let be the total quantity of some item requested by the end of period through period advance pre-orders. Those made from the end of until the end of . The key property of the data set is that for every item the value is effectively known and available for use at period , i.e. well before . For example, is known at the end of and corresponds to the quantity requested at the end of accumulated via pre-orders made by the end of . In the definition of the data set structure the index corresponds to the “period” field, whereas is the “delivery date” and is the value in the “quantity” field. The “item_code” key is intentionally omitted in order to simplify the explanation of the key aspects of the Diagonal Feeding representation. Prior discussion implies that the -period advance pre-order to be fulfilled at the end of period seems to reflect certain beliefs about the market environment in period . Therefore, the time series of the volume of period- pre-orders made periods in advance can be considered to be anticipatory in relation to the time series . For example, is known with certainty at , represents the total demanded quantity by the end of accumulated between the end of and the end of , and effectively reflects information periods ahead forward-looking on . In order to be able to utilize the anticipatory nature of time series with different “delivery dates” and at the same time to be able to forecast the pre-order structure, the data set in the Diagonal Feeding (-frontier) representation is represented as following:
In Equation 1, the target is the output. Its final representation is the lower diagonal . Thus, represents the pre-order structure for the next periods beginning with . The objective is to predict the lower diagonal of the matrix by triangularly reshaping the multivariate time series for each “item” and introducing some redundancy that localizes relevant anticipatory features. Since the quantity is known at time , each diagonal in the scheme above is known at , . This representation allows the value (the volume of period- pre-orders made periods in advance) to be used for forecasting the value , whenever
, i.e. the moment, when is revealed, is strictly earlier than the moment , when becomes known. Therefore, the aim is to predict , the pre-order structure for the next periods from , based on the preorders and their history, known by , where cat
stands for concatenation of vectors and; further, is empty. The values in the target that seem to be most relevant for practical demand forecasting are on its main diagonal with , since they are the earliest future volumes.
The demand volume data set contains time series for each items with many of them having sparse volume records due absent periods. They represent zero-volume orders. Thus, the multivariate time series for each item and “delivery date” were rebuilt from the data set by explicitly setting the volume to zero at absent periods. To test the viability of the proposed representation, only items with at least periods with actual non-zero recorded volume were kept. After filling the missing periods on the time series of each item respectively with zeroes, they were transformed into the diagonal representation and then pooled into one sample . The pre-processed data set was split into development and holdout set, with the latter containing the last periods available for each item ( depending on the particular item).
The Spearman Rank correlation 222http://bit.ly/2zLvNNf
was computed on the development sub-sample to estimate the “informativeness”; it is invariant under monotonic transformations of the data and thus insensitive to scale. It was applied to the total demand data setfor the selected subset of the items. A table containing the triangular grouping of the values is obtained, which the Diagonal feeding is built upon. Each value (left-right, top-bottom) in the table is the rank correlation between and computed across all periods and items. For instance, it shows that the most significant correlation () is between with computed over pooled sample of different items and periods. Since the values along the “delivery date” axis of the data set are in fact cumulative (gross), the correlation analysis is applied to the first differences . Making this difference allows for measuring the pure anticipatory information content of in the target in the Diagonal Feeding representation of . The correlations of this differenced data (net monthly orders) confirms that much of the observed correlation was due to the accumulated (integrated) nature of the data. The dramatic drop in the correlation between and shows that the orders of the net current periods are poorly predictable by the -period ahead net pre-orders meaning that they are likely to have different drivers. Nevertheless, the relatively small change in values within the sample Spearman rank correlation on the data and its counterpart on the data demonstrates that the period ahead net pre-orders and lagged net current period orders have generally a little more predictive content for the future pre-order structure than the pre-orders made more than period in advance.
4.1 Experimental results
The Diagonal Feeding transforms the demand forecasting problem from a time series analysis setting, where the goal is to model simultaneously for every , to a supervised multi-output regression problem, where it seeks to learn . Extensive numerical experiments were conducted using various common regression models involving cross-validated grid search over the hyper-parameters. Besides the described pre-process, no further data-engineering was carried out. The best estimator in each model class was picked via -fold cross-validation on the development set (the first observed periods of each selected item). The final test scores were computed on the held out data set using by averaging SMAPE across periods and items:
where is the forecast integer volume of the -th element in the advance preorder structure of item in the forecast starting with the future period . An assessment of the SMAPE scores of the best models across all the targets in the diagonal representation for reveals that the best model is a collection of
independent large ensembles of the gradient boosted regression trees withestimators in each ensemble. The next best model with generally lower scores is the
-layer dense ReLu network withhidden units each. A histograms of the computed for each in the structure of the gradient boosted ensemble will depict that most accurate predictions are for the next periods’s gross order volume (). However, the best prediction accuracy drops dramatically when the full grid search experiment repeated for the differenced data (). An evaluation of the holdout SMAPE scores for the best models on the reveals that even the best model, -nearest neighbor regression, completely fails to predict the net volume for the next period.
To validate Diagonal Feeding, this study carried out an assessment of various methods for demand forecasting. In total, thirteen different methods were assessed. For conciseness, this work focuses on the third data set seen in Figure 1
and on its delivery date 0 (same period). The evaluated methods are 1) Adaboost, 2) ARIMAX, 3) ARIMA, 4) Bayesian Structural Time Series (BSTS), 5) Bayesian Structural Time Series with a Bayesian Classifier (BSTS Classifier), 6) Ensemble of Gradient Boosting (Ensemble), 7) Ridge regression (Ridge), 8) Kernel regression (Kernel), 9) Lasso, 10) Neural Network (NN), 11) Poisson regression (Poisson), 12) Random Forest (RF), 13) Support Vector Regression (SVR).
Each of them had as a target value three different options: a) Quantity (non-transformed), b) Log-transformed quantity, c) Min-Max transformed quantity. Additionally, Diagonal Feeding, presented in section 4, was evaluated for regression methods. Thus, three settings were considered: a) No Diagonal Feeding, b) Diagonal Feeding with an item by item training (One by One). In this case, a vector containing the input of a specific item is fed individually to a model, c) Diagonal Feeding fitting the model on the full data set (All Items). Here, a matrix with the input from all items is used. In all three cases, an individual vector corresponding to a given item is obtained as an output. For a), extensive feature engineering was conducted and 360 features were generated. The specific features are documented in the code base provided. The training set consisted of 37 periods and the test set of 8. To evaluate the performance of the models, the Symmetric Mean Absolute Percent Error (SMAPE) is used. It is defined as with being the forecasted value and the actual value at time respectively. The results reveal that the median SMAPE for all methods is 0,42 and for methods with Diagonal Feeding is 0,43. The best Top 10 models had a median SMAPE of 0,31 and the Top 10 for those using Diagonal Feeding exclusively was 0,37.
The key insight from the analysis of the data set is that the next period’s gross total demand volume is mostly determined by the currently known one-period ahead pre-orders for the period (). The correlation analysis and the results of the grid search experiment confirm the observation that the net next period’s volume is the difference between and . Viewed through Diagonal Feeding, it is mostly independent of the history of net pre-orders for the period and is thus less predictable from advance pre-order data, as indicated by a correlation analysis and results from a grid search experiment. The success of forecasting the , especially in contrast to the other next period’s pre-order volumes for , might be attributed to the observed high correlation of the one-period ahead pre-order volume . Further, Diagonal Feeding delivers results comparable to those obtained doing extensive feature engineering; in this case, more than 300 features were generated. Along these lines, exploring different transformations of the target value is essential. For some models using log-transform or min-max delivered good results. For example, a Neural Network without a transformed quantity fitted on the full data set had a SMAPE of 1,13, with a log transformation, it was 0,38.
The best model was Adaboost with an SMAPE of 0,17. It was followed by the Ensemble of Random Forests with 0,18. Yet, it is important to remember that these models had extensive feature engineering with over 300 features generated. For Diagonal Feeding, the best method was a random forest with log transform and fitted on the full data set. It obtained 0,34. This was significantly better than the average of methods trained on the data set with feature engineering, which obtained a significantly higher SMAPE of 0,42.
This work introduced Diagonal Feeding, a technique specially useful for forecasting Build-To-Order products. It helps improving accuracy whenever future delivery dates are known. This approach does not require domain knowledge, extensive feature engineering or advanced technical skills. The results from multiple experiments show that there is no go-to technique for time series prediction. In addition, this research made available a highly-relevant and novel data set. A challenge in developing methods for demand forecasting of BTO products is the lack of public data sets. As a further line of work, it is worth exploring the impact on Diagonal Feeding of transforming the target variable, as it was shown, certain transformations perform better than others. From an algorithmic point of view the approach can be strengthened with non-parametric pre-processing techniques to filter out anomalies such as in , , , 
, including multichannel anomaly detection,, performing online aggregation of different forecasting models via long-term aggregation strategies, , along with approaches to model quasi-periodic data, , and extraction of trends in the presence of non-stationary noise with long tails, , . Given the potential of BTO supply chains, it is expected that an increasing number of researchers will direct their attention to this area and add additional entries to the literature.
The research in section 2 was partially supported by the Russian Foundation for Basic Research grant 16-29-09649 ofi m. The research presented in other sections was supported by the Mexican National Council for Science and Technology (CONACYT), 2018-000009-01EXTF-00154.
-  Agell, N., Carricano, M.: Adopcion e impacto del Big Data y Advanced Analytics en España. ESADE Business and Law School (May 2018)
-  Ahmed, N.K., Atiya, A.F., El Gayar, N., El-Shishiny, H.: An empirical comparison of machine learning models for time series forecasting. Econometric Reviews 29(5) (2010)
-  Artemov, A., Burnaev, E.: Ensembles of detectors for online detection of transient changes. In: Proc. SPIE. vol. 9875 (2015)
-  Artemov, A., Burnaev, E.: Detecting performance degradation of software-intensive systems in the presence of trends and long-range dependence. In: IEEE 16th ICDMW. pp. 29–36 (2016)
Artemov, A., Burnaev, E.: Optimal estimation of a signal perturbed by a fractional brownian noise. Theory of Probability & Its Applications (2016)
-  Artemov, A., Burnaev, E., Lokot, A.: Nonparametric decomposition of quasi-periodic time series for change-point detection. In: Proc. SPIE. vol. 9875 (2015)
-  Attar, K.S.: Regression for Demand Forecasting 5(1) (2016)
-  Bogataj, D., Bogataj, M.: Npv approach to material requirements planning theory - a 50-year review of these research achievements. International Journal of Production Research (2018)
-  Burnaev, E.V., Golubev, G.K.: On one problem in multichannel signal detection. Problems of Information Transmission 53(4), 368–380 (Oct 2017)
-  Chase, C.W.: Demand-Driven Forecasting. John Wiley & Sons, Inc., Hoboken, NJ, USA (aug 2013)
-  Christensen, W.J., Germain, R.: Build-to-order and just-in-time as predictors of applied supply chain knowledge and market performance. Journal of Operations Management 23(5) (nov 2004)
-  Fleisch, E., Tellkamp, C.: Inventory inaccuracy and supply chain performance: a simulation study of a retail supply chain. International Journal of Production Economics 95(3), 373–385 (mar 2005)
-  Gilliand, M.: Business Forecasting. John Wiley & Sons, Inc., Hoboken, NJ, USA (dec 2015)
-  Gruen, T.W., Corsten, D.S., Bharadwaj, S.: Retail out of stocks: A worldwide examination of extent, causes, and consumer responses (2002)
-  Gunasekaran, A., Ngai, E.: Build-to-order supply chain management: A literature review and framework for development. Journal of Operations Management 23, 423–451 (07 2005)
-  Ishimtsev, V., Bernstein, A., Burnaev, E., Nazarov, I.: Conformal k-nn anomaly detector for univariate data streams. Proceedings of Machine Learning Research, vol. 60, pp. 213–227. PMLR, Stockholm, Sweden (13–16 Jun 2017)
-  Jain, C.: Benchmarking new product forecasting. The Journal of Business Forecasting (2005)
-  Kahn, K.B.: An exploratory investigation of new product forecasting practices (2002)
-  Kaul, R.: Retail out-of-stock management: An outcome-based approach (2013)
-  Korotin, A., V’yugin, V., Burnaev, E.: Aggregating strategies for long-term forecasting. Proceedings of Machine Learning Research, vol. 91, pp. 63–82. PMLR (11–13 Jun 2018)
-  Lu, C.J., Lee, T.S.: Sales forecasting for computer wholesalers: A comparison of multivariate adaptive regression splines and artificial neural networks. Decision Support Systems (2012)
-  Manisha, G., Vijayalakshmi, M.: Inter Time Series Sales Forecasting. Jascse 2(1) (2013)
-  Pompa, C., Burke, T.: Data Science and Analytics Skills Shortage: Equipping the APEC Workforce with the Competencies Demanded by Employers. APEC Human Resource Development Working Group (2017)
-  Rivera, R., Burnaev, E.: Forecasting of commercial sales with large scale Gaussian Processes. Proc. IEEE 17th ICDMW pp. 625–634 (2017)
-  Rivera, R., Nazarov, I., Burnaev, E.: Towards forecast techniques for business analysts of large commercial data sets using matrix factorization methods. Journal of Physics: Conference Series 1117, 012010 (nov 2018)
-  Safin, A., Burnaev, E.: Conformal kernel expected similarity for anomaly detection in time-series data. Advances in Systems Science and Applications (2017)
-  Sharma, A., Laplaca, P.: Marketing in the emerging era of build-to-order manufacturing. Industrial Marketing Management 34, 476–486 (07 2005)
-  Swaminathan, J.M., Tayur, S.R.: Models for supply chains in e-business. Management Science 49(10), 1387–1406 (oct 2003)
-  Volkhonskiy, D., Burnaev, E., Nouretdinov, I., Gammerman, A., Vovk, V.: Inductive conformal martingales for change-point detection. Proceedings of Machine Learning Research, vol. 60, pp. 132–153. PMLR (2017)
Wan, X.l., Zhang, Z.: Exploring an Interactive Value-Adding Data-Driven Model of Consumer Electronics Supply Chain Based on Least Squares Support Vector Machine. Scientific Programming (2016)
-  Yelland, P.M., Kim, S., Stratulate, R.: A Bayesian Model for Sales Forecasting at Sun Microsystems. Interfaces 40(2), 118–129 (2010)