Skip to main content
Springer Nature Link
Log in

Intermittent demand forecasting with transformer neural networks

  • Original Research
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Intermittent demand forecasting is an important yet challenging task in many organizations. While prior research has been focused on traditional methods such as Croston’s method and its variants, limited research has been conducted using advanced machine learning or deep learning methods. In this study, we introduce Transformer, a recently developed deep learning approach, to forecast intermittent demand. Its effectiveness is empirically tested with a dataset of 925 intermittent demand items from an airline spare parts provider and compared with that of two traditional methods such as Croston’s and the Syntetos–Boylan approximation as well as several popular neural network architectures including feedforward neural networks, recurrent neural networks, and long short-term memory. Our results based on six different forecasting performance measures show that Transformer performs very well against other methods in a variety of settings. We also examine how data sparsity impacts model performance and find that different models perform similarly when sparsity is low. Although the performance of all models generally gets worse as the sparsity level increases, the advantage of Transformer over other models increases with sparsity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. https://d2l.ai/chapter_recurrent-modern/lstm.html.

  2. https://medium.com/mlearning-ai/transformer-implementation-for-time-series-forecasting-a9db2db5c820.

  3. https://www.medium.com/mlearning-ai/transformer-implementation-for-time-series-forecasting-a9db2db5c820

  4. https://www.medium.com/mlearning-ai/transformer-implementation-for-time-series-forecasting-a9db2db5c820.

References

  • Babai, M., Dallery, Y., Boubaker, S., & Kalai, R. (2019). A new method to forecast intermittent demand in the presence of inventory obsolescence. International Journal of Production Economics, 209, 30–41.

    Article  Google Scholar 

  • Babai, M. Z., Syntetos, A. A., & Teunter, R. (2014). Intermittent demand forecasting: An empirical study on accuracy and the risk of obsolescence. International Journal of Production Economics, 157, 212–219.

    Article  Google Scholar 

  • Babai, M. Z., Tsadiras, A., & Papadopoulos, C. (2020). On the empirical performance of some new neural network methods for forecasting intermittent demand. IMA Journal of Management Mathematics, 31, 281–305.

    Article  Google Scholar 

  • Bahdanau, D., Cho, K., and Bengio, Y., (2015). Neural machine translation by jointly learning to align and translate. In: Proceedings of international conference on learning representations

  • Boylan, J. E., & Babai, M. Z. (2016). On the performance of overlapping and non-overlapping temporal demand aggregation approaches. International Journal of Production Economics, 181, 136–144.

    Article  Google Scholar 

  • Boylan, J. E., & Syntetos, A. A. (2007). The accuracy of a modified Croston procedure. International Journal of Production Economics, 107, 511–517.

    Article  Google Scholar 

  • Boylan, J. E., & Syntetos, A. A. (2021). Intermittent demand forecasting: Context, methods, and applications. Wiley.

    Book  Google Scholar 

  • Croston, J. D. (1972). Forecasting and stock control for intermittent demand. Operational Research Quarterly, 23, 289–303.

    Article  Google Scholar 

  • Doszyń, M. (2019). Intermittent demand forecasting in the enterprise: Empirical verification. Journal of Forecasting, 38, 459–469.

    Article  Google Scholar 

  • Eaves, A. H. C., & Kingsman, B. G. (2004). Forecasting for the ordering and stockholding of spare parts. Journal of the Operational Research Society, 55(4), 431–437.

    Article  Google Scholar 

  • Gutierrez, R. S., Solis, A. O., & Mukhopadhyay, S. (2008). Lumpy demand forecasting using neural networks. International Journal of Production Economics, 111, 409–420.

    Article  Google Scholar 

  • Hasni, M., Aguir, M., Babai, M., & Jemai, Z. (2019). On the performance of adjusted bootstrapping methods for intermittent demand forecasting. International Journal of Production Economics, 216, 145–153.

    Article  Google Scholar 

  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.

    Article  Google Scholar 

  • Hu, Q., Boylan, J. E., Chen, H., & Labib, A. (2018). OR in spare parts management: A review. European Journal of Operational Research, 266, 395–414.

    Article  Google Scholar 

  • Hua, Z. S., Zhang, B., Yang, J., & Tan, D. S. (2007). A new approach of forecasting intermittent demand for spare parts inventories in the process industries. Journal of Operational Research Society, 58, 52–61.

    Article  Google Scholar 

  • Hua, Z., & Zhang, B. (2006). A hybrid support vector machines and logistic regression approach for forecasting intermittent demand of spare parts. Applied Mathematics and Computation, 181(2), 1035–1048.

    Article  Google Scholar 

  • Johnston, F. R., Boylan, J. E., & Shale, E. (2003). An examination of the size of orders from customers, their characterisation and the implications for inventory control of slow moving items. Journal of the Operational Research Society, 54, 833–837.

  • Kingma, D. P. and Ba, J. (2017). Adam: A method of stochastic optimization. arXiv:1412.6980.

  • Kourentzes, N. (2013). Intermittent demand forecasts with neural networks. International Journal of Production Economics, 143(1), 198–206.

    Article  Google Scholar 

  • Levén, E., & Segerstedt, A. (2004). Inventory control with a modified Croston procedure and Erlang distribution. International Journal of Production Economics, 90, 361–367.

    Article  Google Scholar 

  • Lolli, F., Gamberini, R., Regattieri, A., Balugani, E., Gatos, T., & Gucci, S. (2017). Single-hidden layer neural networks for forecasting intermittent demand. International Journal of Production Economics, 183, 116–128.

    Article  Google Scholar 

  • Mukhopadhyay, S., Solis, A. O., & Gutierrez, R. S. (2012). The accuracy of non-traditional versus traditional methods of forecasting lumpy demand. Journal of Forecasting, 31, 721–735.

    Article  Google Scholar 

  • Nasiri Pour, A., Rostami Tabar, B., & Rahimzadeh, A. (2008). A hybrid neural network and traditional approach for forecasting lumpy demand. Engineering and Technology, 40, 384–390.

    Google Scholar 

  • Nikolopoulos, K. (2021). We need to talk about intermittent demand forecasting. European Journal of Operational Research, 291, 549–559.

    Article  Google Scholar 

  • Nikolopoulos, K. I., Babai, M. Z., & Bozos, K. (2016). Forecasting supply chain sporadic demand with nearest neighbor approaches. International Journal of Production Economics, 177, 139–148.

    Article  Google Scholar 

  • Nikolopoulos, K., Syntetos, A. A., Boylan, J. E., Petropoulos, F., & Assimakopoulos, V. (2011). An aggregate–disaggregate intermittent demand approach (ADIDA) to forecasting: An empirical proposition and analysis. Journal of the Operational Research Society, 62(3), 544–554.

    Article  Google Scholar 

  • Niu, Z., Zhong, G., & Yu, H. (2021). A review on the attention mechanism of deep learning. Neurocomputing, 452, 48–62.

    Article  Google Scholar 

  • Petropoulos, F., Kourentzes, N., & Nikolopoulos, K. (2016). Another look at estimators for intermittent demand. International Journal of Production Economics, 181, 154–161.

    Article  Google Scholar 

  • Pince, C., Turrini, L., & Meissner, J. (2021). Intermittent demand forecasting for spare parts: A critical review. Omega, 105, 102513.

    Article  Google Scholar 

  • Prestwich, S. D., Tarim, S. A., Rossi, R., & Hnich, B. (2014). Forecasting intermittent demand by hyperbolic-exponential smoothing. International Journal of Forecasting, 30, 928–933.

    Article  Google Scholar 

  • Shale, E. A., Boylan, J. E., & Johnston, F. R. (2006). Forecasting for intermittent demand: The estimation of an unbiased average. Journal of Operational Research Society, 57, 588–592.

    Article  Google Scholar 

  • Syntetos, A. A., Babai, M. Z., & Gardner, E. S., Jr. (2015). Forecasting intermittent inventory demands: Simple parametric methods vs. bootstrapping. Journal of Business Research, 68, 1746–1752.

    Article  Google Scholar 

  • Syntetos, A. A., Babai, Z., Boylan, J. E., Kolassa, S., & Nikolopoulos, K. (2016). Supply chain forecasting: Theory, practice, their gap and the future. European Journal of Operational Research, 252, 1–26.

    Article  Google Scholar 

  • Syntetos, A. A., & Boylan, J. E. (2001). On the bias of intermittent demand estimates. International Journal of Production Economics, 71, 457–466.

    Article  Google Scholar 

  • Syntetos, A. A., & Boylan, J. E. (2005). The accuracy of intermittent demand estimates. International Journal of Forecasting, 21, 303–314.

    Article  Google Scholar 

  • Teunter, R., & Sani, B. (2009). On the bias of Croston’s forecasting method. European Journal of Operational Research, 194(1), 177–183.

    Article  Google Scholar 

  • Teunter, R. H., Syntetos, A. A., & Babai, M. Z. (2011). Intermittent demand: Linking forecasting to inventory obsolescence. European Journal of Operational Research, 214(3), 606–615.

    Article  Google Scholar 

  • Vaswani, A. Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, (2017). Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in neural information processing systems 30, pp. 5998–6008.

  • Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., and Sun, L., (2022). Transformers in time series: A survey. arXiv. arXiv:2202.07125.

  • Willemain, T. R., Smart, C. N., & Schwarz, H. F. (2004). A new approach to forecasting intermittent demand for service parts inventories. International Journal of Forecasting, 20, 375–387.

    Article  Google Scholar 

  • Yang, Y., Ding, C., Lee, S., Yu, L., & Ma, F. (2021). A modified Teunter-Syntetos-Babai method for intermittent demand forecasting. Journal of Management Science and Engineering, 6(1), 53–63.

    Article  Google Scholar 

  • Zhang, G., Patuwo, B. E., & Hu, M. Y. (1998). Forecasting with artificial neural networks: The state of the art. International Journal of Forecasting, 14(1), 35–62.

  • Zhang, A., Lipton, Z. C., Li, M., and Smola, A. J. (2021). Dive into deep learning, arXiv:2106.11342.

  • Zhou, C. X., & Viswanathan, S. (2011). Comparison of a new bootstrapping method with parametric approaches for safety stock determination in service parts inventory systems. International Journal of Production Economics, 133(1), 481–485.

    Article  Google Scholar 

  • Zhu, S., Dekker, R., van Jaarsveld, W., Renjie, R. W., & Koning, A. J. (2017). An improved method for forecasting spare parts demand using extreme value theory. European Journal of Operational Research, 261(1), 169–181.

    Article  Google Scholar 

  • Zhuang, Z., Yu, Y., & Chen, A. (2022). A combined forecasting method for intermittent demand using the automotive aftermarket data. Data Science and Management, 5(2), 43–56.

    Article  Google Scholar 

Download references

Funding

The authors have no relevant financial or non-financial interests to disclose.

Author information

Authors and Affiliations

  1. Department of Management, Robinson College of Business, Georgia State University, Atlanta, GA, 30303, USA

    G. Peter Zhang

  2. Institute for Insight, Robinson College of Business, Georgia State University, Atlanta, GA, 30303, USA

    Yusen Xia & Maohua Xie

Authors
  1. G. Peter Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yusen Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maohua Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yusen Xia.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, G.P., Xia, Y. & Xie, M. Intermittent demand forecasting with transformer neural networks. Ann Oper Res 339, 1051–1072 (2024). https://doi.org/10.1007/s10479-023-05447-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-023-05447-7

Keywords