Skip to main content
SpringerLink
Log in

Accurate workload prediction for edge data centers: Savitzky-Golay filter, CNN and BiLSTM with attention mechanism

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Workload prediction is a fundamental task in edge data centers, which aims to accurately estimate the workload to achieve an in-situ resource provisioning for workload execution. In this paper, we propose a deep learning model termed SG-CBA to predict workload, which is powered by Savitzky-Golay filter (SG filter), Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) with Attention mechanism. First, raw time series of the workload is normalized and smoothed by a preprocessing module with SG filter. Following that, we establish a deep learning module based on CNN and BiLSTM with attention mechanism to extract and process the features for the accurate workload prediction. Real-world workload from Alibaba cluster is adopted to validate our proposed model in the experiments. Experimental results demonstrate that SG-CBA can achieve accurate workload prediction, which outperforms the alternatives, including BTH-ARIMA, LSTNet, OCRO-MLNN, Recurrent Neural Network (RNN), Gated Recurrent Unit (GRU), LSTM and BiLSTM under different evaluation metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Aazam M, Zeadally S, Harras KA (2018) Deploying fog computing in industrial internet of things and industry 4.0. IEEE Trans Ind Inf 14(10):4674–4682. https://doi.org/10.1109/TII.2018.2855198

    Article  Google Scholar 

  2. Ahmed U, Mukhiya SK, Srivastava G, Lamo Y, Lin JCW (2021) Attention-based deep entropy active learning using lexical algorithm for mental health treatment. Front Psychol 12:471. https://doi.org/10.3389/fpsyg.2021.642347

    Google Scholar 

  3. Antonescu A-F, Braun T (2016) Simulation of SLA-based VM-scaling algorithms for cloud-distributed applications. Futur Gener Comput Syst 54(C):260–273. https://doi.org/10.1016/j.future.2015.01.015

    Article  Google Scholar 

  4. Alhussein M, Aurangzeb K, Haider SI (2020) Hybrid CNN-LSTM model for short-term individual household load forecasting. IEEE Access 8:180,544–180,557. https://doi.org/10.1109/ACCESS.2020.3028281

    Article  Google Scholar 

  5. Awawdeh M, Faisal T, Bashir AF, Sheikh A (2019) Application of outlier detection using re-weighted least squares and r-squared for IoT extracted data. In: 2019 Advances in science and engineering technology international conferences (ASET). https://doi.org/10.1109/ICASET.2019.8714261, pp 1–6

  6. Bi J, Yuan H, Zhou M (2019) Temporal prediction of multiapplication consolidated workloads in distributed clouds. IEEE Trans Autom Sci Eng 16(4):1763–1773. https://doi.org/10.1109/TASE.2019.2895801

    Article  Google Scholar 

  7. Bi J, Zhang L, Yuan H, Zhou M (2018) Hybrid task prediction based on wavelet decomposition and arima model in cloud data center. In: 15Th IEEE international conference on networking, sensing and control. https://doi.org/10.1109/ICNSC.2018.8361342, pp 1–6

  8. Alibaba trace (2018). https://github.com/alibaba/clusterdata

  9. Chen Z, Hu J, Min G, Zomaya AY, El-Ghazawi T (2020) Towards accurate prediction for high-dimensional and highly-variable cloud workloads with deep learning. IEEE Trans Parallel Distrib Syst 31(4):923–934. https://doi.org/10.1109/TPDS.2019.2953745

    Article  Google Scholar 

  10. Cisco (2020) Cisco annual internet report (2018–2023) white paper. https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.html. Accessed on January 21, 2021

  11. Elsisi M, Tran MQ, Mahmoud K, Lehtonen M, Darwish MMF (2021) Deep learning-based industry 4.0 and internet of things towards effective energy management for smart buildings. Sensors 21(4):1038. https://doi.org/10.3390/s21041038

    Article  Google Scholar 

  12. Guo Q, Huo R, Meng H, Xinhua E, Liu J, Huang T, Liu Y (2018) Research on LSTM-based load prediction for edge data centers. In: 2018 IEEE 4Th international conference on computer and communications (ICCC). https://doi.org/10.1109/CompComm.2018.8780738, pp 1825–1829

  13. Guo Y, Yao W (2018) Applying gated recurrent units pproaches for workload prediction. In: NOMS 2018 - 2018 IEEE/IFIP Network operations and management symposium. https://doi.org/10.1109/NOMS.2018.8406290, pp 1–6

  14. Gupta S, Dinesh DA (2017) Resource usage prediction of cloud workloads using deep bidirectional long short term memory networks. In: 2017 IEEE International conference on advanced networks and telecommunications systems (ANTS). https://doi.org/10.1109/ANTS.2017.8384098, pp 1–6

  15. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2016.90, pp 770–778

  16. Hu Y, Deng B, Peng F, Wang D (2016) Workload prediction for cloud computing elasticity mechanism. pp 244–249. https://doi.org/10.1109/ICCCBDA.2016.7529565

  17. Ashutosh JK, Singh K (2018) Workload prediction in cloud using artificial neural network and adaptive differential evolution. Fut Gener Comput Syst: FGCS 81:41–52. https://doi.org/10.1016/j.future.2017.10.047

    Article  Google Scholar 

  18. Kiranyaz S, Avci O, Abdeljaber O, Ince T, Inman D (2021) 1d convolutional neural networks and applications: A survey. Mech Syst Signal Process 151(2021). https://doi.org/10.1016/j.ymssp.2020.107398

  19. Kumar A, Mazumdar S (2016) Forecasting HPC workload using ARMA models and ssa. In: 2016 International conference on information technology (ICIT). https://doi.org/10.1109/ICIT.2016.065, pp 294–297

  20. Lai G, Chang WC, Yang Y, Liu H (2018) Modeling long- and short-term temporal patterns with deep neural networks. In: The 41st international ACM SIGIR conference. https://doi.org/10.1145/3209978.3210006

  21. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–44. https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  22. Li Z, Zhou X, Qin Y (2019) A survey of mobile edge computing in the industrial internet. In: 2019 7Th international conference on information, communication and networks (ICICN). https://doi.org/10.1109/ICICN.2019.8834959, pp 94–98

  23. Lin JCW, Shao Y, Djenouri Y, Yun U (2021) ASRNN: a recurrent neural network with an attention model for sequence labeling. Knowl-Based Syst 212(106):548. https://doi.org/10.1016/j.knosys.2020.106548

    Google Scholar 

  24. Lin JCW, Srivastava G, Zhang Y, Djenouri Y, Aloqaily M (2021) Privacy-preserving multiobjective sanitization model in 6G IoT environments. IEEE Internet Things J 8(7):5340–5349. https://doi.org/10.1109/JIOT.2020.3032896

    Article  Google Scholar 

  25. Miao Y, Wu G, Li M, Ghoneim A, Hossain MS (2020) Intelligent task prediction and computation offloading based on mobile-edge cloud computing. Futur Gener Comput Syst 102:925–931. https://doi.org/10.1016/j.future.2019.09.035

    Article  Google Scholar 

  26. Nguyen C, Klein C, Elmroth E (2019) Multivariate LSTM-based location-aware workload prediction for edge data centers. In: 2019 19Th IEEE/ACM international symposium on cluster, cloud and grid computing (CCGRID). https://doi.org/10.1109/CCGRID.2019.00048, pp 341–350

  27. Nguyen H, Woo S, Im J, Jun TJ, Kim D (2016) A workload prediction approach using models stacking based on recurrent neural network and autoencoder. In: 2016 IEEE 18Th international conference on high performance computing and communications; IEEE 14th international conference on smart city; IEEE 2nd international conference on data science and systems (HPCC/smartcity/DSS). https://doi.org/10.1109/HPCC-SmartCity-DSS.2016.0133, pp 929–936

  28. Nguyen T, Nguyen T, Nguyen BM, Nguyen G (2019) Efficient time-series forecasting using neural network and opposition-based coral reefs optimization. Int J Comput Intell Syst 12:1144–1161. https://doi.org/10.2991/ijcis.d.190930.003

    Article  Google Scholar 

  29. Ozcan M, Odaci F, Ari I (2019) Remote debugging for containerized applications in edge computing environments. In: 2019 IEEE International conference on edge computing (EDGE). https://doi.org/10.1109/EDGE.2019.00021, pp 30–32

  30. Ramezani F, Naderpour M (2017) A fuzzy virtual machine workload prediction method for cloud environments. In: 2017 IEEE International conference on fuzzy systems (FUZZ-IEEE). https://doi.org/10.1109/FUZZ-IEEE.2017.8015450, pp 1–6

  31. Ranjbari M, Torkestani JA (2018) A learning automata-based algorithm for energy and sla efficient consolidation of virtual machines in cloud data centers. J Parallel Distrib Comput 113:55–62. https://doi.org/10.1016/j.jpdc.2017.10.009

    Article  Google Scholar 

  32. Rikhtegar N, Keshtgari M, Bushehrian O, Pujolle G (2021) Bite: a dynamic bi-level traffic engineering model for load balancing and energy efficiency in data center networks. Appl Intell 51:4623–4648. https://doi.org/10.1007/s10489-020-02003-9

    Article  Google Scholar 

  33. Savitzky A, Golay M (1964) Smoothing and differentiation of data by simplified least squares procedures. Anal Chem 36:1627–1639. https://doi.org/10.1021/ac60214a047

    Article  Google Scholar 

  34. Shi Q, Yin J, Cai J, Cichocki A, Zeng J (2020) Block hankel tensor arima for multiple short time series forecasting. Proc AAAI Conf Artif Intell 34(4):5758–5766. https://doi.org/10.1609/aaai.v34i04.6032

    Google Scholar 

  35. Shih SY, Sun FK, Lee HY (2019) Temporal pattern attention for multivariate time series forecasting. Mach Learn 108:1421–1441. https://doi.org/10.1007/s10994-019-05815-0

    Article  MathSciNet  Google Scholar 

  36. Song B, Yu Y, Zhou Y, Wang Z, Du S (2018) Host load prediction with long short-term memory in cloud computing. J Supercomput 74:6554–6568. https://doi.org/10.1007/s11227-017-2044-4

    Article  Google Scholar 

  37. Srivastava G, Lin JCW, Zhang X, Li Y (2021) Large-scale high-utility sequential pattern analytics in internet of things. IEEE Internet Things J 8(16):12,669–12,678. https://doi.org/10.1109/JIOT.2020.3026826

    Article  Google Scholar 

  38. Wamba GM, Li Y, Orgerie AC, Beldiceanu N, Menaud JM (2017) Cloud workload prediction and generation models. In: 2017 29Th international symposium on computer architecture and high performance computing (SBAC-PAD). https://doi.org/10.1109/SBAC-PAD.2017.19, pp 89–96

  39. Yousuf Uddin M, Ahmad S (2020) A review on edge to cloud: Paradigm shift from large data centers to small centers of data everywhere. In: 2020 International conference on inventive computation technologies (ICICT). https://doi.org/10.1109/ICICT48043.2020.9112457

  40. Zhang Q, Yang L, Yan Z, Chen Z, Li P (2018) An efficient deep learning model to predict cloud workload for industry informatics. IEEE Trans Industrial Inf 14(7):3170–3178. https://doi.org/10.1109/TII.2018.2808910

    Article  Google Scholar 

  41. Zhang W, Li B, Zhao D, Gong F, Lu Q (2016) Workload prediction for cloud cluster using a recurrent neural network. In: 2016 International conference on identification, information and knowledge in the internet of things (IIKI). https://doi.org/10.1109/IIKI.2016.39, pp 104–109

  42. Zhong W, Zhuang Y, Sun J, Gu J (2018) A load prediction model for cloud computing using PSO-based weighted wavelet support vector machine. Appl Intell 48:4072–4083. https://doi.org/10.1007/s10489-018-1194-2

    Article  Google Scholar 

  43. Zhou J, He Z, Song YN, Wang H, Yang X, Lian W, Dai HN (2020) Precious metal price prediction based on deep regularization self-attention regression. IEEE Access 8:2178–2187. https://doi.org/10.1109/ACCESS.2019.2962202

    Article  Google Scholar 

  44. Zhou X, Lin F, Yang L, Nie J, Tan Q, Zeng W, Zhang N (2016) Load balancing prediction method of cloud storage based on analytic hierarchy process and hybrid hierarchical genetic algorithm. Springerplus 5(1):1989. https://doi.org/10.1186/s40064-016-3619-x

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by National Natural Science Foundation of China under Grant 62002071, Guangzhou Basic Research Project under Grant 202102020420, Program of Marine Economy Development (Six Marine Industries) Special Foundation of Department of Natural Resources of Guangdong Province under Grant GDNRC [2020]056, Science and Technology Projects of Guangzhou under Grant 202007040006, Top Youth Talent Project of Zhujiang Talent Program under Grant 2019QN01X516, and Guangdong Provincial Key Laboratory of Cyber-Physical System under Grant 2020B1212060069.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China

    Lei Chen, Weiwen Zhang & Haiming Ye

Authors
  1. Lei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiwen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haiming Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weiwen Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, L., Zhang, W. & Ye, H. Accurate workload prediction for edge data centers: Savitzky-Golay filter, CNN and BiLSTM with attention mechanism. Appl Intell 52, 13027–13042 (2022). https://doi.org/10.1007/s10489-021-03110-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03110-x

Keywords

Search

Navigation