Abstract
Predicting the resource demands of online tasks plays an important role in data centers, which can help cloud providers to better allocate resources and to schedule tasks. To cope with the huge number of workloads in a data center, workloads are usually clustered first and then prediction is conducted for each cluster. However, training different models for different clusters separately reduces the overall utilization of the data in the data center, potentially reducing the prediction ability of the whole predicting system. Inspired by federated learning, we propose Performer, a Transformer-based forecasting model for clustered massive time-series. Each cluster of workloads is viewed as a local dataset owned by a training worker and all workers cooperate to train a global prediction model, while local models are trained by workers respectively. By combining global model and local models in an encoder-decoder architecture, Performer can learn global information and local information to perform predictions while keeping low model deployment costs. By splitting time-series into blocks and calculating self-attention inner-blocks, Performer keeps good prediction accuracy with lower computation cost than other Transformer-based time-series forecasting methods, making it more suitable for data center usage. Experiments on an online tasks workload dataset show Performer is an effective method in the scenario of cluster-based forecasting.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Tang, X., Liao, X., Zheng, J., Yang, X.: Energy efficient job scheduling with workload prediction on cloud data center. Clust. Comput. 21(3), 1581–1593 (2018). https://doi.org/10.1007/s10586-018-2154-7
Cortez, E., Bonde, A., Muzio, A., Russinovich, M., Fontoura, M., Bianchini, R.: Resource central: understanding and predicting workloads for improved resource management in large cloud platforms. In: Proceedings of the 26th Symposium on Operating Systems Principles, pp. 153–167 (2017)
Islam, S., Keung, J., Lee, K., Liu, A.: Empirical prediction models for adaptive resource provisioning in the cloud. Futur. Gener. Comput. Syst. 28(1), 155–162 (2012)
Yu, Y., Jindal, V., Yen, I.L., Bastani, F.: Integrating clustering and learning for improved workload prediction in the cloud. In: 2016 IEEE 9th International Conference on Cloud Computing (CLOUD), pp. 876–879 (2016)
McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)
Kulkarni, V., Kulkarni, M., Pant, A.: Survey of personalization techniques for federated learning. In: 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), pp. 794–797. IEEE (2020)
Box, G.E., Jenkins, G.M.: Some recent advances in forecasting and control. J. R. Stat. Soc. C Appl. Stat. 17(2), 91–109 (1968)
Zivot, E., Wang, J.: Vector autoregressive models for multivariate time series. In: Modeling Financial Time Series with S-PLUS®, pp. 385–429 (2006). https://doi.org/10.1007/978-0-387-32348-0_11
Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Box, G.E., Jenkins, G.M., Reinsel, G.C., Ljung, G.M.: Time Series Analysis: Forecasting and Control. Wiley, Hoboken (2015)
Rzadca, K., et al.: Autopilot: workload autoscaling at google. In: Proceedings of the Fifteenth European Conference on Computer Systems, pp. 1–16 (2020)
Ca Lheiros, R.N., Masoumi, E., Ranjan, R., Buyya, R.: Workload prediction using ARIMA model and its impact on cloud applications’ QoS. IEEE Trans. Cloud Comput. 3(4), 449–458 (2015)
Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001). https://doi.org/10.1023/A:1010933404324
Song, B., Yu, Y., Zhou, Yu., Wang, Z., Du, S.: Host load prediction with long short-term memory in cloud computing. J. Supercomput. 74(12), 6554–6568 (2017). https://doi.org/10.1007/s11227-017-2044-4
Lai, G., Chang, W.C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 95–104 (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wu, N., Green, B., Ben, X., O’Banion, S.: Deep transformer models for time series forecasting: the influenza prevalence case. arXiv preprint arXiv:2001.08317 (2020)
Song, H., Rajan, D., Thiagarajan, J.J., Spanias, A.: Attend and diagnose: clinical time series analysis using attention models. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, vol. 32, No. 1 (2018)
Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., Eickhoff, C.: A transformer-based framework for multivariate time series representation learning. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2114–2124 (2021)
Lim, B., Arık, S.Ö., Loeff, N., Pfister, T.: Temporal fusion transformers for interpretable multi-horizon time series forecasting. Int. J. Forecast. 37(4), 1748–1764 (2021)
Ma, J., Shou, Z., Zareian, A., Mansour, H., Vetro, A., Chang, S.F.: CDSA: cross-dimensional self-attention for multivariate, geo-tagged time series imputation. arXiv preprint arXiv:1905.09904 (2019)
Li, S., et al.: Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Adv. Neural. Inf. Process. Syst. 32, 5243–5253 (2019)
Zhou, H., et al.: Informer: beyond efficient transformer for long sequence time-series forecasting. Proc. AAAI Conf. Artif. Intell. 35(12), 11106–11115 (2021)
Mansour, Y., Mohri, M., Ro, J., Suresh, A.T.: Three approaches for personalization with applications to federated learning. Comput. Sci. arXiv preprint arXiv:2002.10619 (2020)
Wang, K., Mathews, R., Kiddon, C., Eichner, H., Beaufays, F., Ramage, D.: Federated evaluation of on-device personalization. arXiv preprint arXiv:1910.10252 (2019)
Smith, V., Chiang, C.K., Sanjabi, M., Talwalkar, A.S.: Federated multi-task learning. Adv. Neural Inf. Process. Syst. 30 (2017)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR (2017)
Acknowledgement
This research is partially supported by Guangdong Natural Science Foundation of China (2018B030312002).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Qi, W., Yao, J., Li, J., Wu, W. (2023). Performer: A Resource Demand Forecasting Method for Data Centers. In: Yu, C., Zhou, J., Song, X., Lu, Z. (eds) Green, Pervasive, and Cloud Computing. GPC 2022. Lecture Notes in Computer Science, vol 13744. Springer, Cham. https://doi.org/10.1007/978-3-031-26118-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-26118-3_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26117-6
Online ISBN: 978-3-031-26118-3
eBook Packages: Computer ScienceComputer Science (R0)