Abstract
Currently, low resource utilization and high costs of cloud platform are becoming big challenges to cloud provider. However, due to confidentiality, few cloud platform providers are willing to publish resource utilization data of their cloud platform. This poses great difficulties in designing an effective cloud resource scheduler. Fortunately, Alibaba released its cloud resource usage data in September 2017. This paper analyzes Alibaba cloud trace data deeply from different aspects and reveals some important features of resource utilization. These features will help to design effective resource management approaches for cloud platform: (1) The maximum resource utilization of online services is closely related to their average utilization. (2) The longer a batch instance runs, the longer it may last. (3) The type of job that runs in a container can be estimated according to the amount of consumed resources and life time of this container. (4) Actual resources used by different batch jobs vary with time greatly and static resource allocation would make resource wasted seriously.
This work was supported by the National Natural Science Foundation of China (61602351).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hindman, B., Konwinski, A., Zaharia, M., et al.: Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, pp. 429–483. USENIX (2011)
Delimitrou, C., Kozyrakis, C.: Quasar: resource-efficient and QoS-aware cluster management. In: Proceedings of International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 127–144. ACM (2014)
Poggi, N., Carrera, D., Gavalda, R., et al.: Characterization of workload and resource consumption for an online travel and booking site. In: Proceedings of IEEE International Symposium on Workload Characterization, pp. 1–10. IEEE (2010)
Zheng, Z., Yu, L., Tang, W., et al.: Co-analysis of RAS log and job log on Blue Gene/P. In: Proceedings of Parallel & Distributed Processing Symposium, pp. 840–851. IEEE (2011)
Li, C., Dai, B., Kuang, Z., et al.: Research on task scheduling with multiple constraints based on genetic algorithm in cloud computing environment. J. Chin. Comput. Syst. 38(9), 1945–1949 (2017)
Reiss, C., Tumanov, A., Ganger, G.R., et al.: Heterogeneity and dynamicity of clouds at scale: google trace analysis. In: Proceedings of ACM Symposium on Cloud Computing, pp. 1–13. ACM (2012)
Verma, A., Pedrosa, L., Korupolu, M., et al.: Large-scale cluster management at Google with Borg. In: Proceedings of Tenth European Conference on Computer Systems. ACM (2015)
Cortez, E., Bonde, A., Muzio, A., et al.: Resource central: understanding and predicting workloads for improved resource management in large cloud platforms. In: Proceedings of Symposium on Operating Systems Principles, pp. 153–167 (2017)
Delimitrou, C., Kozyrakis, C.: HCloud: resource-efficient provisioning in shared cloud systems. In: Proceedings of International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 473–488. ACM (2016)
Lu, C., Ye, K., Xu, G.: Imbalance in the cloud: an analysis on Alibaba cluster trace. In: Proceedings of IEEE International Conference on Big Data. IEEE (2017)
Yang, S., Zhang, Q.: Research on k-nearest neighbor text classification algorithm of approximation set of rough set. J. Chin. Comput. Syst. 38(10), 2192–2196 (2017)
Meng, X., Bradley, J., Yavuz, B., et al.: MLlib: machine learning in apache spark. J. Mach. Learn. Res. 17(1), 1235–1241 (2015)
Abadi, M., Barham, P., Chen, J., et al.: TensorFlow: a system for large-scale machine learning. In: Proceedings of the 12th USENIX Symposium on Operating System Design and Implementation, pp. 265–283. USENIX (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Deng, L., Ren, YL., Xu, F., He, H., Li, C. (2018). Resource Utilization Analysis of Alibaba Cloud. In: Huang, DS., Bevilacqua, V., Premaratne, P., Gupta, P. (eds) Intelligent Computing Theories and Application. ICIC 2018. Lecture Notes in Computer Science(), vol 10954. Springer, Cham. https://doi.org/10.1007/978-3-319-95930-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-95930-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-95929-0
Online ISBN: 978-3-319-95930-6
eBook Packages: Computer ScienceComputer Science (R0)