research-article

Conditional Generative Adversarial Network Based Workload Generation for Cloud Cluster

Authors:

Jun Xu,

Junwei Liu,

Jun Yao,

Ao Ma,

Lei Xu,

Xinghua ZhaoAuthors Info & Claims

ACAI '22: Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence

Article No.: 65, Pages 1 - 6

https://doi.org/10.1145/3579654.3579723

Published: 14 March 2023 Publication History

Get Access

Abstract

Improving cluster utilization by scheduling and decision-making is a long-standing research problem for cloud vendors. The workload models can improve decision-making by providing a provision of the future workload for the public cloud scheduler. However, capturing the correlations in real traces was proven to be hard in former research. In this paper, we introduce a conditional generative adversarial network (CGAN) based model, which can provide a long-time provision for the cloud cluster. In the proposed approach, the workload model is generated by three-stage training with conditional GAN, which is not limited by the assumption of Poisson distribution. Besides, the conditional representation is redesigned based on Time2Vec, which makes the inter-job correlations among virtual machines (VMs) can be modeled accurately. According to our validation, the job arrival model accuracy is raised from 52.7% to 80.3% compared with the Poisson regression method, which indicated that the proposed CGAN-based model is a more universal and accurate generative model for large-scale cloud clusters.

References

[1]

Hadary, O., Marshall, L., Menache, I., Pan, A., Greeff, E.E., Dion, D., Dorminey, S., Joshi, S., Chen, Y., Russinovich, M., and others 2020. Protean:${$VM$}$ allocation service at scale. 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20) (2020), 845–861.

Google Scholar

[2]

Lu, C., Ye, K., Xu, G., Xu, C.-Z. and Bai, T. 2017. Imbalance in the cloud: An analysis on Alibaba cluster trace. 2017 IEEE International Conference on Big Data (Big Data) (Boston, MA, Dec. 2017), 2884–2892.

Crossref

Google Scholar

[3]

Reiss, C., Wilkes, J. and Hellerstein, J.L. 2011. Google cluster-usage traces: format+ schema. Google Inc., White Paper. 1, (2011).

Google Scholar

[4]

Calzarossa, M.C., Massari, L. and Tessera, D. 2016. Workload characterization: A survey revisited. ACM Computing Surveys (CSUR). 48, 3 (2016), 1–43.

Digital Library

Google Scholar

[5]

Verma, A., Korupolu, M. and Wilkes, J. 2014. Evaluating job packing in warehouse-scale computing. 2014 IEEE International Conference on Cluster Computing (CLUSTER) (2014), 48–56.

Crossref

Google Scholar

[6]

Khan, A., Yan, X., Tao, S. and Anerousis, N. 2012. Workload characterization and prediction in the cloud: A multiple time series approach. 2012 IEEE Network Operations and Management Symposium (2012), 1287–1294.

Crossref

Google Scholar

[7]

Bergsma, S., Zeyl, T., Senderovich, A. and Beck, J.C. 2021. Generating Complex, Realistic Cloud Workloads using Recurrent Neural Networks. Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles CD-ROM (Virtual Event Germany, Oct. 2021), 376–391.

Digital Library

Google Scholar

[8]

Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B. and Bharath, A.A. 2018. Generative Adversarial Networks: An Overview. IEEE Signal Processing Magazine. 35, 1 (Jan. 2018), 53–65.

Crossref

Google Scholar

[9]

Wang, K., Gou, C., Duan, Y., Lin, Y., Zheng, X. and Wang, F.-Y. 2017. Generative adversarial networks: introduction and outlook. IEEE/CAA Journal of Automatica Sinica. 4, 4 (2017), 588–598.

Crossref

Google Scholar

[10]

Gauthier, J. 2014. Conditional generative adversarial nets for convolutional face generation. Class project for Stanford CS231N: convolutional neural networks for visual recognition, Winter semester. 2014, 5 (2014), 2.

Google Scholar

[11]

Mirza, M. and Osindero, S. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784. (2014).

Google Scholar

[12]

Kazemi, S.M., Goel, R., Eghbali, S., Ramanan, J., Sahota, J., Thakur, S., Wu, S., Smyth, C., Poupart, P. and Brubaker, M. 2019. Time2vec: Learning a vector representation of time. arXiv preprint arXiv:1907.05321. (2019).

Google Scholar

[13]

Cortez, E., Bonde, A., Muzio, A., Russinovich, M., Fontoura, M. and Bianchini, R. 2017. Resource central: Understanding and predicting workloads for improved resource management in large cloud platforms. Proceedings of the 26th Symposium on Operating Systems Principles (2017), 153–167.

Digital Library

Google Scholar

Cited By

View all

Yildiz MBaiocchi A(2024)Data-Driven Workload Generation Based on Google Data Center Measurements2024 IEEE 25th International Conference on High Performance Switching and Routing (HPSR)10.1109/HPSR62440.2024.10635925(143-148)Online publication date: 22-Jul-2024
https://doi.org/10.1109/HPSR62440.2024.10635925
Liang YRuan NYi LSu X(2024)An approach to workload generation for modern data centers: A view from Alibaba traceBenchCouncil Transactions on Benchmarks, Standards and Evaluations10.1016/j.tbench.2024.1001644:1(100164)Online publication date: Mar-2024
https://doi.org/10.1016/j.tbench.2024.100164

Index Terms

Conditional Generative Adversarial Network Based Workload Generation for Cloud Cluster

Index terms have been assigned to the content through auto-classification.

Recommendations

Next generation cloud computing

The landscape of cloud computing has significantly changed over the last decade. Not only have more providers and service offerings crowded the space, but also cloud infrastructure that was traditionally limited to single provider data centers is now ...
Cloud service engineering
ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2

Building on compute and storage virtualization, Cloud Computing provides scalable, network-centric, abstracted IT infrastructure, platforms, and applications as on-demand services that are billed by consumption. Cloud Service Engineering is the ...
A genetic algorithm for workload scheduling in cloud based e-learning
CloudCP '12: Proceedings of the 2nd International Workshop on Cloud Computing Platforms

Cloud computing is a game-changing paradigm in modern computing, offering a better utilization of IT resources and at the same time reducing the operational costs and increasing the elasticity of the IT infrastructure. Clouds exist in various shapes and ...

Comments

Information & Contributors

Information

Published In

ACAI '22: Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence

December 2022

770 pages

ISBN:9781450398336

DOI:10.1145/3579654

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 March 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ACAI 2022

ACAI 2022: 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence

December 23 - 25, 2022

Sanya, China

Acceptance Rates

Overall Acceptance Rate 173 of 395 submissions, 44%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
48
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yildiz MBaiocchi A(2024)Data-Driven Workload Generation Based on Google Data Center Measurements2024 IEEE 25th International Conference on High Performance Switching and Routing (HPSR)10.1109/HPSR62440.2024.10635925(143-148)Online publication date: 22-Jul-2024
https://doi.org/10.1109/HPSR62440.2024.10635925
Liang YRuan NYi LSu X(2024)An approach to workload generation for modern data centers: A view from Alibaba traceBenchCouncil Transactions on Benchmarks, Standards and Evaluations10.1016/j.tbench.2024.1001644:1(100164)Online publication date: Mar-2024
https://doi.org/10.1016/j.tbench.2024.100164

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

References

Cited By

Index Terms

Recommendations

Next generation cloud computing

Cloud service engineering

A genetic algorithm for workload scheduling in cloud based e-learning

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

HTML Format

Share

Share this Publication link

Share on social media

Affiliations