Skip to main content
SpringerLink
Log in
Book cover

International Conference on Knowledge Science, Engineering and Management

KSEM 2021: Knowledge Science, Engineering and Management pp 670–683Cite as

QBT: Efficient and Flexible Resource Allocation Method for Data Center of State Grid Scenario

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12817))

Abstract

With the rapid development of computer technology, service platforms have increasingly become the industry's first choice for building service center. In order to use resources more efficiently, flexible resource allocation is a very important part of the service platform construction. In the service of State Grid, the issue of flexible resource allocation is super important. However, the business of the State Grid is very complicated, and the service will include the invocation of various business logics, and resource allocation cannot be simply carried out. In order to solve this problem, this paper designs an efficient and flexible resource allocation scheme-QBT (quota boosting task) based on this project. QBT has designed three functional modules: log collection, log analysis, and flexible resource allocation. Log collection and log analysis will restore the real request path and construct it as a DAG. The system will count the QPS of each DAG, and calculate the load corresponding to the service on the DAG. Based on this information, the elastic resource allocation will perform elastic resource allocation in the two situations when the remaining resources are sufficient and insufficient. The experimental results show that the QBT design scheme can successfully solve the resource scheduling problem of the National Grid Service Platform.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Zhu, M., et al.: Public vehicles for future urban transportation. IEEE Trans. Intell. Transp. Syst. 17(12), 3344–3353 (2016)

    Article  Google Scholar 

  2. Zhang, Q., Huang, T., Zhu, Y., Qiu, M.: A case study of sensor data collection and analysis in smart city: provenance in smart food supply chain. Int. J. Distribut. Sensor Netw. 9(11), 382132 (2013)

    Article  Google Scholar 

  3. Qiu, M., et al.: RNA nanotechnology for computer design and in vivo computation. Philos. Trans. R. Soc. A: Math. Phys. Eng. Sci. 371(2000), 20120310 (2013)

    Article  Google Scholar 

  4. Tao, L., Golikov, S., Gai, K., Qiu, M.: A reusable software component for integrated syntax and semantic validation for services computing. In: IEEE Symposium on Service-Oriented System Engineering (SOSE), pp. 127–132 (2015)

    Google Scholar 

  5. Gai, K., Qiu, M., Sun, X., Zhao, H.: Security and privacy issues: a survey on FinTech. In: International Conference on Smart Computing and Communication, pp. 236–247 (2016)

    Google Scholar 

  6. Qiu, M., Zhang, K., Huang, M.: An empirical study of web interface design on small display devices. In: IEEE/WIC/ACM International Conference on Web Intelligence (WI 2004), pp. 29–35 (2004)

    Google Scholar 

  7. Dai, W., Qiu, M., Qiu, L., Chen, L., Wu, A.: Who moved my data? Privacy protection in smartphones. IEEE Commun. Mag. 55(1), 20–25 (2017)

    Article  Google Scholar 

  8. Zhao, H., Chen, M., Qiu, M., Gai, K., Liu, M.: A novel pre-cache schema for high performance Android system. Futur. Gener. Comput. Syst. 56, 766–772 (2016)

    Article  Google Scholar 

  9. Shao, Z., et al.: Real-time dynamic voltage loop scheduling for multi-core embedded systems. IEEE Trans. Circuits Syst. II 54(5), 445–449 (2007)

    Article  Google Scholar 

  10. Qiu, H., Qiu, M., Memmi, G., Ming, Z., Liu, M.: A Dynamic Scalable Blockchain Based Communication Architecture for IoT. In: Qiu, M. (ed.) SmartBlock 2018. LNCS, vol. 11373, pp. 159–166. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-05764-0_17

    Chapter  Google Scholar 

  11. Gao, Y., Iqbal, S., Zhang, P., Qiu, M.: Performance and power analysis of high-density multi-GPGPU architectures: a preliminary case study. In: IEEE 17th International Conference on High Performance Computing (HPCC) (2015)

    Google Scholar 

  12. Schultz, J.C., Appel, H.M., Ferrieri, A.P., Arnold, T.M.: Flexible resource allocation during plant defense responses. Front. Plant Sci. 4, 324 (2013)

    Article  Google Scholar 

  13. Yaghubi-Namaad, M., Rahbar, A.G., Alizadeh, B.: Adaptive modulation and flexible resource allocation in space-division- multiplexed elastic optical networks. J. Opt. Commun. Netw. 10(3), 240–251 (2018)

    Article  Google Scholar 

  14. Katz, D., Schieber, B., Shachnai, H.: Flexible resource allocation to interval jobs. Algorithmica 81(8), 3217–3244 (2019)

    Article  MathSciNet  Google Scholar 

  15. Sawyer, N., Smith, D.B.: Flexible resource allocation in device-to-device communications using Stackelberg game theory. IEEE Trans. Commun. 67(1), 653–667 (2019)

    Article  Google Scholar 

  16. Angalakudati, M., et al.: Business analytics for flexible resource allocation under random emergencies. Manage. Sci. 60(6), 1552–1573 (2014)

    Article  Google Scholar 

  17. Tang, X., Li, K., Qiu, M., Sha, E.H.-M.: A hierarchical reliability-driven scheduling algorithm in grid systems. J. Parallel Distribut. Comput. 72(4), 525–535 (2012)

    Article  Google Scholar 

  18. Dai, W., Qiu, L., Wu, A., Qiu, M.: Cloud infrastructure resource allocation for big data applications. IEEE Trans. Big Data 4(3), 313–324 (2016)

    Article  Google Scholar 

  19. Guo, Y., Zhuge, Q., Hu, J., Yi, J., Qiu, M., Sha, E.H.: Data placement and duplication for embedded multicore systems with scratch pad memory. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 32(6), 809–817 (2013)

    Article  Google Scholar 

  20. Qiu, M., Ming, Z., Wang, J., Yang, L.T., Xiang, Y.: Enabling cloud computing in emergency management systems. IEEE Cloud Comput. 1(4), 60–67 (2014)

    Article  Google Scholar 

  21. Niu, J., Liu, C., Gao, Y., Qiu, M.: Energy efficient task assignment with guaranteed probability satisfying timing constraints for embedded systems. IEEE Trans. Parallel Distrib. Syst. 25(8), 2043–2052 (2013)

    Article  Google Scholar 

  22. Gai, K., Qiu, M.: Optimal resource allocation using reinforcement learning for IoT content-centric services. Appl. Soft Comput. 70, 12–21 (2018)

    Article  Google Scholar 

  23. Gai, K., Qiu, M.: Reinforcement learning-based content-centric services in mobile sensing. IEEE Netw. 32(4), 34–39 (2018)

    Article  Google Scholar 

  24. Kalliski, M., Engell, S.: Real-time resource efficiency indicators for monitoring and optimization of batch-processing plants. Can. J. Chem. Eng. 95(2), 265–280 (2017)

    Article  Google Scholar 

  25. Yadav, Y., Rama Krishna, C.: Real-time resource monitoring approach for detection of hotspot for virtual machine migration. Int. J. Inf. Technol. 11(4), 639–646 (2018)

    Google Scholar 

  26. Vega, C., Roquero, P., Leira, R., Gonzalez, I., Aracil, J.: Loginson: a transform and load system for very large-scale log analysis in large IT infrastructures. J. Supercomput. 73(9), 3879–3900 (2017)

    Article  Google Scholar 

  27. Huang, M., Dong, H., Wang, C.: Web link and transaction log analyses of digital archive websites. Tʻu shu kuan hsüeh yü tzŭ hsün kʻo hsüeh 39(2), 65–82 (2013)

    Google Scholar 

  28. DiCostanzo, D., Ayan, A., Woollard, J., Gupta, N.: SU-F-R-12: prediction of truebeam hardware issues using trajectory log analysis. Med. Phys. (Lancaster) 43(6), 3375 (2016)

    Google Scholar 

  29. Brebner, P.C.: Is your cloud elastic enough performance modeling the elasticity of infrastructure as a service (IaaS) cloud applications. In: ICPE 2012 Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering, pp.263–266. New York, USA (2012)

    Google Scholar 

  30. Shen, Z., Subbiah, S., Gu, X.: CloudScale: elastic resource scaling for multi tenant cloud systems. In: Proceedings of the 2nd ACM Symposium on Cloud Computing, pp. 1–14. New York, USA (Oct. 2011)

    Google Scholar 

  31. Hadji, M., Zeghlache, D.: Minimum cost maximum flow algorithm for dynamic resource allocation in clouds. In: IEEE 5th International Conference on Cloud Computing (CLOUD), pp.876–882. Honolulu (2012)

    Google Scholar 

  32. Duong, T.N.B., Li, X., Goh, R.S.M.: A framework for dynamic resource provisioning and adaptation in IaaS clouds. In: IEEE Third International Conference on Cloud Computing Technology and Science (CloudCom), pp. 312–319. Athens (2011)

    Google Scholar 

  33. Rao, J., Bu, X., Wang, K.: Self-adaptive provisioning of virtualized resources in cloud computing. ACM SIGMETRICS Perform. Eval. Rev. 39(1), 321–322 (2011)

    Article  Google Scholar 

  34. Buyya, R., Garg, S.K., Calheiros, R.N.: SLA-oriented resource provisioning for cloud computing challenges, architecture, and solutions. In: International Conference on Cloud and Service Computing (CSC), pp. 1–10. Hong Kong, China (2011)

    Google Scholar 

  35. Kranas, P.: ElaaS: an innovative Elasticity as a Service framewor-k for dynamic management across the cloud stack layers. In: Proceedings of Complex, Intelligent and Software Intensive Systems (CISIS), pp. 1042–1049. Palermo, Italy, 4–6 July (2012)

    Google Scholar 

  36. He, S., Guo, L., Guo, Y., Wu, C., Ghanem, M., Han, R.: Elastic application container: a lightweight approach for cloud resource provisioning. In: 2012 IEEE 26th International Conference on Advanced Information Networking and Applications (AINA), pp. 15–22. Fukuoka (2012)

    Google Scholar 

  37. Gong, Z., Gu, X., Wilkes, J.: PRESS: predictive elastic resource scaling for cloud systems. In: Proceedings of Network and Service Management, pp. 9–16. Niagara Falls, USA, 25–29 Oct. (2010)

    Google Scholar 

  38. Jie, Y., Qiu, J., Li, Y.: A profile-based approach to just-in-time scalability for cloud applications. In: IEEE International Conference on Cloud Computing, pp.9–16. Bangalore (2009)

    Google Scholar 

  39. Xu, W., et al.: Predictive control for dynamic resource allocation in enterprise data centers. In: 10th IEEE/IFIP Date of Conference, pp.115–126. Vancouver, Canada, 3–7 April (2006)

    Google Scholar 

  40. Gai, K., Qiu, M., Zhao, H., Liu, M.: Energy-aware optimal task assignment for mobile heterogeneous embedded systems in cloud computing. In: IEEE 3rd International Conference on Cyber Security and Cloud Computing (CSCloud) (2016)

    Google Scholar 

  41. Chen, L., Duan, Y., Qiu, M., Xiong, J., Gai, K.: Adaptive resource allocation optimization in heterogeneous mobile cloud systems. In: IEEE 2nd International Conference on Cyber Security and Cloud Computing (CSCloud) (2015)

    Google Scholar 

  42. Xu, Y., Li, K., Khac, T.T., Qiu, M.: A multiple priority queueing genetic algorithm for task scheduling on heterogeneous computing systems. In: IEEE 14th International Conference on High Performance Computing (HPCC) (2012)

    Google Scholar 

  43. Thakur, K., Qiu, M., Gai, K., Ali, M.L.: An investigation on cyber security threats and security models. In: IEEE 2nd International Conference on Cyber Security and Cloud Computing (CSCloud) (2015)

    Google Scholar 

  44. Gai, K., Qiu, M., Elnagdy, S.A.: A novel secure big data cyber incident analytics framework for cloud-based cybersecurity insurance. In: IEEE 2nd International Conference on Big Data Security on Cloud (CSCloud) (2016)

    Google Scholar 

  45. Zhang, Z., Wu, J., Deng, J., Qiu, M.: Jamming ACK attack to wireless networks and a mitigation approach. In: IEEE GLOBECOM, pp. 1–5 (2008)

    Google Scholar 

Download references

Acknowledgment

This work financially supported by Science and Technology Program of State Grid Corporation of China under Grant No.: 5700-202055183A-0-0-00, which named Research on Technology of Big Data Monitoring Analysis in Power Grid by Coordination of Data Middle platform & Edge Calculation. Without their help, it would be much harder to finish the program and this paper.

Author information

Authors and Affiliations

  1. State Grid Corporation of China, Beijing, China

    Zhengdong Ren & Jun Wang

  2. State Grid Shanghai Energy Interconnection Research Institute, Beijing, China

    Guangxian Lyu & Peng Liu

  3. Peking University, Beijing, China

    Wang Zhou & Yu Huang

Authors
  1. Zhengdong Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guangxian Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wang Zhou .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Han Qiu

  2. Ibaraki University, Hitachi, Japan

    Cheng Zhang

  3. University of Kentucky, Lexington, KY, USA

    Zongming Fei

  4. Texas A&M University – Commerce, Commerce, TX, USA

    Meikang Qiu

  5. Princeton University, Princeton, NJ, USA

    Sun-Yuan Kung

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ren, Z., Wang, J., Lyu, G., Liu, P., Zhou, W., Huang, Y. (2021). QBT: Efficient and Flexible Resource Allocation Method for Data Center of State Grid Scenario. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, SY. (eds) Knowledge Science, Engineering and Management. KSEM 2021. Lecture Notes in Computer Science(), vol 12817. Springer, Cham. https://doi.org/10.1007/978-3-030-82153-1_55

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-82153-1_55

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-82152-4

  • Online ISBN: 978-3-030-82153-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation