Skip to main content
SpringerLink
Log in

Scheduling Microservice Containers on Large Core Machines Through Placement and Coalescing

  • Conference paper
  • First Online:
Job Scheduling Strategies for Parallel Processing (JSSPP 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12985))

Included in the following conference series:

Abstract

Current applications deployed on the cloud use the microservice programming model to enable rapid deployment. Due to the loosely coupled nature of the interactions between microservices, they are well suited for the distributed nature of cloud systems. However, we are seeing a trend of increasing core counts on newer server hardware and scheduling microservices agnostic to the local organization of cores and memory on these systems leads to sub-optimal performance. In this paper, we propose a placement scheme to map containers of a microservice to various cores on such machines to maximize performance. We further study the impact of various parameters such as packet sizes and database sizes on the placement scheme and demonstrate that our placement scheme increases throughput by 22% while simultaneously lowering tail latency. Finally, we propose a mechanism to dynamically coalesce services on commonly called paths into a single container and demonstrate a further 7.5% improvement in throughput.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amd \(\mu \)prof. https://developer.amd.com/amd-uprof

  2. Docker swarm overview. https://docs.docker.com/engine/swarm/

  3. Facebook social networks. http://networkrepository.com/socfb

  4. Kubernetes: Production-grade container orchestration. https://kubernetes.io

  5. Kubernetes topology manager. https://kubernetes.io/docs/tasks/administer-cluster/topology-manager/

  6. Netperf: A network performance benchmark. https://linux.die.net/man/1/netperf

  7. Networkx: Network analysis in python. https://networkx.org

  8. perf: Linux profiling with performance counters. https://perf.wiki.kernel.org/index.php/Main_Page

  9. Sock shop microservice application. https://microservices-demo.github.io

  10. Tshark: Terminal based wireshark. https://www.wireshark.org/docs/man-pages/tshark.html

  11. Alles, G.R., Carissimi, A., Schnorr, L.M.: Assessing the computation and communication overhead of Linux containers for HPC applications. In: 2018 Symposium on High Performance Computing Systems (WSCAD), pp. 116–123. IEEE (2018)

    Google Scholar 

  12. Athlur, S., Sondhi, N., Batra, S., Kalambur, S., Sitaram, D.: Cache characterization of workloads in a microservice environment. In: 2019 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM), pp. 45–50. IEEE (2019)

    Google Scholar 

  13. Buzato, F.H., Goldman, A., Batista, D.: Efficient resources utilization by different microservices deployment models. In: 2018 IEEE 17th International Symposium on Network Computing and Applications (NCA), pp. 1–4. IEEE (2018)

    Google Scholar 

  14. Caculo, S., Lahiri, K., Kalambur, S.: Characterizing the scale-up performance of microservices using teastore. In: 2020 IEEE International Symposium on Workload Characterization (IISWC), pp. 48–59. IEEE (2020)

    Google Scholar 

  15. Chen, J., Chiew, K., Ye, D., Zhu, L., Chen, W.: AAGA: affinity-aware grouping for allocation of virtual machines. In: 2013 IEEE 27th International Conference on Advanced Information Networking and Applications (AINA), pp. 235–242. IEEE (2013)

    Google Scholar 

  16. Chen, S., Delimitrou, C., Martínez, J.F.: Parties: QoS-aware resource partitioning for multiple interactive services. In: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 107–120 (2019)

    Google Scholar 

  17. Daoud, M.I., Kharma, N.: A high performance algorithm for static task scheduling in heterogeneous distributed computing systems. J. Parallel Distrib. Comput. 68(4), 399–409 (2008)

    Article  Google Scholar 

  18. Dean, J., Barroso, L.A.: The tail at scale. Commun. ACM 56(2), 74–80 (2013)

    Article  Google Scholar 

  19. Delimitrou, C., Kozyrakis, C.: Paragon: QoS-aware scheduling for heterogeneous datacenters. ACM SIGPLAN Not. 48, 77–88 (2013)

    Article  Google Scholar 

  20. Dragoni, N., et al.: Microservices: yesterday, today, and tomorrow. In: Dragoni, M., Meyer, B. (eds.) Present and Ulterior Software Engineering, pp. 195–216. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67425-4_12

    Chapter  Google Scholar 

  21. Felter, W., Ferreira, A., Rajamony, R., Rubio, J.: An updated performance comparison of virtual machines and Linux containers. In: 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp. 171–172. IEEE (2015)

    Google Scholar 

  22. Gan, Y., et al.: An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 3–18. ACM (2019)

    Google Scholar 

  23. Georgiou, Y., Jeannot, E., Mercier, G., Villiermet, A.: Topology-aware job mapping. Int. J. High Perform. Comput. Appl. 32(1), 14–27 (2018)

    Article  Google Scholar 

  24. Glozer, W.: WRK - a http benchmarking tool. https://github.com/wg/wrk

  25. Goutham, K.S., Ujjaini Kempaiah, B., John Mampilli, R., Kalambur, S.: Performance sensitivity of operating system parameters in microservice environments (in press)

    Google Scholar 

  26. Guo, Y., Yao, W.: A container scheduling strategy based on neighborhood division in micro service. In: NOMS 2018–2018 IEEE/IFIP Network Operations and Management Symposium, pp. 1–6. IEEE (2018)

    Google Scholar 

  27. Hu, Y., De Laat, C., Zhao, Z.: Multi-objective container deployment on heterogeneous clusters. In: 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), pp. 592–599. IEEE (2019)

    Google Scholar 

  28. Hu, Y., Zhou, H., de Laat, C., Zhao, Z.: ECSched: efficient container scheduling on heterogeneous clusters. In: Aldinucci, M., Padovani, L., Torquati, M. (eds.) Euro-Par 2018. LNCS, vol. 11014, pp. 365–377. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-96983-1_26

    Chapter  Google Scholar 

  29. Hu, Y., Zhou, H., de Laat, C., Zhao, Z.: Concurrent container scheduling on heterogeneous clusters with multi-resource constraints. Futur. Gener. Comput. Syst. 102, 562–573 (2020)

    Article  Google Scholar 

  30. Kaewkasi, C., Chuenmuneewong, K.: Improvement of container scheduling for docker using ant colony optimization. In: 2017 9th International Conference on Knowledge and Smart Technology (KST), pp. 254–259. IEEE (2017)

    Google Scholar 

  31. Kaffes, K., Yadwadkar, N.J., Kozyrakis, C.: Centralized core-granular scheduling for serverless functions. In: Proceedings of the ACM Symposium on Cloud Computing, pp. 158–164 (2019)

    Google Scholar 

  32. Karypis, G., Schloegel, K., Kumar, V.: Parmetis. Parallel graph partitioning and sparse matrix ordering library. Version 2 (2003)

    Google Scholar 

  33. von Kistowski, J., Eismann, S., Schmitt, N., Bauer, A., Grohmann, J., Kounev, S.: TeaStore: a micro-service reference application for benchmarking, modeling and resource management research. In: 2018 IEEE 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 223–236. IEEE (2018)

    Google Scholar 

  34. Kramer, S.: Gigaom–the biggest thing amazon got right: the platform (2011)

    Google Scholar 

  35. Liu, B., Li, P., Lin, W., Shu, N., Li, Y., Chang, V.: A new container scheduling algorithm based on multi-objective optimization. Soft. Comput. 22(23), 7741–7752 (2018). https://doi.org/10.1007/s00500-018-3403-7

    Article  Google Scholar 

  36. Mao, Y., Oak, J., Pompili, A., Beer, D., Han, T., Hu, P.: DRAPS: dynamic and resource-aware placement scheme for docker containers in a heterogeneous cluster. In: 2017 IEEE 36th International Performance Computing and Communications Conference (IPCCC), pp. 1–8. IEEE (2017)

    Google Scholar 

  37. Mars, J., Tang, L., Hundt, R., Skadron, K., Soffa, M.L.: Bubble-up: increasing utilization in modern warehouse scale computers via sensible co-locations. In: Proceedings of the 44th Annual IEEE/ACM International Symposium on Microarchitecture, pp. 248–259 (2011)

    Google Scholar 

  38. Mars, J., Vachharajani, N., Hundt, R., Soffa, M.L.: Contention aware execution: online contention detection and response. In: Proceedings of the 8th annual IEEE/ACM International Symposium on Code Generation and Optimization, pp. 257–265 (2010)

    Google Scholar 

  39. Mauro, T.: Adopting microservices at Netflix: lessons for architectural design (2015). https://www.nginx.com/blog/microservices-at-netflix-architectural-best-practices

  40. Nathuji, R., Kansal, A., Ghaffarkhah, A.: Q-clouds: managing performance interference effects for QoS-aware clouds. In: Proceedings of the 5th European Conference on Computer Systems, pp. 237–250 (2010)

    Google Scholar 

  41. Novaković, D., Vasić, N., Novaković, S., Kostić, D., Bianchini, R.: DeepDive: transparently identifying and managing performance interference in virtualized environments. In: Presented as Part of the 2013 USENIX Annual Technical Conference ATC 2013), pp. 219–230 (2013)

    Google Scholar 

  42. Patel, T., Tiwari, D.: CLITE: efficient and QoS-aware co-location of multiple latency-critical jobs for warehouse scale computers. In: 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 193–206. IEEE (2020)

    Google Scholar 

  43. Petrucci, V., et al.: Octopus-man: QoS-driven task management for heterogeneous multicores in warehouse-scale computers. In: 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), pp. 246–258. IEEE (2015)

    Google Scholar 

  44. Rahman, J., Lama, P.: Predicting the end-to-end tail latency of containerized microservices in the cloud. In: 2019 IEEE International Conference on Cloud Engineering (IC2E), pp. 200–210. IEEE (2019)

    Google Scholar 

  45. Sampaio, A.R., Rubin, J., Beschastnikh, I., Rosa, N.S.: Improving microservice-based applications with runtime placement adaptation. J. Internet Serv. Appl. 10(1), 1–30 (2019). https://doi.org/10.1186/s13174-019-0104-0

    Article  Google Scholar 

  46. Sonnek, J., Greensky, J., Reutiman, R., Chandra, A.: Starling: minimizing communication overhead in virtualized computing platforms using decentralized affinity-aware migration. In: 2010 39th International Conference on Parallel Processing, pp. 228–237. IEEE (2010)

    Google Scholar 

  47. Sriraman, A., Wenisch, T.F.: \(\mu \)tune: auto-tuned threading for \(\{\)OLDI\(\}\) microservices. In: 13th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 2018), pp. 177–194 (2018)

    Google Scholar 

  48. Tang, L., Mars, J., Zhang, X., Hagmann, R., Hundt, R., Tune, E.: Optimizing Google’s warehouse scale computers: the NUMA experience. In: 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA), pp. 188–197. IEEE (2013)

    Google Scholar 

  49. Tene, G.: WRK2 - a constant throughput, correct latency recording variant of wrk. https://github.com/giltene/wrk2

  50. Thiyyakat, M., Kalambur, S., Sitaram, D.: Improving resource isolation of critical tasks in a workload. In: Klusáček, D., Cirne, W., Desai, N. (eds.) JSSPP 2020. LNCS, vol. 12326, pp. 45–67. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63171-0_3

    Chapter  Google Scholar 

  51. Ueda, T., Nakaike, T., Ohara, M.: Workload characterization for microservices. In: 2016 IEEE International Symposium on Workload Characterization (IISWC), pp. 1–10. IEEE (2016)

    Google Scholar 

  52. Xu, X., Yu, H., Pei, X.: A novel resource scheduling approach in container based clouds. In: 2014 IEEE 17th International Conference on Computational Science and Engineering, pp. 257–264. IEEE (2014)

    Google Scholar 

  53. Zhang, D., Yan, B.H., Feng, Z., Zhang, C., Wang, Y.X.: Container oriented job scheduling using linear programming model. In: 2017 3rd International Conference on Information Management (ICIM), pp. 174–180. IEEE (2017)

    Google Scholar 

Download references

Acknowledgement

We would like to thank AMD India Private Limited for their funding of this project.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, PES University, Bangalore, India

    Vishal Rao, Vishnu Singh, K. S. Goutham, Bharani Ujjaini Kempaiah, Ruben John Mampilli & Subramaniam Kalambur

  2. Cloud Computing Innovation Council of India (CCICI), Bangalore, India

    Dinkar Sitaram

Authors
  1. Vishal Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vishnu Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. K. S. Goutham
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bharani Ujjaini Kempaiah
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ruben John Mampilli
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Subramaniam Kalambur
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dinkar Sitaram
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. CESNET, Prague, Czech Republic

    Dalibor Klusáček

  2. Google, Mountain View, CA, USA

    Walfredo Cirne

  3. Apple, Cupertino, CA, USA

    Gonzalo P. Rodrigo

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rao, V. et al. (2021). Scheduling Microservice Containers on Large Core Machines Through Placement and Coalescing. In: Klusáček, D., Cirne, W., Rodrigo, G.P. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2021. Lecture Notes in Computer Science(), vol 12985. Springer, Cham. https://doi.org/10.1007/978-3-030-88224-2_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88224-2_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88223-5

  • Online ISBN: 978-3-030-88224-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation