Skip to main content
SpringerLink
Log in

Performance Evaluation of Container-Based High Performance Computing Ecosystem Using OpenPOWER

  • Conference paper
  • First Online:
Book cover High Performance Computing (ISC High Performance 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10524))

Included in the following conference series:

Abstract

Container-based High Performance Computing (HPC) has started gaining popularity due to its almost negligible performance penalty compared to the BareMetal hardware. Although HPC hardware architectures and programming models are continuously evolving, the platform models are suffering from the HPC community’s awareness. Power-awareness, hardware and application-aware co-design along with security related concerns have attracted the most in recent time to empower the platform models. Otherside, the OpenPOWER ecosystems have stepped into our life to fulfill the thirst of exploiting the last drop of a performance benefit from our invested system. It has Power8 compliant processor with a larger cache, big fat instructions and data path accompanied by the popular coherent accelerator. In this paper, we have proposed a container-based HPC ecosystem established using OpenPOWER machine. The performance of the designed and developed ecosystem is evaluated stressing on different subcomponents of the system such as processor, memory, IO, and interconnect. Finally, the results are compared with the performance of the equivalent environments made of virtual machines and BareMetal hardware.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    https://lwn.net/Articles/531114/.

  2. 2.

    https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt.

  3. 3.

    https://docs.docker.com/.

  4. 4.

    https://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.vsphere.networking.doc%2FGUID-CC021803-30EA-444D-BCBE-618E0D836B9F.html.

  5. 5.

    https://www.theubercloud.com/.

  6. 6.

    http://www.nersc.gov/.

  7. 7.

    http://www.iozone.org/.

References

  1. Adufu, T., Choi, J., Kim, Y.: Is container-based technology a winner for high performance scientific applications? In: 17th Asia-Pacific Network Operations and Management Symposium: Managing a Very Connected World, APNOMS 2015, pp. 507–510 (2015)

    Google Scholar 

  2. Basili, V.R., Carver, J., Cruzes, D.S., Hochstein, L., Hollingsworth, J.K., Shull, F., Zelkowitz, M.V.: Understanding the high performance computing community: a software engineer’s perspective. IEEE Softw. 25(4), 29–36 (2008)

    Article  Google Scholar 

  3. Bernstein, D.: Containers and cloud: from LXC to Docker to kubernetes. IEEE Cloud Comput. 1, 81–84 (2014)

    Article  Google Scholar 

  4. Beserra, D., Moreno, E.D., Endo, P.T., Barreto, J., Sadok, D., Fernandes, S.: Performance analysis of LXC for HPC environments. In: Proceedings - 2015 9th International Conference on Complex, Intelligent, and Software Intensive Systems, CISIS 2015, pp. 358–363 (2015)

    Google Scholar 

  5. Brief, S.: Dell solutions for high performance computing. Technical report, June 2015

    Google Scholar 

  6. Canon, R.S., Jacobsen, D.: Shifter: containers for HPC. In: Cray Users Group Conference (CUG 2016) (2016)

    Google Scholar 

  7. Carter, J., Oliker, L., Shalf, J.: Performance evaluation of scientific applications on modern parallel vector systems. In: Daydé, M., Palma, J.M.L.M., Coutinho, Á.L.G.A., Pacitti, E., Lopes, J.C. (eds.) VECPAR 2006. LNCS, vol. 4395, pp. 490–503. Springer, Heidelberg (2007). doi:10.1007/978-3-540-71351-7_38

    Chapter  Google Scholar 

  8. Cisco; Red Hat: Linux Containers: Why They’re in Your Future and What Has to Happen First Application Delivery: Today’s Challenges. Technical report (2014)

    Google Scholar 

  9. Conway, S., Joseph, E.C., Sorensen, B.: An Approach for Designing HPC Systems with Better Balance and Performance, vol. (1), pp. 1–7 (2016)

    Google Scholar 

  10. Dongarra, J.J., Luszczek, P.: Overview of the HPC challenge benchmark suite. In: Proceeding of SPEC Benchmark Workshop. Citeseer (2006)

    Google Scholar 

  11. Dunigan, T.H., Vetter, J.S., White, J.B., Worley, P.H.: Performance evaluation of the cray X1 distributed shared-memory architecture. IEEE Micro 25, 30–40 (2005)

    Article  Google Scholar 

  12. Felter, W., Ferreira, A., Rajamony, R., Rubio, J.: An updated performance comparison of virtual machines and Linux containers. In: 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp. 171–172 (2015)

    Google Scholar 

  13. Gerlach, W., Tang, W., Keegan, K., Harrison, T., Wilke, A., Bischof, J., Dsouza, M., Devoid, S., Murphy-Olson, D., Desai, N., Meyer, F.: Skyport - container-based execution environment management for multi-cloud scientific workflows. In: Proceedings of DataCloud 2014: 5th International Workshop on Data Intensive Computing in the Clouds - Held in Conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 25–32 (2015)

    Google Scholar 

  14. Gupta, A., Kale, L.V., Gioachin, F., March, V., Suen, C.H., Lee, B.S., Faraboschi, P., Kaufmann, R., Milojicic, D.: The Who, What, Why, and How of high performance computing in the cloud. In: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science, pp. 306–314 (2013)

    Google Scholar 

  15. Gupta, A., Kalé, L.V., Milojicic, D.S., Faraboschi, P., Kaufmann, R., March, V., Gioachin, F., Suen, C.H., Lee, B.S.: Exploring the performance and mapping of HPC applications to platforms in the cloud. In: Proceedings of the 21st International Symposium on High-Performance Parallel and Distributed Computing - HPDC 2012, pp. 121–122 (2012)

    Google Scholar 

  16. He, Q., Zhou, S., Kobler, B., Duffy, D., Mcglynn, T.: Case study for running HPC applications in public clouds. In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing - HPDC 2010, pp. 395–401 (2010)

    Google Scholar 

  17. Jackson, K., Ramakrishnan, L., Muriki, K., Canon, S., Cholia, S., Shalf, J., Wasserman, H., Wright, N.: Performance analysis of high performance computing applications on the amazon web services cloud. In: 2010 IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom) (2010)

    Google Scholar 

  18. Jacobsen, D.M., Canon, R.S.: Contain this, unleashing Docker for HPC. In: Cray User Group 2015 (2015)

    Google Scholar 

  19. Kanso, A., Huang, H., Gherbi, A.: Can Linux containers clustering solutions offer high availability? In: Second Workshop on Containers (WoC) - Colocated with IC2E 2016 (2016)

    Google Scholar 

  20. Liu, D., Zhao, L.: The research and implementation of cloud computing platform based on Docker. In: 2014 11th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), pp. 475–478 (2014)

    Google Scholar 

  21. Lu, K., Chi, W., Liu, Y., Tang, H.: HPVZ: a high performance virtual computing environment for super computers. Technical report 111072 (2009)

    Google Scholar 

  22. Mancini, M., Aloisio, G.: How advanced cloud technologies can impact and change HPC environments for simulation. In: Proceedings of the 2015 International Conference on High Performance Computing and Simulation, HPCS 2015, No. Cmcc, pp. 667–668 (2015)

    Google Scholar 

  23. Matthews, J.N., Hu, W., Hapuarachchi, M., Deshane, T., Dimatos, D., Hamilton, G., McCabe, M., Owens, J.: Quantifying the performance isolation properties of virtualization systems. In: Proceedings of the 2007 Workshop on Experimental Computer Science (ExpCS 2007), p. 6 (2007)

    Google Scholar 

  24. Ruiu, P., Terzo, O., Carlino, G., Prandi, R., Falzone, A., Maggi, P., Torterolo, L., Usai, E., Perego, G.: HPC CloudPills: on-demand deployment and execution of HPC application in cloud environments. In: Proceedings - 2014 9th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing, 3PGCIC 2014, pp. 82–88 (2015)

    Google Scholar 

  25. Ruiz, C., Jeanvoine, E., Nussbaum, L.: Performance evaluation of containers for HPC. In: Hunold, S., Costan, A., Giménez, D., Iosup, A., Ricci, L., Gómez Requena, M.E., Scarano, V., Varbanescu, A.L., Scott, S.L., Lankes, S., Weidendorfer, J., Alexander, M. (eds.) Euro-Par 2015. LNCS, vol. 9523, pp. 813–824. Springer, Cham (2015). doi:10.1007/978-3-319-27308-2_65

    Chapter  Google Scholar 

  26. Scheepers, M.J.: Virtualization and containerization of application infrastructure: a comparison. In: 21st Twente Student Conference on IT, pp. 1–7 (2014)

    Google Scholar 

  27. Soltesz, S., Pötzl, H., Fiuczynski, M.E., Bavier, A., Peterson, L.: Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors. ACM SIGOPS Operating Syst. Rev. 41(3), 275–287 (2007)

    Article  Google Scholar 

  28. Stankovski, V., Taherizadeh, S., Taylor, I., Jones, A., Mastroianni, C., Becker, B., Suhartanto, H.: Towards an environment supporting resilience, high-availability, reproducibility and reliability for cloud applications. In: 2015 IEEE/ACM 8th International Conference on Utility and Cloud Computing (UCC), pp. 383–386 (2015)

    Google Scholar 

  29. Tang, W., Wilkening, J., Desai, N., Gerlach, W., Wilke, A., Meyer, F., Bischof, J., Gerlach, W., Wilke, A., Desai, N., Meyer, F.: A scalable data analysis platform for metagenomics. In: 2013 IEEE International Conference on Big Data, pp. 21–26 (2013)

    Google Scholar 

  30. Varghese, B., Subba, L.T., Thai, L., Barker, A.: Container-based cloud virtual machine benchmarking. In: 2016 IEEE International Conference on Cloud Engineering (IC2E), pp. 192–201 (2016)

    Google Scholar 

  31. Weidner, O., Atkinson, M., Barker, A., Vicente, R.F.: Rethinking high performance computing platforms: challenges, opportunities and recommendations, pp. 19–26 (2016)

    Google Scholar 

  32. Xavier, M., Neves, M., Rossi, F., Ferreto, T., Lange, T., De Rose, C.: Performance evaluation of container-based virtualization for high performance computing environments, pp. 233–240 (2013)

    Google Scholar 

  33. Yu, H.e., Huang, W.: Building a virtual HPC cluster with auto scaling by the Docker. In: Computing Research Repository, vol. abs/1509.0, p. 4 (2015)

    Google Scholar 

  34. Zhang, J., Lu, X., Arnold, M., Panda, D.K.: MVAPICH2 over OpenStack with SR-IOV: an efficient approach to build HPC clouds. In: Proceedings - 2015 IEEE/ACM 15th International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2015, pp. 71–80 (2015)

    Google Scholar 

  35. Zhang, J., Lu, X., Panda, D.K.: Performance characterization of hypervisor-and container-based virtualization for HPC on SR-IOV enabled InfiniBand clusters. In: 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 1777–1784 (2016)

    Google Scholar 

  36. Zheng, C., Thain, D.: Integrating containers into workflows. In: Proceedings of the 8th International Workshop on Virtualization Technologies in Distributed Computing - VTDC 2015, vol. 2, pp. 31–38 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. High Performance Computing Lab, Department of Computer Science and Engineering, Indian Institute of Technology, Roorkee, 247667, India

    Animesh Kuity & Sateesh Kumar Peddoju

Authors
  1. Animesh Kuity
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sateesh Kumar Peddoju
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Animesh Kuity .

Editor information

Editors and Affiliations

  1. Deutsches Klimarechenzentrum (DKRZ), Hamburg, Hamburg, Germany

    Julian M. Kunkel

  2. TITECH, Tokyo, Japan

    Rio Yokota

  3. Department of Computer Science, University of Delaware, Newark, Delaware, USA

    Michela Taufer

  4. Lawrence Berkeley National Laboratory, Berkeley, California, USA

    John Shalf

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Kuity, A., Peddoju, S.K. (2017). Performance Evaluation of Container-Based High Performance Computing Ecosystem Using OpenPOWER. In: Kunkel, J., Yokota, R., Taufer, M., Shalf, J. (eds) High Performance Computing. ISC High Performance 2017. Lecture Notes in Computer Science(), vol 10524. Springer, Cham. https://doi.org/10.1007/978-3-319-67630-2_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67630-2_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67629-6

  • Online ISBN: 978-3-319-67630-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation