Abstract
IHK/McKernel is a lightweight multi-kernel operating system that is designed for extreme-scale HPC systems. The basic idea of IHK/McKernel is to run Linux and a lightweight kernel (LWK) side by side on each compute node to provide both LWK scalability and full Linux compatibility. IHK/McKernel is one of the first multi-kernels that has been evaluated at large scale and that has demonstrated the advantages of the multi-kernel approach. This chapter describes the architecture of IHK/McKernel, provides insights into some of its unique features, and describes its ability to outperform Linux through experiments. We also discuss our experiences and lessons learned so far.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bailey, D., Barszcz, E., Barton, J., Browning, D., Carter, R., Dagum, L., et al. (1991). The nas parallel benchmarks. The International Journal of High Performance Computing Applications, 5(3), 63–73.
BDEC Committee, (2017). The BDEC “Pathways to convergence” report. http://www.exascale.org/bdec/.
Brown, N. (2018). Overlay filesystem documentation. https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt.
Cavium, (2014). ThunderX_CP family of workload optimized compute processors.
Clements, A. T., Kaashoek, M. F., & Zeldovich, N. (2012). Scalable address spaces using RCU balanced trees. In Proceedings of the Seventeenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’12.
Dongarra, J., Heroux, M. A., & Luszczek, P. (2015). HPCG benchmark: A new metric for ranking high performance computing systems. Technical Report UT-EECS-15-736, University of Tennessee, Electrical Engineering and Computer Science Department.
Gerofi, B., Riesen, R., Takagi, M., Boku, T., Ishikawa, Y., & Wisniewski, R. W. (2018). Performance and scalability of lightweight multi-kernel based operating systems. In 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS).
Gerofi, B., Shimada, A., Hori, A., & Ishikawa, Y. (2013). Partially separated page tables for efficient operating system assisted hierarchical memory management on heterogeneous architectures. In 13th International Symposium on Cluster, Cloud and Grid Computing (CCGrid).
Gerofi, B., Shimada, A., Hori, A., Masamichi, T., & Ishikawa, Y. (2014). CMCP: A novel page replacement policy for system level hierarchical memory management on many-cores. In Proceedings of the 23rd International Symposium on High-performance Parallel and Distributed Computing, HPDC (pp. 73–84). New York, NY, USA: ACM.
Gerofi, B., Takagi, M., Hori, A., Nakamura, G., Shirasawa, T., & Ishikawa, Y. (2016). On the scalability, performance isolation and device driver transparency of the IHK/McKernel hybrid lightweight kernel. In 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (pp. 1041–1050).
Henson, V. E., & Yang, U. M. (2002). BoomerAMG: A parallel algebraic multigrid solver and preconditioner. Applied Numerical Mathematics, 41, 155–177.
Heroux, M. A., Doerfler, D. W., Crozier, P. S., Willenbring, J. M., Edwards, H. C., Williams, A., et al. (2009). Improving performance via Mini-applications. Technical Report SAND2009-5574, Sandia National Laboratories.
InfiniBand Trade Association. (2016). InfiniBand Architecture Specification, Release 1.3.1.
Ishikawa, K.-I., Kuramashi, Y., Ukawa, A., & Boku, T. (2017). CCS QCD application. https://github.com/fiber-miniapp/ccs-qcd.
Jeffers, J., & Reinders, J. (2013). Intel Xeon Phi coprocessor high performance programming. Burlington: Morgan Kaufmann.
Joint Center for Advanced HPC (JCAHPC) (2018). Basic specification of Oakforest-PACS. http://jcahpc.jp/files/OFP-basic.pdf.
Mauerer, W. (2010). Professional Linux kernel architecture (1st ed.). Birmingham: Wrox Press.
Merkel, D. (2014). Docker: Lightweight Linux containers for consistent development and deployment. Linux Journal, 2014(239).
RIKEN Advanced Institute for Computational Science (2018). K computer. http://www.aics.riken.jp/en/k-computer/about/.
Shimosawa, T. (2011). Operating system organization for manycore systems. http://www.ipsj.or.jp/magazine/hakase/2011/OS01.html.
Shimosawa, T., Gerofi, B., Takagi, M., Nakamura, G., Shirasawa, T., Saeki, Y. (2014). Interface for heterogeneous Kernels: A framework to enable hybrid OS designs targeting high performance computing on manycore architectures. In 21th International Conference on High Performance Computing, HiPC.
Yoshida, T., Hondou, M., Tabata, T., Kan, R., Kiyota, N., Kojima, H., et al. (2015). Sparc64 XIfx: Fujitsu’s next-generation processor for high-performance computing. IEEE Micro, 35(2), 6–14.
Acknowledgements
This work has been partially funded by MEXT’s program for the Development and Improvement of Next Generation Ultra High-Speed Computer Systems under its subsidies for operating the Specific Advanced Large Research Facilities in Japan.
We acknowledge Tomoki Shirasawa, Gou Nakamura, and Ken Sato from Hitachi for their McKernel development efforts. We thank Prof. Taisuke Boku from The University of Tsukuba and Prof. Nakajima Kengo from The University of Tokyo for their support to let us access the Oakforest-PACS supercomputer. We also thank the Intel mOS team for all the invaluable discussions on design considerations of multi-kernel operating systems and for providing us with information on the configuration of mini-applications we used for evaluation. Finally, we acknowledge Takahiro Ogura and Toshiro Saiki from Fujitsu for their relentless support regarding the operation of the OFP machine.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Gerofi, B., Takagi, M., Ishikawa, Y. (2019). IHK/McKernel. In: Gerofi, B., Ishikawa, Y., Riesen, R., Wisniewski, R.W. (eds) Operating Systems for Supercomputers and High Performance Computing. High-Performance Computing Series, vol 1. Springer, Singapore. https://doi.org/10.1007/978-981-13-6624-6_17
Download citation
DOI: https://doi.org/10.1007/978-981-13-6624-6_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-6623-9
Online ISBN: 978-981-13-6624-6
eBook Packages: Computer ScienceComputer Science (R0)