Abstract
With the continuous improvement of supercomputer performance and the integration of artificial intelligence with traditional scientific computing, the scale of applications is gradually increasing, from millions to tens of millions of computing cores, which raises great challenges to achieve high scalability and efficiency of parallel applications on super-large-scale systems. Taking the Sunway exascale prototype system as an example, in this paper we first analyze the challenges of high scalability and high efficiency for parallel applications in the exascale era. To overcome these challenges, the optimization technologies used in the parallel supporting environment software on the Sunway exascale prototype system are highlighted, including the parallel operating system, input/output (I/O) optimization technology, ultra-large-scale parallel debugging technology, 10-million-core parallel algorithm, and mixed-precision method. Parallel operating systems and I/O optimization technology mainly support large-scale system scaling, while the ultra-large-scale parallel debugging technology, 10-million-core parallel algorithm, and mixed-precision method mainly enhance the efficiency of large-scale applications. Finally, the contributions to various applications running on the Sunway exascale prototype system are introduced, verifying the effectiveness of the parallel supporting environment design.
摘要
随着超级计算机性能不断提高，人工智能与传统科学计算的进一步融合，应用的并行规模逐渐增加，从数百万个计算核心到数千万个计算核心，这对超大规模系统上实现并行应用的高可扩展性和高效率提出巨大挑战。本文首先以神威E级原型系统为例，分析了E级时代并行应用的高可扩展性和高效率面临的挑战。为克服这些挑战，重点介绍了神威E级原型系统上并行支撑环境软件的优化技术，包括并行操作系统、I/O优化技术、超大规模并行调试技术、千万核心并行算法、混合精度方法等。并行操作系统和I/O优化技术主要支持大规模系统扩展，而超大规模并行调试技术、千万核心并行算法和混合精度方法主要提升大规模应用的效率。最后，介绍了运行在神威E级原型系统上的应用程序取得的重要成果，从而验证了并行支撑环境设计的有效性。
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The data that support the findings of this study are available from the corresponding authors upon reasonable request.
References
Arute F, Arya K, Babbush R, et al., 2019. Quantum supremacy using a programmable superconducting processor. Nature, 574(7779):505–510. https://doi.org/10.1038/s41586-019-1666-5
Berendsen HJC, van der Spoel D, van Drunen R, 1995. Gromacs: a message-passing parallel molecular dynamics implementation. Comput Phys Commun, 91(1–3):43–56. https://doi.org/10.1016/0010-4655(95)00042-E
Buluc A, Gilbert JR, 2012. Parallel sparse matrix-matrix multiplication and indexing: implementation and experiments. SIAM J Sci Comput, 34(4):C170–C191. https://doi.org/10.1137/110848244
Chen Q, Chen K, Chen ZN, et al., 2020. Lessons learned from optimizing the Sunway storage system for higher application I/O performance. J Comput Sci Technol, 35(1):47–60. https://doi.org/10.1007/s11390-020-9798-5
Derouillat J, Beck A, Pérez F, et al., 2018. SMILEI: a collaborative, open-source, multi-purpose particle-in-cell code for plasma simulation. Comput Phys Commun, 222:351–373. https://doi.org/10.1016/j.cpc.2017.09.024
Fu HH, Liao JF, Yang JZ, et al., 2016. The Sunway TaihuLight supercomputer: system and applications. Sci China Inform Sci, 59(7):072001. https://doi.org/10.1007/s11432-016-5588-7
Gu J, Feng JW, Hao XY, et al., 2021. Establishing a nonhydrostatic global atmospheric modeling system (iAMAS) at 3-km horizontal resolution with online integrated aerosol feedbacks on the Sunway supercomputer of China. https://arxiv.org/abs/2112.04668v1
Guo C, Liu Y, Xiong M, et al., 2019. General-purpose quantum circuit simulator with projected entangled-pair states and the quantum supremacy frontier. Phys Rev Lett, 123(19):190501. https://doi.org/10.1103/PhysRevLett.123.190501
Guo C, Zhao YW, Huang HL, 2021. Verifying random quantum circuits with arbitrary geometry using tensor network states algorithm. Phys Rev Lett, 126(7):070502. https://doi.org/10.1103/PhysRevLett.126.070502
Hluchý L, Bobák M, Müller H, et al., 2020. Heterogeneous exascale computing. In: Kovács L, Haidegger T, Szakál A (Eds.), Recent Advances in Intelligent Engineering. Springer, Cham, p.81–110. https://doi.org/10.1007/978-3-030-14350-3_5
Hofer P, Mössenböck H, 2014. Efficient and accurate stack trace sampling in the Java hotspot virtual machine. Proc 5th ACM/SPEC Int Conf on Performance Engineering, p.277–280. https://doi.org/10.1145/2568088.2576759
Hua Y, Shi X, Jin H, et al., 2019. Software-defined QoS for I/O in exascale computing. CCF Trans High Perform Comput, 1(1):49–59. https://doi.org/10.1007/s42514-019-00005-9
Huang C, Zhang F, Newman M, et al., 2020. Classical simulation of quantum supremacy circuits. https://arxiv.org/abs/2005.06787
Ji X, Yang B, Zhang TY, et al., 2019. Automatic, application-aware I/O forwarding resource allocation. Proc 17th USENIX Conf on File and Storage Technologies, p.265–279.
Jia WL, Wang H, Chen MH, et al., 2020. Pushing the limit of molecular dynamics with ab initio accuracy to 100 million atoms with machine learning. Proc Int Conf for High Performance Computing, Networking, Storage and Analysis, p.1–14. https://doi.org/10.1109/SC41405.2020.00009
Kurth T, Treichler S, Romero J, et al., 2018. Exascale deep learning for climate analytics. Proc Int Conf for High Performance Computing, Networking, Storage and Analysis, p.649–660. https://doi.org/10.1109/SC.2018.00054
Li F, Liu X, Liu Y, et al., 2021. SW_Qsim: a minimize-memory quantum simulator with high-performance on a new Sunway supercomputer. Proc Int Conf for High Performance Computing, Networking, Storage and Analysis, p.1–13.
Li MF, Chen JS, Xiao Q, et al., 2022. Bridging the gap between deep learning and frustrated quantum spin system for extreme-scale simulations on new generation of Sunway supercomputer. IEEE Trans Parall Distrib Syst, 33(11):2846–2859. https://doi.org/10.1109/TPDS.2022.3145163
Lin F, Liu Y, Guo YY, et al., 2021. ELS: emulation system for debugging and tuning large-scale parallel programs on small clusters. J Supercomput, 77(2):1635–1666. https://doi.org/10.1007/s11227-020-03319-6
Lindahl E, Hess B, van der Spoel D, 2001. GROMACS 3.0: a package for molecular simulation and trajectory analysis. J Mol Model, 7(8):306–317. https://doi.org/10.1007/s008940100045
Liu S, Gao J, Liu X, et al., 2021. Establishing high performance AI ecosystem on Sunway platform. CCF Trans High Perform Comput, 3(3):224–241. https://doi.org/10.1007/s42514-021-00072-x
Liu Y, Liu X, Li F, et al., 2021. Closing the “quantum supremacy” gap: achieving real-time simulation of a random quantum circuit using a new Sunway supercomputer. Proc Int Conf for High Performance Computing, Networking, Storage and Analysis, Article 3. https://doi.org/10.1145/3458817.3487399
Ma YJ, Lv S, Liu YQ, 2012. Introduction and application of cluster file system Lustre. Sci Technol Inform, (5):139–140 (in Chinese).
Madduri K, Ibrahim KZ, Williams S, et al., 2011. Gyrokinetic toroidal simulations on leading multi- and many-core HPC systems. Proc Int Conf for High Performance Computing, Networking, Storage and Analysis, p.1–12. https://doi.org/10.1145/2063384.2063415
Markov IL, Shi YY, 2008. Simulating quantum computation by contracting tensor networks. SIAM J Comput, 38(3):963–981. https://doi.org/10.1137/050644756
Merrill D, Garland M, 2017. Merge-based parallel sparse matrix-vector multiplication. Proc Int Conf for High Performance Computing, Networking, Storage and Analysis, p.678–689. https://doi.org/10.1109/SC.2016.57
Micikevicius P, Narang S, Alben J, et al., 2018. Mixed precision training. Proc 6th Int Conf on Learning Representations.
Pan F, Zhang P, 2021. Simulating the Sycamore quantum supremacy circuits. https://arxiv.org/abs/2103.03074v1
Peng D, Feng Y, Liu Y, et al., 2022. Jdebug: a fast, non-intrusive and scalable fault locating tool for ten-million-scale parallel applications. IEEE Trans Parall Distrib Syst, 33(12):3491–3504. https://doi.org/10.1109/TPDS.2022.3157690
Shang HH, Li F, Zhang YQ, et al., 2021a. Extreme-scale ab initio quantum Raman spectra simulations on the leadership HPC system in China. Proc Int Conf for High Performance Computing, Networking, Storage and Analysis, Article 6. https://doi.org/10.1145/3458817.3487402
Shang HH, Li F, Zhang YQ, et al., 2021b. Accelerating all-electron ab initio simulation of Raman spectra for biological systems. Proc Int Conf for High Performance Computing, Networking, Storage and Analysis, Article 41. https://doi.org/10.1145/3458817.3476160
Shang HH, Chen X, Gao XY, et al., 2021c. TensorKMC: kinetic Monte Carlo simulation of 50 trillion atoms driven by deep learning on a new generation of Sunway supercomputer. Proc Int Conf for High Performance Computing, Networking, Storage and Analysis, Article 73. https://doi.org/10.1145/3458817.3476174
Shi X, Li M, Liu W, et al., 2017. SSDUP: a traffic-aware SSD burst buffer for HPC systems. Proc Int Conf on Supercomputing, p.1–10. https://doi.org/10.1145/3079079.3079087
Shoeybi M, Patwary M, Puri R, et al., 2019. Megatron-LM: training multi-billion parameter language models using model parallelism. https://arxiv.org/abs/1909.08053
Trott O, Olson AJ, 2009. AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. J Comput Chem, 31(2):455–461. https://doi.org/10.1002/jcc.21334
Villalonga B, Boixo S, Nelson B, et al., 2019. A flexible highperformance simulator for verifying and benchmarking quantum circuits implemented on real hardware. NPJ Quant Inform, 5(1):86. https://doi.org/10.1038/s41534-019-0196-1
Villalonga B, Lyakh D, Boixo S, et al., 2020. Establishing the quantum supremacy frontier with a 281 Pflop/s simulation. Quant Sci Technol, 5(3):034003. https://doi.org/10.1088/2058-9565/ab7eeb
Xiao JY, Chen JS, Zheng JS, et al., 2021. Symplectic structure-preserving particle-in-cell whole-volume simulation of tokamak plasmas to 111.3 trillion particles and 25.7 billion grids. Proc Int Conf for High Performance Computing, Networking, Storage and Analysis, Article 2. https://doi.org/10.1145/3458817.3487398
Yang B, Ji X, Ma XS, et al., 2019. End-to-end I/O monitoring on a leading supercomputer. Proc 16th USENIX Conf on Networked Systems Design and Implementation, p.379–394.
Yang B, Zou YL, Liu WG, et al., 2022. An end-to-end and adaptive I/O optimization tool for modern HPC storage systems. IEEE Int Parallel and Distributed Processing Symp, p.1294–1304. https://doi.org/10.1109/IPDPS53621.2022.00128
Ye YJ, Song ZY, Zhou SC, et al., 2022. swNEMO_v4.0: an ocean model based on NEMO4 for the new-generation Sunway supercomputer. Geosci Model Dev, 15(14):5739–5756. https://doi.org/10.5194/gmd-15-5739-2022
Author information
Authors and Affiliations
Contributions
Xin LIU and Dexun CHEN designed the research. Heng GUO, Yuling YANG, and Jie GAO performed the simulations. Yunlong FENG and Longde CHEN analyzed the results. Xiaobin HE and Xin CHEN drafted the paper. Xiaona DIAO and Zuoning CHEN helped organize the paper. Xiaobin HE and Xin CHEN revised and finalized the paper.
Corresponding authors
Additional information
Compliance with ethics guidelines
Xiaobin HE, Xin CHEN, Heng GUO, Xin LIU, Dexun CHEN, Yuling YANG, Jie GAO, Yunlong FENG, Longde CHEN, Xiaona DIAO, and Zuoning CHEN declare that they have no conflict of interest.
Xiaobin HE received his BE degree from the Harbin Institute of Technology, Harbin, China, in 2006, and his MS degree from Shanghai Jiao Tong University, Shanghai, China, in 2009. He is currently an associate researcher at the National Research Center of Parallel Computer Engineering and Technology, Beijing, China. His main research interests include high-performance computing and distributed storage systems.
Xin CHEN received his BE degree from the National Digital Switching System Engineering & Technological Research Center (NDSC), Zhengzhou, China, in 2016, and his MS degree from NDSC in 2018. He is a research assistant at the National Research Center of Parallel Computer Engineering and Technology, Beijing, China. His research activities focus on high-performance parallel computation and applications.
Xin LIU received her PhD degree from PLA Information Engineering University, Zhengzhou, China, in 2006. She is currently a research fellow at the National Research Center of Parallel Computer Engineering and Technology, Beijing, China. She is a designer of the scientific and engineering application platform of the Sunway TaihuLight System, responsible for the large-scale parallel algorithm research and application software development. Her research interests include parallel algorithms and parallel application software.
Dexun CHEN received his PhD degree from Tsinghua University, Beijing, China, in 2021. He is currently a research fellow at the National Research Center of Parallel Computer Engineering and Technology, Beijing, China. His research interests include high-performance computing and parallel application software.
Project supported by the Key R&D Program of Zhejiang Province, China (No. 2022C01250) and the National Key R&D Program of China (No. 2019YFA0709402)
Rights and permissions
About this article
Cite this article
He, X., Chen, X., Guo, H. et al. Scalability and efficiency challenges for the exascale supercomputing system: practice of a parallel supporting environment on the Sunway exascale prototype system. Front Inform Technol Electron Eng 24, 41–58 (2023). https://doi.org/10.1631/FITEE.2200412
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2200412