Skip to main content

Advertisement

SpringerLink
Log in

Parallel optimization of method of characteristics based on Sunway Bluelight II supercomputer

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

With the development of nuclear energy technology, reactor physical calculations have higher requirements for calculation accuracy and speed, and it has become an inevitable trend to use high-performance computers for reactor simulation calculations. The method of characteristics (MOC) is currently recognized as the preferred method for simulating neutron transport in the nuclear reactor core. Based on the architecture of Sunway many-core processor and Sunway Bluelight II supercomputer, this paper proposes a fine grained and universal two-level parallelization, including thread-level parallelization and process-level parallelization. In the thread-level parallelization, the methods such as job pipeline optimization, load balancing across CPEs, and I/O optimization are proposed for acceleration. In the process-level parallelization, a mapping method from software to hardware is proposed. This method can make full use of the hardware of Sunway supercomputers and improve the computing efficiency and data transmission efficiency. For the first time, the OpenMOC program is transplanted and parallelly optimized on the Sunway supercomputers, which enriched the application ecology of Sunway supercomputers. Compared with the original program, the two-level parallelization can achieve up to 18.6x speedup. Moreover, our parallelization is capable to run on more than 3750 processes of Sunway Bluelight II supercomputer with good strong and weak scalability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Availability of Data and Materials

Not applicable.

References

  1. Askew J (1972) A characteristics formulation of the neutron transport equation in complicated geometries. Technical report, United Kingdom Atomic Energy Authority

  2. Shang H, Chen X, Gao X, Lin R, Wang L, Li F, Xiao Q, Xu L, Sun Q, Zhu L, et al. (2021) Tensorkmc: kinetic monte carlo simulation of 50 trillion atoms driven by deep learning on a new generation of sunway supercomputer. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp 1–14

  3. Li M, Chen J, Xiao Q, Wang F, Jiang Q, Zhao X, Lin R, An H, Liang X, He L (2022) Bridging the gap between deep learning and frustrated quantum spin system for extreme-scale simulations on new generation of Sunway supercomputer. IEEE Trans Parallel Distrib Syst 33(11):2846–2859

    Article  Google Scholar 

  4. Boyd W, Shaner S, Li L, Forget B, Smith K (2014) The OpenMOC method of characteristics neutral particle transport code. Ann Nucl Energy 68:43–52

    Article  Google Scholar 

  5. Gunow G, Forget B, Smith K (2019) Full core 3D simulation of the BEAVRS benchmark with OpenMOC. Ann Nucl Energy 134:299–304

    Article  Google Scholar 

  6. Nuclear Science & Engineering Department of MIT. Complete documentation on OpenMOC (2023). https://mit-crpg.github.io/OpenMOC. Accessed Apr 2023

  7. Nuclear Science & Engineering Department of MIT (2023) A method of characteristics code (OpenMOC) for nuclear reactor physics calculations. https://github.com/mit-crpg/OpenMOC. Accessed Apr 2023

  8. Nuclear Science & Engineering Department of MIT (2023) The final equations applied in OpenMOC. https://mit-crpg.github.io/OpenMOC/methods/method_of_characteristics.html. Accessed Apr 2023

  9. Nuclear Science & Engineering Department of MIT (2023). The transport sweep algorithm. https://mit-crpg.github.io/OpenMOC/methods/eigenvalue_calculations.html. Accessed Apr 2023

  10. Wu W, Yu Y, Luo Q, Yao D, Li Q, Chai X (2020) Verification of the 3D capability of OpenMOC with the C5G7 3D extension benchmark. Ann Nucl Energy 140:107293

    Article  Google Scholar 

  11. Gunow GA (2018) Full core 3d neutron transport simulation using the method of characteristics with linear sources. Ph.D. thesis, Massachusetts Institute of Technology

  12. Lewis E, Smith M, Tsoulfanidis N, Palmiotti G, Taiwo T, Blomquist R (2001) Benchmark specification for Deterministic 2-D/3-D MOX fuel assembly transport calculations without spatial homogenization (C5G7 MOX). NEA/NSC 280:2001

    Google Scholar 

  13. Liu X, Sun J, Zheng L, Wang S, Liu Y, Wei T (2020) Parallelization and optimization of NSGA-II on Sunway TaihuLight system. IEEE Trans Parallel Distrib Syst 32(4):975–987

    Article  Google Scholar 

  14. Zhu Q, Luo H, Yang C, Ding M, Yin W, Yuan X (2021) Enabling and scaling the HPCG benchmark on the newest generation Sunway supercomputer with 42 million heterogeneous cores. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp 1–13

  15. Lindley B, Hosking J, Smith P, Powney D, Tollit B, Newton T, Perry R, Ware T, Smith P (2017) Current status of the reactor physics code WIMS and recent developments. Ann Nucl Energy 102:148–157

    Article  Google Scholar 

  16. Li Y, Zhang B, He Q, Wang D, Wu H, Cao L, Shen W (2018) Development and verification of PWR-core fuel management calculation code system NECP-Bamboo: part I Bamboo-Lattice. Nucl Eng Des 335:432–440

    Article  Google Scholar 

  17. DRAGON HA (2014) DONJON5, the contribution of École Polytechnique de Montréal to the SALOME platform. In: Proceedings of the 3rd International Conference on Physics and Technology of Reactors and Applications. Tetouan, Morocco

  18. Zhu Y-L, Chen X-W, Hao C, Wang Y-Z, Xu Y-L (2022) Implementation of high-fidelity neutronics and thermal-hydraulic coupling calculations in HNET. Nucl Sci Tech 33(11):1–13

    Article  Google Scholar 

  19. Shaner S, Gunow G, Benoit F, Smith K (2016) Verification of the 3D method of characteristics solver in OpenMOC

  20. Tramm JR, Gunow G, He T, Smith KS, Forget B, Siegel AR (2016) A task-based parallelism and vectorized approach to 3D method of characteristics (MOC) reactor simulation for high performance computing architectures. Comput Phys Commun 202:141–150

    Article  MathSciNet  Google Scholar 

  21. Choi N, Kang J, Joo H-g (2018) Massively parallel method of characteristics neutron transport calculation with anisotropic scattering treatment on GPUs. In: Proceedings of the International Conference on High Performance Computing in Asia-Pacific Region, pp 148–158

  22. Zhang Z, Wang K, Li Q (2013) Accelerating a three-dimensional MOC calculation using GPU with CUDA and two-level GCMFD method. Ann Nucl Energy 62:445–451

    Article  Google Scholar 

  23. Han Y, Jiang X, Wang D (2014) CMFD and GPU acceleration on method of characteristics for hexagonal cores. Nucl Eng Des 280:210–222

    Article  Google Scholar 

  24. Choi S, Lee D (2021) Three-dimensional method of characteristics/diamond-difference transport analysis method in STREAM for whole-core neutron transport calculation. Comput Phys Commun 260:107332

    Article  MathSciNet  MATH  Google Scholar 

  25. Liu X, Guo H, Sun R, Chen Z (2018) The characteristic analysis and exascale scalability research of large scale parallel applications on Sunway TaihuLight supercomputer. J Comput 14(10):2209–2220

    Google Scholar 

  26. Yang C, Xue W, Fu H, You H, Wang X, Ao Y, Liu F, Gan L, Xu P, Wang L, et al (2016) 10M-core scalable fully-implicit solver for nonhydrostatic atmospheric dynamics. In: SC’16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, pp 57–68

  27. Fu H, Yin W, Yang G, Chen X, Liu W (2017) 18.9-Pflops nonlinear earthquake simulation on Sunway TaihuLight: enabling depiction of 18-Hz and 8-meter scenarios. In: The International Conference for High Performance Computing, Networking, Storage and Analysis

  28. Liu Y, Liu X, Li F, Fu H, Yang Y, Song J, Zhao P, Wang Z, Peng D, Chen H, et al (2021) Closing the“ quantum supremacy” gap: achieving real-time simulation of a random quantum circuit using a new sunway supercomputer. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp 1–12

  29. Li M, Lin H, Chen J, Diaz JM, Xiao Q, Lin R, Wang F, Gao GR, An H (2021) swFLOW: a large-scale distributed framework for deep learning on Sunway TaihuLight supercomputer. Inf Sci 570:831–847

    Article  MathSciNet  Google Scholar 

  30. Li M, Yang C, Sun Q, Ma W-J, Cao W-L, Ao Y-L (2019) Enabling highly efficient k-means computations on the SW26010 many-core processor of Sunway TaihuLight. J Comput Sci Technol 34(1):77–93

    Article  Google Scholar 

  31. Ye Y, Song Z, Zhou S, Liu Y, Shu Q, Wang B, Liu W, Qiao F, Wang L (2022) swNEMO_v4. 0: an ocean model based on NEMO4 for the new-generation Sunway supercomputer. Geosci Model Dev 15(14), 5739–5756

  32. Shang H, Shen L, Fan Y, Xu Z, Guo C, Liu J, Zhou W, Ma H, Lin R, Yang Y, et al (2022) Large-scale simulation of quantum computational chemistry on a new sunway supercomputer. arXiv:2207.03711

  33. Li F, Liu X, Liu Y, Zhao P, Yang Y, Shang H, Sun W, Wang Z, Dong E, Chen D (2021) SW_Qsim: a minimize-memory quantum simulator with high-performance on a new sunway supercomputer. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp 1–13

  34. Chen G, Zhao L, Yu W, Ren H, Fu H (2018) A novel acceleration method for DGTD algorithm on sunway TaihuLight. In: 2018 IEEE Asia-Pacific Conference on Antennas and Propagation (APCAP). IEEE, pp 153–154

  35. Zhang Z, Luan Z, Xu C, Gong M, Xu S (2018) Accelerating lattice QCD on sunway many-core processor. In: 2018 IEEE International Conference on Parallel and Distributed Processing with Applications, Ubiquitous Computing and Communications, Big Data and Cloud Computing, Social Computing and Networking, Sustainable Computing and Communications (ISPA/IUCC/BDCloud/SocialCom/SustainCom). IEEE, pp 605–612

  36. Zhao J, Chang Y, Li D, Xia C, Cui H, Zhang K, Feng X (2018) On retargeting the ai programming framework to new hardwares. In: IFIP International Conference on Network and Parallel Computing. Springer, pp 39–51

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant 62002186, 2021 Shandong Youth Innovation Talent Introduction and Education Plan (Parallel Computing Industrial Software Innovation Team Based on Chinese supercomputer), and the unveiling project of Qilu University of Technology (Shandong Academy of Sciences) under Grant 2022JBZ01-01.

Funding

1. National Natural Science Foundation of China under Grant 62002186. 2. 2021 Shandong Youth Innovation Talent Introduction and Education Plan (Parallel Computing Industrial Software Innovation Team Based on Chinese supercomputer). 3. The unveiling project of Qilu University of Technology (Shandong Academy of Sciences) under Grant 2022JBZ01-01.

Author information

Authors and Affiliations

  1. Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China

    Renjiang Chen, Tao Liu, Zhaoyuan Liu, Li Wang, Min Tian, Ying Guo, Jingshan Pan, Xiaoming Wu & Meihong Yang

Authors
  1. Renjiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaoyuan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Li Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Min Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ying Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jingshan Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xiaoming Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Meihong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

RC and TL wrote the main manuscript text and did most of the experiments. ZL provided the original OpenMOC and knowledge of nuclear energy. LW and MT helped to compile part of the programs on Sunway Bluelight II supercomputer. YG and JP provided assistance with the use of the Sunway Bluelight II supercomputer. TL, XW, and MY reviewed the manuscript.

Corresponding author

Correspondence to Tao Liu.

Ethics declarations

Conflict of interest

Not applicable.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, R., Liu, T., Liu, Z. et al. Parallel optimization of method of characteristics based on Sunway Bluelight II supercomputer. J Supercomput 79, 16275–16299 (2023). https://doi.org/10.1007/s11227-023-05313-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05313-0

Keywords

Search

Navigation