Abstract
This paper reports on experiences gained and practices adopted when using the latest features of OpenMP to port a variety of HPC applications and mini-apps based on different computational motifs (BerkeleyGW, WDMApp/XGC, GAMESS, GESTS, and GridMini) to accelerator-based, leadership-class, high-performance supercomputer systems at the Department of Energy. As recent enhancements to OpenMP become available in implementations, there is a need to share the results of experimentation with them in order to better understand their behavior in practice, to identify pitfalls, and to learn how they can be effectively deployed in scientific codes. Additionally, we identify best practices from these experiences that we can share with the rest of the OpenMP community.
Supported by Exascale Computing Project (ECP) OpenMP Hackathon hosted by SOLLVE and NERSC [25].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
The mini-apps are design to quickly and systematically assess the features of various programming models and associated libraries.
- 3.
No such differentiation is normally needed for CPU/multicore builds.
References
BerkeleyGW. http://www.berkeleygw.org. Accessed 25 July 2021, 06:48:38
BerkeleyGW CUDA version. https://gitlab.com/NESAP/berkeleygw/berkeleygw-cuda. Accessed 25 July 2021, 06:48:38
BerkeleyGW development mainline. https://github.com/BerkeleyGW. Accessed 25 July 2021, 06:48:38
BerkeleyGW kernels and miniapps. https://gitlab.com/NESAP/berkeleygw/berkeleygw-kernels. Accessed 25 July 2021, 06:48:38
Kokkos Pull Request #3808. https://github.com/kokkos/kokkos/pull/3808. Accessed 25 July 2021, 06:48:38
OLCF Summit. https://www.olcf.ornl.gov/summit/. Accessed 25 July 2021, 06:48:38
WDMApp. https://www.exascaleproject.org/research-project/wdmapp. Accessed 25 July 2021, 06:48:38
XGC1. https://hbps.pppl.gov/computing/xgc-1. Accessed 25 July 2021, 06:48:38
Daley, C., Ahmed, H., Williams, S., Wright, N.: A case study of porting HPGMG from CUDA to OpenMP target offload. In: Milfeld, K., de Supinski, B.R., Koesterke, L., Klinkenberg, J. (eds.) IWOMP 2020. LNCS, vol. 12295, pp. 37–51. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58144-2_3
Davis, J.H., Daley, C., Pophale, S., Huber, T., Chandrasekaran, S., Wright, N.J.: Performance assessment of OpenMP compilers targeting NVIDIA V100 GPUs. In: Seventh Workshop on Accelerator Programming Using Directives (WACCPD-2020) (2020)
Del Ben, M.: BerkeleyGW Si214 Benchmarks (2021)
Del Ben, M., Yang, C., Li, Z., Jornada, F.H.d., Louie, S.G., Deslippe, J.: Accelerating large-scale excited-state \(GW\) calculations on leadership HPC systems. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. SC ’20. IEEE Press (2020)
Deslippe, J., Samsonidze, G., Strubbe, D.A., Jain, M., Cohen, M.L., Louie, S.G.: Berkeleygw: a massively parallel computer package for the calculation of the quasiparticle and optical properties of materials and nanostructures. Comput. Phys. Commun. 183(6), 1269–1289 (2012)
Edwards, H.C., Trott, C.R., Sunderland, D.: Kokkos: enabling manycore performance portability through polymorphic memory access patterns. J. Parallel Distrib. Comput. 74(12), 3202–3216 (2014). Domain-Specific Languages and High-Level Frameworks for High-Performance Computing
Giannozzi, P.: Quantum espresso: a modular and open-source software project for quantum simulations of materials. J. Phys. Cond. Matter 21(39), 395502 (2009)
Gonze, X., et al.: ABINIT: First-principles approach to material and nanosystem properties. Comput. Phys. Commun. 180(12), 2582–2615 (2009)
Hager, R., Yoon, E., Ku, S., D’Azevedo, E., Worley, P., Chang, C.: A fully non-linear multi-species Fokker-Planck-Landau collision operator for simulation of fusion plasma. J. Comput. Phys. 315, 644–660 (2016)
Hybertsen, M., Louie, S.G.: First-principles theory of quasiparticles: calculation of band gaps in semiconductors and insulators. Phys. Rev. Lett. 55(13), 1418 (1985)
Hybertsen, M.S., Louie, S.G.: Electron correlation in semiconductors and insulators: band gaps and quasiparticle energies. Phys. Rev. B 34(8), 5390 (1986)
Ku, S., et al.: A fast low-to-high confinement mode bifurcation dynamics in the boundary-plasma gyrokinetic code XGC1. Phys. Plasmas 25(5), 056107 (2018)
Kwack, J., et al.: OpenMP roadmap for accelerators across DOE Pre-Exascale/Exascale machines. In: Presentation at the ECP 2021 Annual Meeting on Apr 15 2021 (2021)
OpenMP.org: OpenMP Application Programming Interface version 4.5 (2015)
Rohlfing, M., Louie, S.G.: Electron-hole excitations and optical spectra from first principles. Phys. Rev. B 62, 4927–4944 (2000)
Soler, J.M., et al.: The SIESTA method for ab-initio order-N materials simulation. J. Phys.: Conden. Matter 14(11), 2745–2779 (2002)
SOLLVE and NERSC: January 2021 ECP OpenMP Hackathon by SOLLVE and NERSC (2021 [Online]), the event happened on 22, 27, 28, 29 Jan 2021. https://sites.google.com/view/ecpomphackjan2021. Accessed 7 Apr 2021
Tancogne-Dejean, N., et al.: Octopus, a computational framework for exploring light-driven phenomena and quantum dynamics in extended and finite systems. J. Chem. Phys. 152(12), 124119 (2020)
Yang, C.: 8 Steps to 3.7 TFLOP/s on NVIDIA V100 GPU: Roofline Analysis and Other Tricks (2020). https://arxiv.org/abs/2008.11326
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chapman, B. et al. (2021). Outcomes of OpenMP Hackathon: OpenMP Application Experiences with the Offloading Model (Part I). In: McIntosh-Smith, S., de Supinski, B.R., Klinkenberg, J. (eds) OpenMP: Enabling Massive Node-Level Parallelism. IWOMP 2021. Lecture Notes in Computer Science(), vol 12870. Springer, Cham. https://doi.org/10.1007/978-3-030-85262-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-85262-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85261-0
Online ISBN: 978-3-030-85262-7
eBook Packages: Computer ScienceComputer Science (R0)