Abstract
In this paper, we document the workflow of our practice to port a PETSc application with OpenACC to a supercomputer, Titan, at Oak Ridge National Laboratory. Our experience shows a few lines of code modifications with OpenACC directives can give us a speedup of 1.34x in a PETSc-based Poisson solver (conjugate gradient method with algebraic multigrid preconditioner). This demonstrates the feasibility of enabling GPU capability in PETSc with OpenACC. We hope our work can serve as a reference to those who are interested in porting their legacy PETSc applications to modern heterogeneous platforms.
This manuscript has been co-authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Balay, S., Abhyankar, S., Adams, M.F., Brown, J., Brune, P., Buschelman, K., Dalcin, L., Eijkhout, V., Gropp, W.D., Kaushik, D., Knepley, M.G., McInnes, L.C., Rupp, K., Smith, B.F., Zampini, S., Zhang, H., Zhang, H.: PETSc Web page (2016). http://www.mcs.anl.gov/petsc
PETSc, Applications and Publications. https://www.mcs.anl.gov/petsc/publications/index.html. Accessed 31 Aug 2017
Minden, V., Smith, B.F., Knepley, M.G.: Preliminary implementation of PETSc using GPUs. In: Proceedings of the 2010 International Workshop of GPU Solutions to Multiscale Problems in Science and Engineering (2010)
PETSc. https://bitbucket.org/petsc/petsc. Accessed 31 Aug 2017
OLCF, Titan Cray XK7. https://www.olcf.ornl.gov/computing-resources/titan-cray-xk7/. Accessed 31 Aug 2017
Chuang, P.-Y., Barba, L.A.: Using AmgX to Accelerate PETSc-Based CFD Codes, figshare (2017). https://doi.org/10.6084/m9.figshare.5018774.v1
Chuang, P.-Y., Barba, L.A.: AmgXWrapper: An interface between PETSc and the NVIDIA AmgX library. J. Open Source Softw. 2(16), 280 (2017)
Larkin, J.: Chapter 2: Profile-guided development with OpenACC. In: Farber, R. (ed.) Parallel Programing with OpenACC. Elsevier (2017)
Kraus, J.: Multi-GPU Programming with MPI. In: GTC 2017, San Jose (2017)
OLCF, PETSc-OpenACC. https://github.com/olcf/PETSC-OpenACC. Accessed 31 Aug 2017
Knüpfer, A., et al.: Score-P: a joint performance measurement run-time infrastructure for periscope, scalasca, TAU, and vampir. In: Brunst, H., Müller, M., Nagel, W., Resch, M. (eds.) Tools for High Performance Computing 2011. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31476-6_7
NVIDIA, Profiler User’s Guide. http://docs.nvidia.com/cuda/profiler-users-guide. Accessed 24 Aug 2017
Mesnard, O., Barba, L.A.: Reproducible and replicable computational fluid dynamics: it’s harder than you think. Comput. Sci. Eng. 19(4), 44–55 (2017)
Acknowledgement
This research used resources of the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory, which is supported by the Office of Science of the Department of Energy under Contract DE-AC05-00OR22725.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Chuang, PY., Foertter, F.S. (2018). An Example of Porting PETSc Applications to Heterogeneous Platforms with OpenACC. In: Chandrasekaran, S., Juckeland, G. (eds) Accelerator Programming Using Directives. WACCPD 2017. Lecture Notes in Computer Science(), vol 10732. Springer, Cham. https://doi.org/10.1007/978-3-319-74896-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-74896-2_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-74895-5
Online ISBN: 978-3-319-74896-2
eBook Packages: Computer ScienceComputer Science (R0)