Skip to main content
SpringerLink
Log in
Book cover

International Workshop on OpenMP

IWOMP 2019: OpenMP: Conquering the Full Hardware Spectrum pp 21–35Cite as

Score-P and OMPT: Navigating the Perils of Callback-Driven Parallel Runtime Introspection

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 11718))

Abstract

Event-based performance analysis aims at modeling the behavior of parallel applications through a series of state transitions during execution. Different approaches to obtain such transition points for OpenMP programs include source-level instrumentation (e.g., OPARI) and callback-driven runtime support (e.g., OMPT).

In this paper, we revisit a previous evaluation and comparison of OPARI and an LLVM OMPT implementation—now updated to the OpenMP 5.0 specification—in the context of Score-P. We describe the challenges faced while trying to use OMPT as a drop-in replacement for the existing instrumentation-based approach and the changes in event order that could not be avoided. Furthermore, we provide details on Score-P measurements using OPARI and OMPT as event sources with the EPCC and SPEC OpenMP benchmark suites.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    In addition, the master thread reuses the encountering thread’s location.

  2. 2.

    See, for example, Example_barrier_regions.1.c from the OpenMP 4.5 Examples [23] where the implicit barrier of the inner parallel region is omitted.

  3. 3.

    To convert addresses into file names and line numbers, we rely on the Binary File Descriptor library (BFD) [1] and debug symbols in the binary.

  4. 4.

    We used OMP_PROC_BIND=close and OMP_PLACES={0}:12 for all measurements.

  5. 5.

    The addr2line lookup is done only once per address and is negligible.

References

  1. GNU Binutils. https://sourceware.org/binutils/

  2. LLVM runtime with experimental changes for OMPT. https://github.com/OpenMPToolsInterface/LLVM-openmp/commits/tool_finalization_tr7, branch tool\(\_\)finalization\(\_\)tr7, commit dcf2962eb6d92d82e74bd374f27e6ef836a5e2b3

  3. Support for the OpenMP language in LLVM. http://openmp.llvm.org

  4. Benedict, S., Petkov, V., Gerndt, M.: Periscope: an online-based distributed performance analysis tool. In: Müller, M.S., Resch, M.M., Schulz, A., Nagel, W.E. (eds.) Tools for High Performance Computing 2009, pp. 1–16. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-11261-4_1

    Chapter  Google Scholar 

  5. Bull, J.M., Reid, F., McDonnell, N.: A microbenchmark suite for OpenMP tasks. In: Chapman, B.M., Massaioli, F., Müller, M.S., Rorro, M. (eds.) IWOMP 2012. LNCS, vol. 7312, pp. 271–274. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30961-8_24

    Chapter  Google Scholar 

  6. Eichenberger, A.E., et al.: OMPT: an OpenMP tools application programming interface for performance analysis. In: Rendell, A.P., Chapman, B.M., Müller, M.S. (eds.) IWOMP 2013. LNCS, vol. 8122, pp. 171–185. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40698-0_13

    Chapter  Google Scholar 

  7. Eschweiler, D., Wagner, M., Geimer, M., Knüpfer, A., Nagel, W.E., Wolf, F.: Open trace format 2 - the next generation of scalable trace formats and support libraries. In: Proceedings of the Internatioanl Conference on Parallel Computing (ParCo), Ghent, Belgium, 30 August–2 September 2011. Advances in Parallel Computing, vol. 22, pp. 481–490. IOS Press (2012). https://doi.org/10.3233/978-1-61499-041-3-481

  8. Feld, C., Convent, S., Hermanns, M.A., Protze, J., Geimer, M.: [Reproducibility] Score-P and OMPT: navigating the perils of callback-driven parallel runtime introspection, June 2019. https://doi.org/10.5281/zenodo.3251871

  9. Fürlinger, K., Gerndt, M.: ompP: A profiling tool for OpenMP. In: Mueller, M.S., Chapman, B.M., de Supinski, B.R., Malony, A.D., Voss, M. (eds.) IWOMP -2005. LNCS, vol. 4315, pp. 15–23. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68555-5_2

    Chapter  Google Scholar 

  10. Geimer, M., Wolf, F., Wylie, B.J.N., Ábrahám, E., Becker, D., Mohr, B.: The Scalasca performance toolset architecture. Concurr. Comput. Pract. Exp. 22(6), 702–719 (2010). https://doi.org/10.1002/cpe.1556

    Article  Google Scholar 

  11. Huck, K.A., Malony, A.D., Shende, S., Jacobsen, D.W.: Integrated measurement for cross-platform OpenMP performance analysis. In: DeRose, L., de Supinski, B.R., Olivier, S.L., Chapman, B.M., Müller, M.S. (eds.) IWOMP 2014. LNCS, vol. 8766, pp. 146–160. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11454-5_11

    Chapter  Google Scholar 

  12. Itzkowitz, M., Mazurov, O., Copty, N., Lin, Y.: An OpenMP runtime API for profiling. White paper (2002). http://www.compunity.org/futures/omp-api.html

  13. Jülich Supercomputing Centre: JURECA: modular supercomputer at Jülich supercomputing centre. J. Large-Scale Res. Facil. 4(A132) (2018). https://doi.org/10.17815/jlsrf-4-121-1

  14. Knüpfer, A., et al.: The Vampir performance analysis tool-set. In: Resch, M., Keller, R., Himmler, V., Krammer, B., Schulz, A. (eds.) Tools for High Performance Computing, pp. 139–155. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68564-7_9

    Chapter  Google Scholar 

  15. Knüpfer, A., et al.: Score-P: a joint performance measurement run-time infrastructure for Periscope, Scalasca, TAU, and Vampir. In: Brunst, H., Müller, M., Nagel, W., Resch, M. (eds.) Tools for High Performance Computing 2011, pp. 79–91. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31476-6_7

    Chapter  Google Scholar 

  16. Liao, C., Hernandez, O., Chapman, B., Chen, W., Zheng, W.: OpenUH: an optimizing, portable OpenMP compiler. Concurr. Comput. Pract. Exp. 19(18), 2317–2332 (2007)

    Article  Google Scholar 

  17. Lorenz, D., Dietrich, R., Tschüter, R., Wolf, F.: A Comparison between OPARI2 and the OpenMP tools interface in the context of Score-P. In: DeRose, L., de Supinski, B.R., Olivier, S.L., Chapman, B.M., Müller, M.S. (eds.) IWOMP 2014. LNCS, vol. 8766, pp. 161–172. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11454-5_12

    Chapter  Google Scholar 

  18. Lorenz, D., Mohr, B., Rössel, C., Schmidl, D., Wolf, F.: How to reconcile event-based performance analysis with tasking in OpenMP. In: Sato, M., Hanawa, T., Müller, M.S., Chapman, B.M., de Supinski, B.R. (eds.) IWOMP 2010. LNCS, vol. 6132, pp. 109–121. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13217-9_9

    Chapter  Google Scholar 

  19. Lührs, S., Rohe, D., Schnurpfeil, A., Thust, K., Frings, W.: Flexible and generic workflow management. In: Parallel Computing: On the Road to Exascale. Advances in Parallel Computing, vol. 27, pp. 431–438. International Conference on Parallel Computing 2015, Edinburgh, UK, 1–4 September 2015. IOS Press, Amsterdam, September 2016. https://doi.org/10.3233/978-1-61499-621-7-431. https://www.fz-juelich.de/jsc/jube/

  20. Mohr, B., Malony, A., Shende, S., Wolf, F.: Design and prototype of a performance tool interface for OpenMP. J. Supercomput. 23, 105–128 (2002). https://doi.org/10.1023/A:1015741304337

    Article  MATH  Google Scholar 

  21. Mohr, B., et al.: A performance monitoring interface for OpenMP. In: Proceedings of the 4th European Workshop on OpenMP (EWOMP 2002), Rome, Italy, September 2002

    Google Scholar 

  22. Müller, M., et al.: SPEC OMP2012 an application benchmark suite for parallel systems using OpenMP. In: Proceedings of the 8th International Conference on OpenMP in a Heterogeneous World, pp. 223–236, June 2012. https://doi.org/10.1007/978-3-642-30961-8_17

    Chapter  Google Scholar 

  23. OpenMP Architecture Review Board: OpenMP Application Programming Interface - Examples - Version 4.5.0. http://www.openmp.org/wp-content/uploads/openmp-examples-4.5.0.pdf

  24. OpenMP Architecture Review Board: TR4: OpenMP Version 5.0 Preview 1. Specification, November 2016. http://www.openmp.org/wp-content/uploads/openmp-tr4.pdf

  25. OpenMP Architecture Review Board: TR6: OpenMP Version 5.0 Preview 2. Specification, November 2017. http://www.openmp.org/wp-content/uploads/openmp-TR6.pdf

  26. OpenMP Architecture Review Board: OpenMP Application Program Interface Version 5.0. Specification, November 2018. https://www.openmp.org/wp-content/uploads/OpenMP-API-Specification-5.0.pdf

  27. Protze, J., Hahnfeld, J., Ahn, D.H., Schulz, M., Müller, M.S.: OpenMP tools interface: synchronization information for data race detection. In: de Supinski, B.R., Olivier, S.L., Terboven, C., Chapman, B.M., Müller, M.S. (eds.) IWOMP 2017. LNCS, vol. 10468, pp. 249–265. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65578-9_17

    Chapter  Google Scholar 

  28. Saviankou, P., Knobloch, M., Visser, A., Mohr, B.: Cube v4: from performance report explorer to performance analysis tool. Procedia Comput. Sci. 51, 1343–1352 (2015). https://doi.org/10.1016/j.procs.2015.05.320

    Article  Google Scholar 

  29. Schöne, R., Tschüter, R., Ilsche, T., Schuchart, J., Hackenberg, D., Nagel, W.E.: Extending the functionality of Score-P through plugins: interfaces and use cases. In: Niethammer, C., Gracia, J., Hilbrich, T., Knüpfer, A., Resch, M.M., Nagel, W.E. (eds.) Tools for High Performance Computing 2016, pp. 59–82. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56702-0_4

    Chapter  Google Scholar 

  30. Shende, S.S., Malony, A.D.: The Tau parallel performance system. Int. J. High Perform. Comput. Appl. 20(2), 287–311 (2006). https://doi.org/10.1177/1094342006064482

    Article  Google Scholar 

  31. Zhukov, I., Feld, C., Geimer, M., Knobloch, M., Mohr, B., Saviankou, P.: Scalasca v2: back to the future. In: Niethammer, C., Gracia, J., Knüpfer, A., Resch, M., Nagel, W. (eds.) Tools for High Performance Computing, pp. 1–24. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16012-2_1

    Chapter  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the computing time granted through JARA-HPC on the supercomputer JURECA [13] at Forschungszentrum Jülich. Part of this work was performed under the POP2 project and has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement 824080.

Author information

Authors and Affiliations

  1. Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany

    Christian Feld, Marc-André Hermanns, Markus Geimer & Bernd Mohr

  2. JARA-HPC, Jülich, Germany

    Marc-André Hermanns

  3. IT Center, RWTH Aachen University, Aachen, Germany

    Simon Convent & Joachim Protze

Authors
  1. Christian Feld
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Convent
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marc-André Hermanns
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joachim Protze
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Markus Geimer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bernd Mohr
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Feld .

Editor information

Editors and Affiliations

  1. University of Auckland, Auckland, New Zealand

    Xing Fan

  2. Lawrence Livermore National Laboratory, Livermore, CA, USA

    Bronis R. de Supinski

  3. University of Auckland, Auckland, New Zealand

    Oliver Sinnen

  4. University of Auckland, Auckland, New Zealand

    Nasser Giacaman

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Feld, C., Convent, S., Hermanns, MA., Protze, J., Geimer, M., Mohr, B. (2019). Score-P and OMPT: Navigating the Perils of Callback-Driven Parallel Runtime Introspection. In: Fan, X., de Supinski, B., Sinnen, O., Giacaman, N. (eds) OpenMP: Conquering the Full Hardware Spectrum. IWOMP 2019. Lecture Notes in Computer Science(), vol 11718. Springer, Cham. https://doi.org/10.1007/978-3-030-28596-8_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-28596-8_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-28595-1

  • Online ISBN: 978-3-030-28596-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation