research-article

Public Access

Graph500 on OpenSHMEM: Using A Practical Survey of Past Work to Motivate Novel Algorithmic Developments

Authors:

Howard Pritchard,

Zoran Budimlić,

Vivek SarkarAuthors Info & Claims

PAW17: Proceedings of the Second Annual PGAS Applications Workshop

Article No.: 2, Pages 1 - 8

https://doi.org/10.1145/3144779.3144781

Published: 12 November 2017 Publication History

Abstract

Graph500 is an open specification of a graph-based benchmark for high-performance computing (HPC). The core computational kernel of Graph500 is a breadth-first search of an undirected graph. Unlike many other HPC benchmarks, Graph500 is therefore characterized by heavily irregular and fine-grain computation, memory accesses, and network communication. Therefore, it can serve as a more realistic stress test of modern HPC hardware, software, and algorithmic techniques than other benchmarking efforts.

On the other hand, OpenSHMEM is an open, PGAS, and SPMD specification of a communication model for communicating across large numbers of processing elements. OpenSHMEM explicitly focuses on applications characterized by fine-grain communication, of which Graph500 is one example.

Therefore, there is a natural synergy between the communication patterns of Graph500 and the capabilities of OpenSHMEM. In this work we explore that synergy by developing several novel implementations of Graph500 on various OpenSHMEM implementations. We contribute a review of the state-of-the-art in distributed Graph500 implementations, as well as a performance and programmability comparison between the state-of-the-art and our own OpenSHMEM-based implementations. Our results demonstrate improved scaling of Graph500's BFS kernel out to 1,024 nodes of the Edison supercomputer, achieving ~2.5x performance improvement relative to the highest performing reference implementation at that scale.

References

[1]

CityHash. https://github.com/google/cityhash.

[2]

MurmurHash. https://github.com/aappleby/smhasher.

[3]

Openshmem application programming interface version 1.3. http://openshmem.org/site/sites/default/site_files/OpenSHMEM-1.3.pdf.

[4]

Openshmem graph500 implementations. https://github.com/habanero-rice/hclib/tree/resourceworkers/test/performance-regression/full-apps/graph500-2.1.4/oshmem.

[5]

Top 500. https://www.top500.org/.

[6]

M. Adams. Hpgmg 1.0: a benchmark for ranking high performance computing systems. 2014.

[7]

F. Checconi and F. Petrini. Massive data analytics: the graph 500 on ibm blue gene/q. IBM Journal of Research and Development, 57(1/2):10--1, 2013.

Digital Library

[8]

J. Dinan and M. Flajslik. Contexts: A Mechanism for High Throughput Communication in OpenSHMEM. In Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models, pages 10:1--10:9, New York, NY, USA, 2014. ACM.

Digital Library

[9]

E. F. DAzevedo and N. Imam. Graph 500 in openshmem. In Workshop on OpenSHMEM and Related Technologies, pages 154--163. Springer, 2014.

Digital Library

[10]

M. Grossman. Mpi allgather microbenchmark. https://github.com/habanero-rice/hclib/blob/resource_workers/test/performance-regression/cpu-only/graph500-2.1.4/oshmem/mpi_allgather.c.

[11]

M. Grossman. Openshmem allgather microbenchmark. https://github.com/habanero-rice/hclib/blob/resource_workers/test/performance-regression/cpu-only/graph500-2.1.4/oshmem/oshmem_allgather.c.

[12]

J. Jose, S. Potluri, K. Tomko, and D. K. Panda. Designing scalable graph500 benchmark with hybrid mpi+ openshmem programming models. In International Supercomputing Conference, pages 109--124. Springer, 2013.

[13]

Max Grossman, Joseph Doyle, James Dinan, Howard Pritchard, Kayla Seager, Vivek Sarkar. Implementation and Evaluation of OpenSHMEM Contexts Using OFI Libfabric. In OpenSHMEM Workshop, 2017.

[14]

R. C. Murphy, K. B. Wheeler, B. W. Barrett, and J. A. Ang. Introducing the graph 500. Cray Users Group (CUG), 2010.

[15]

National Energy Research Scientific Computing Center. Edison. http://www.nersc.gov/users/computational-systems/edison/.

Cited By

Rolinger TSussman A(2024)Adaptive Prefetching for Fine-grain Communication in PGAS Programs2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS57955.2024.00071(740-751)Online publication date: 27-May-2024
https://doi.org/10.1109/IPDPS57955.2024.00071
Naughton TAderholdt FBaker MPophale SGorentla Venkata MImam NKindler HWeber FKühne OHalder G(2019)Oak Ridge OpenSHMEM Benchmark SuiteWissenschaftlich Arbeiten in Geographie und Raumwissenschaften10.1007/978-3-030-04918-8_13(202-216)Online publication date: 19-Mar-2019
https://doi.org/10.1007/978-3-030-04918-8_13

Recommendations

Highly scalable graph search for the Graph500 benchmark
HPDC '12: Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing

Graph500 is a new benchmark to rank supercomputers with a large-scale graph search problem. We found that the provided reference implementations are not scalable in a large distributed environment. We devised an optimized method based on 2D partitioning ...
CUDA-Aware OpenSHMEM

GPUDirect RDMA (GDR) brings the high-performance communication capabilities of RDMA networks like InfiniBand (IB) to GPUs. It enables IB network adapters to directly write/read data to/from GPU memory. Partitioned Global Address Space (PGAS) programming ...
Exploiting GPUDirect RDMA in Designing High Performance OpenSHMEM for NVIDIA GPU Clusters
CLUSTER '15: Proceedings of the 2015 IEEE International Conference on Cluster Computing

GPUDirect RDMA (GDR) brings the high-performance communication capabilities of RDMA networks like InfiniBand (IB) to GPUs (referred to as "Device"). It enables IB network adapters to directly write/read data to/from GPU memory. Partitioned Global ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PAW17: Proceedings of the Second Annual PGAS Applications Workshop

November 2017

39 pages

ISBN:9781450351232

DOI:10.1145/3144779

Copyright © 2017 ACM.

© 2017 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the United States Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Los Alamos National Laboratory

Conference

SC '17

Sponsor:

SIGHPC

SC '17: The International Conference for High Performance Computing, Networking, Storage and Analysis

November 12 - 17, 2017

CO, Denver, USA

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
397
Total Downloads

Downloads (Last 12 months)104
Downloads (Last 6 weeks)13

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Rolinger TSussman A(2024)Adaptive Prefetching for Fine-grain Communication in PGAS Programs2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS57955.2024.00071(740-751)Online publication date: 27-May-2024
https://doi.org/10.1109/IPDPS57955.2024.00071
Naughton TAderholdt FBaker MPophale SGorentla Venkata MImam NKindler HWeber FKühne OHalder G(2019)Oak Ridge OpenSHMEM Benchmark SuiteWissenschaftlich Arbeiten in Geographie und Raumwissenschaften10.1007/978-3-030-04918-8_13(202-216)Online publication date: 19-Mar-2019
https://doi.org/10.1007/978-3-030-04918-8_13

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten