skip to main content
10.1145/2488551.2488567acmotherconferencesArticle/Chapter ViewAbstractPublication PageseurompiConference Proceedingsconference-collections
research-article

Optimized process placement for collective I/O operations

Published: 15 September 2013 Publication History

Abstract

Mapping of MPI processes to the available resources is an increasingly complex but important task on modern parallel systems. This paper presents a new approach to optimize the process placement of a parallel application based on its I/O access pattern. The paper introduces the SetMatch process mapping algorithm, which significantly reduces the cost of the communication occurring in collective I/O operations. The effectiveness of the approach has been evaluated for multiple scenarios on a PVFS2 file system. Our results demonstrate significant improvements in the communication time of collective I/O operations as well as improvements in the overall application execution time with our mapping strategy. The generalized SetMatch algorithm was the only mapping strategy that was able to provide adequate performance for all scenarios used in this paper.

References

[1]
A. Bhatele, L. V. Kale, and S. Kumar. Dynamic topology aware load balancing algorithms for molecular dynamics applications. In Proceedings of the 23rd international conference on Supercomputing, ICS '09, pages 110--116, New York, NY, USA, 2009. ACM.
[2]
M. Chaarawi, S. Chandok, and E. Gabriel. Performance Evaluation of Collective Write Algorithms in MPI I/O. In Proceedings of the International Conference on Computational Science (ICCS), volume 5544, pages 185--194, Baton Rouge, USA, 2009.
[3]
M. Chaarawi, E. Gabriel, R. Keller, R. L. Graham, G. Bosilca, and J. J. Dongarra. OMPIO: A Modular Software Architecture for MPI I/O. In Y. Cotronis, A. Danalis, D. Nikolopoulos, and J. Dongarra, editors, Recent Advances in Message Passing Interface, pages 90--98, Santorini, Greece, September 2011. Springer.
[4]
H. Chen, W. Chen, J. Huang, B. Robert, and H. Kuhn. Mpipp: an automatic profile-guided parallel process placement toolset for smp clusters and multiclusters. In Proceedings of the 20th annual international conference on Supercomputing, ICS '06, pages 353--360, New York, NY, USA, 2006. ACM.
[5]
J. M. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. SIGARCH Comput. Archit. News, 21(5):31--38, 1993.
[6]
E. Gabriel, G. E. Fagg, G. Bosilca, T. Angskun, J. J. Dongarra, J. M. Squyres, V. Sahay, P. Kambadur, B. Barrett, A. Lumsdaine, R. H. Castain, D. J. Daniel, R. L. Graham, and T. S. Woodall. Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation. In Proceedings of the 11th European PVM/MPI Users' Group Meeting, pages 97--104, Budapest, Hungary, 2004.
[7]
T. Hoefler and M. Snir. Generic Topology Mapping Strategies for Large-scale Parallel Architectures. In Proceedings of the 2011 ACM International Conference on Supercomputing (ICS'11), pages 75--85. ACM, Jun. 2011.
[8]
E. Jeannot and G. Mercier. Near-optimal placement of MPI processes on hierarchical NUMA architectures. In Proceedings of the 16th international Euro-Par conference on Parallel processing: Part II, pages 199--210. Springer-Verlag, 2010.
[9]
OSU-micro benchmark homepage. http://mvapich.cse.ohio-state.edu/benchmarks/, 2002.
[10]
R. Ross. Parallel I/O Benchmarking Consortium. http://www-unix.mcs.anl.gov/ross/pio-benchmark.html.
[11]
J. Traff. Implementing the MPI process topology mechanism. In Supercomputing, ACM/IEEE 2002 Conference, pages 28--28, 2002.
[12]
P. Wong and R. F. V. der Wijngaart. NAS Parallel Benchmarks I/O Version 2.4. Technical Report. NAS-03-002, Computer Sciences Corporation, NASA Advanced Supercomputing (NAS) Division.
[13]
H. Yu, I.-H. Chung, and J. Moreira. Topology mapping for blue gene/l supercomputer. In Proceedings of the 2006 ACM/IEEE conference on Supercomputing, SC '06, New York, NY, USA, 2006. ACM.

Cited By

View all
  • (2017)TAPIOCA: An I/O Library for Optimized Topology-Aware Data Aggregation on Large-Scale Supercomputers2017 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER.2017.80(70-80)Online publication date: Sep-2017
  • (2016)Topology-aware data aggregation for intensive I/O on large-scale supercomputersProceedings of the First Workshop on Optimization of Communication in HPC10.5555/3018058.3018066(73-81)Online publication date: 13-Nov-2016
  • (2016)Topology-Aware Data Aggregation for Intensive I/O on Large-Scale Supercomputers2016 First International Workshop on Communication Optimizations in HPC (COMHPC)10.1109/COMHPC.2016.013(73-81)Online publication date: Nov-2016

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
EuroMPI '13: Proceedings of the 20th European MPI Users' Group Meeting
September 2013
289 pages
ISBN:9781450319034
DOI:10.1145/2488551
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • ARCOS: Computer Architecture and Technology Area, Universidad Carlos III de Madrid

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 September 2013

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Conference

EuroMPI '13
Sponsor:
  • ARCOS
EuroMPI '13: 20th European MPI Users's Group Meeting
September 15 - 18, 2013
Madrid, Spain

Acceptance Rates

EuroMPI '13 Paper Acceptance Rate 22 of 47 submissions, 47%;
Overall Acceptance Rate 66 of 139 submissions, 47%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 12 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2017)TAPIOCA: An I/O Library for Optimized Topology-Aware Data Aggregation on Large-Scale Supercomputers2017 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER.2017.80(70-80)Online publication date: Sep-2017
  • (2016)Topology-aware data aggregation for intensive I/O on large-scale supercomputersProceedings of the First Workshop on Optimization of Communication in HPC10.5555/3018058.3018066(73-81)Online publication date: 13-Nov-2016
  • (2016)Topology-Aware Data Aggregation for Intensive I/O on Large-Scale Supercomputers2016 First International Workshop on Communication Optimizations in HPC (COMHPC)10.1109/COMHPC.2016.013(73-81)Online publication date: Nov-2016

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media