Abstract
Clusters have become a crucial technology for providing low-cost high performance computing to scientific applications like weather prediction. In addition, networks like Myrinet, InfiniBand and Quadrics have become popular as an interconnection technology for high performance clusters. The high-bandwidth, low-latency characteristics of these networks make them ideally suited to the demanding characteristics of large scale weather simulations. Additionally, these networks have features like efficient and scalable hardware broadcast, reduce and atomic operations. Some of the features have been integrated into the MPI stack for these networks, allowing the user to exploit them for improved performance. In this paper, we evaluate the communication characteristics of a popular weather simulation code MM5 using InfiniBand. We also investigate how special features of InfiniBand like scalable broadcast can benefit MM5 performance. For some workloads, we see that InfiniBand performs up to 34% better than other interconnects. It also performs better in general than other networks for all workloads.
This research is supported in part by Department of Energy’s Grant #DE-FC02-01ER25506; National Science Foundation’s grants #CCR-0204429 and #CCR-0311542; and equipment donations from Intel and Mellanox
Chapter PDF
Similar content being viewed by others
References
Gropp, W., Lusk, E., Doss, N., Skjellum, A.: A High-Performance, Portable Implementation of the MPI, Message Passing Interface Standard. Technical report (Argonne National Laboratory and Mississippi State University)
Fluent CFD, http://www.fluent.com
LSDYNA , http://www.lstc.com
Grell, G.A., Dudhia, J., Stauffer, D.R.: A Description of the Fifth-Generation Penn State/NCAR Mesoscale Model (MM5). Tech. Rep. NCAR/TN-398+STR, National Center for Atmospheric Research, Boulder, Colarado (1994)
Boden, N.J., Cohen, D., et al.: Myrinet: A Gigabit-per-Second Local Area Network. IEEE Micro, 29–35 (1995)
Infiniband Trade Association , http://www.infinibandta.org
Quadrics Ltd , http://www.quadrics.com
Noronha, R., Abu-Ghazaleh, N.B.: Using Programmable NICs for Time Warp Optimization. In: IPDPS (2002)
MPI over InfiniBand Project , http://nowlab.cis.ohio-state.edu/projects/mpi-iba
Liu, J., Mamidala, A., Panda, D.K.: Fast and Scalable MPI-Level Broadcast using InfiniBand’s Hardware Multicast Support. In: IPDPS (2004)
Myrinet E-cards , http://www.myri.com/myrinet/PCIX/m3f2-pcixe.html
Liu, J., Chandrasekaran, B., Yu, W., Wu, J., Buntinas, D., Kini, S., Wyckoff, P., Panda, D.K.: Micro-Benchmark Performance Comparison of High-Speed Cluster Interconnects. IEEE Micro (2004)
Michalakes, J., Canfield, T., Nanjundiah, R., Hammond, S.: Parallel Implementation, Validation and Performance of MM5. In: Sixth Workshop on the Use of Parallel Processors in Meteorology, European Center for Medium Range Weather Forecasting, Reading, U.K. (1994)
Michalakes, J.: A Runtime System Library for Parallel Finite Difference Models with Nesting. Technical Report ANL/MCS-TM-197 (1997)
Parallel MM5 benchmarks (2004), http://www.mmm.ucar.edu/mm5/mpp/helpdesk/20040304a.html
MM5 Community Model , http://www.mmm.ucar.edu/mm5/
mpiP MPI Profiling Tool , http://www.llnl.gov/CASC/mpip
Brightwell, R., Doerfler, D., Underwood, K.D.: A Comparison of 4X InfiniBand and Quadrics Elan-4 Technologies.In: IEEE Conference on Cluster Computing (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Noronha, R., Panda, D.K. (2005). Performance Evaluation of MM5 on Clusters with Modern Interconnects: Scalability and Impact. In: Cunha, J.C., Medeiros, P.D. (eds) Euro-Par 2005 Parallel Processing. Euro-Par 2005. Lecture Notes in Computer Science, vol 3648. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11549468_18
Download citation
DOI: https://doi.org/10.1007/11549468_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28700-1
Online ISBN: 978-3-540-31925-2
eBook Packages: Computer ScienceComputer Science (R0)