Abstract
Porting on grids complex MPI applications involving collective communications requires significant program modification, usually dedicated to a single grid structure. The difficulty comes from the mismatch between programs organizations and grid structures: 1) large grids are hierarchical structures aggregating parallel machines through an interconnection network, decided at runtime and 2) the MPI standard does not currently provide any specific information for topology-aware applications, so almost all MPI applications have been developed following a non-hierarchical and non-flexible vision. In this paper, we propose a generic programming method and a modification of the MPI runtime environment to make MPI applications topology aware. In contrary to previous approaches, topology requirements for the application are given to the grid scheduling system, which exposes the compatible allocated topology to the application.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Atanassov, E.I., Gurov, T.V., Karaivanova, A., Nedjalkov, M.: Monte carlo grid application for electron transport. In: Alexandrov, V.N., van Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds.) ICCS 2006. LNCS, vol. 3993, pp. 616–623. Springer, Heidelberg (2006)
Barnett, M., Gupta, S., Payne, D.G., Shuler, L., van de Geijn, R., Watts, J.: Building a high-performance collective communication library. In: Proc. of SC 1994, pp. 107–116. IEEE, Los Alamitos (1994)
Boutammine, S., Millot, D., Parrot, C.: An adaptive scheduling method for grid computing. In: Nagel, W.E., Walter, W.V., Lehner, W. (eds.) Euro-Par 2006. LNCS, vol. 4128, pp. 188–197. Springer, Heidelberg (2006)
Branford, S., Sahin, C., Thandavan, A., Weihrauch, C., Alexandrov, V.N., Dimov, I.T.: Monte carlo methods for matrix computations on the grid. Future Gener. Comput. Syst. 24(6), 605–612 (2008)
Cappello, F., Caron, E., Dayde, M., et al.: Grid’5000: A large scale and highly reconfigurable grid experimental testbed. In: Proc. The 6th Intl. Workshop on Grid Computing, pp. 99–106 (2005)
Cappello, F., Fraigniaud, P., Mans, B., Rosenberg, A.L.: HiHCoHP: Toward a realistic communication model for hierarchical hyperclusters of heterogeneous processors. In: Proc. of IPDPS. IEEE, Los Alamitos (2001)
Charlot, M., De Fabritis, G., Garcia de Lomana, A.L., Gomez-Garrido, A., Groen, D., et al.: The QosCosGrid project. In: Ibergrid 2007 conference, Centro de Supercomputacion de Galicia (2007)
Coti, C., Herault, T., Peyronnet, S., Rezmerita, A., Cappello, F.: Grid services for MPI. In: Proc. of CCGRID, pp. 417–424. IEEE, Los Alamitos (2008)
Foster, I.T.: Globus toolkit version 4: Software for service-oriented systems. J. Comput. Sci. Technol. 21(4), 513–520 (2006)
Gabriel, E., Resch, M.M., Beisel, T., Keller, R.: Distributed computing in a heterogeneous computing environment. In: Alexandrov, V.N., Dongarra, J. (eds.) PVM/MPI 1998. LNCS, vol. 1497, pp. 180–187. Springer, Heidelberg (1998)
Gropp, W., Lusk, E., Doss, N., Skjellum, A.: High-performance, portable implementation of the MPI message passing interface standard. Parallel Computing 22(6), 789–828 (1996)
Grunberg, M., Genaud, S., Mongenet, C.: Parallel seismic ray tracing in a global earth model. In: Proc. of PDPTA, vol. 3, pp. 1151–1157. CSREA Press (2002)
Karonis, N.T., de Supinski, B.R., Foster, I., Gropp, W., Lusk, E., Bresnahan, J.: Exploiting hierarchy in parallel computer networks to optimize collective operation performance. In: Proc. of SPDP, pp. 377–386. IEEE, Los Alamitos (2000)
Karonis, N.T., Toonen, B.R., Foster, I.T.: MPICH-G2: A grid-enabled implementation of the message passing interface. In: CoRR, cs.DC/0206040 (2002)
Kielmann, T., Hofman, R.F.H., Bal, H.E., Plaat, A., Bhoedjang, R.A.F.: MAGPIE: MPI’s collective communication operations for clustered wide area systems. In: Proc. of PPoPP. ACM Sigplan, vol. 34.8, pp. 131–140. ACM Press, New York (1999)
Kravtsov, V., Carmeli, D., Schuster, A., Yoshpa, B., Silberstein, M., Dubitzky, W.: Quasi-opportunistic supercomputing in grids, hot topic paper. In: Proc. of HPDC (2007)
Kravtsov, V., Swain, M., Dubin, U., Dubitzky, W., Schuster, A.: A fast and efficient algorithm for topology-aware coallocation. In: Bubak, M., van Albada, G.D., Dongarra, J., Sloot, P.M.A. (eds.) ICCS 2008, Part I. LNCS, vol. 5101, pp. 274–283. Springer, Heidelberg (2008)
Matsuda, M., Kudoh, T., Kodama, Y., Takano, R., Ishikawa, Y.: TCP adaptation for MPI on long-and-fat networks. In: Proc. of CLUSTER, pp. 1–10. IEEE, Los Alamitos (2005)
Nascimento, P., Sena, C., da Silva, J., Vianna, D., Boeres, C., Rebello, V.: Managing the execution of large scale mpi applications on computational grids. In: Proc. of SBAC-PAD, pp. 69–76 (2005)
Rabenseifner, R.: Optimization of collective reduction operations. In: Bubak, M., van Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds.) ICCS 2004. LNCS, vol. 3036, pp. 1–9. Springer, Heidelberg (2004)
Thain, D., Tannenbaum, T., Livny, M.: Condor and the grid. In: Grid Computing: Making the Global Infrastructure a Reality. John Wiley & Sons Inc., Chichester (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Coti, C., Herault, T., Cappello, F. (2009). MPI Applications on Grids: A Topology Aware Approach. In: Sips, H., Epema, D., Lin, HX. (eds) Euro-Par 2009 Parallel Processing. Euro-Par 2009. Lecture Notes in Computer Science, vol 5704. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03869-3_45
Download citation
DOI: https://doi.org/10.1007/978-3-642-03869-3_45
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03868-6
Online ISBN: 978-3-642-03869-3
eBook Packages: Computer ScienceComputer Science (R0)