Abstract
In the distributed computing environment, many large-scale scientific applications are irregular applications which perform their computation and I/O on an irregularly discretized mesh. However, most of the previous work in the area of irregular applications focuses mainly on the local environments. In distributed computing environments, since many remotely located scientists should share the data to produce useful results, providing a consistent data replication mechanism to minimize the remote data access time is a critical issue in achieving high-performance bandwidth. We have developed a replication software architecture(RSA) that enables the geographically distributed scientists to easily replicate irregular computations with minimum overheads, while safely sharing large-scale data sets to produce useful results. Since RSA uses database support to store the data-related and computational-related metadata, it can easily be ported to any computing environments. In this paper, we describe the design and implementation of RSA for irregular applications and present performance results on Linux clusters.
This work was supported in part by a Seoul R&BD program.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Rosario, J.M., Choudhary, A.: High performance I/O for parallel computers: Problems and prospects. IEEE Computer 27, 59–68 (1994)
Allcock, B., Foster, I., Nefedova, V., Chervenak, A., Deelman, E., Kesselman, C., Leigh, J., Sim, A., Shoshani, A., Drach, B., Williams, D.: High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies. In: Reich, S., Tzagarakis, M.M., De Bra, P.M.E. (eds.) SC 2001. LNCS, vol. 2266. Springer, Heidelberg (2002)
Das, R., Uysal, M., Saltz, J., Hwang, Y.-S.: Communication optimizations for irregular scientific computations on distributed memory architectures. Journal Parallel and Distributed Computing 22, 462–479 (1994)
Gropp, W., Lusk, E., Thakur, R.: Using MPI-2: Advanced Features of the Message-Passing Interface. MPI Press (1999)
Hanxleden, R.V., Kennedy, K., Saltz, J.: Value-Based Distributions and Alignments in Fortran D. Journal of Programming Languages - Special Issue on Compiling and Run-Time Issues for Distributed Address Space Machines (1994)
Moore, R., Rajasekar, A.: Data and Metadata Collections for Scientific Applications. High Performance Computing and Networking (2001)
Chervenak, A., Deelman, E., Kesselman, C., Pearlman, L., Singh, G.: A Metadata Catalog Service for Data Intensive Applications. GriPhyN technical report (2002)
Chervenak, A., Schuler, R., Kesselman, C., Koranda, S., Moe, B.: Wide Area Data Replication for Scientific Collaborations. In: Proceedings of 6th IEEE/ACM International Workshop on Grid Computing (2005)
Cai, M., Chervenak, A., Frank, M.: A Peer-to-Peer Replica Location Service Based on A Distributed Hash Table. In: Proceedings of the SC 2004 Conference (2004)
Thakur, R., Gropp, W.: Improving the Performance of Collective Operations in MPICH. In: Proceedings of the 10th European PVM/MPI Users’ Group Conference (2003)
No, J., Thakur, R., Choudhary, A.: High-performance scientific data management system. Journal Parallel and Distributed Computing 63, 434–447 (2003)
Karypis, G., Kumar, V.: A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs. Journal on Scientific Computing (1997)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
No, J., Park, C.W., Park, S.S. (2007). A Replication Software Architecture(RSA) for Supporting Irregular Applications on Wide-Area Distributed Computing Environments. In: Stojmenovic, I., Thulasiram, R.K., Yang, L.T., Jia, W., Guo, M., de Mello, R.F. (eds) Parallel and Distributed Processing and Applications. ISPA 2007. Lecture Notes in Computer Science, vol 4742. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74742-0_48
Download citation
DOI: https://doi.org/10.1007/978-3-540-74742-0_48
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74741-3
Online ISBN: 978-3-540-74742-0
eBook Packages: Computer ScienceComputer Science (R0)