Morco: middleware framework for long-running multi-component applications on batch grids
Pages 328 - 331
Abstract
While computational grids with multiple batch systems (batch grids) have been used for efficient executions of loosely-coupled and workflow-based parallel applications, they can also be powerful infrastructures for executing long-running multi-component parallel applications. In this paper, we have constructed a generic middleware framework for executing long-running multi-component applications with execution times much greater than execution time limits of batch queues. Our framework coordinates the distribution, execution, migration and restart of the components of the application on the multiple queues, where the component jobs of the different queues can have different queue waiting and startup times. We have used our framework with a foremost long-running multi-component application for climate modeling, the Community Climate System Model (CCSM). We have performed real multiple-site CCSM runs for 6.5 days of wallclock time spanning three sites with four queues and emulated external workloads. Our experiments indicate that multi-site executions can lead to good throughput of application execution.
References
[1]
}}A. Bhatele, S. Kumar, M. Chao, J. Phillips, Z. Gengbin, and L. Kale. Overcoming Scaling Challenges in Biomolecular Simulations across Multiple Platforms. In IPDPS '08: Proceedings of the 2008 IEEE International Symposium on Parallel and Distributed Processing, pages 1--12, 2008.
[2]
}}J. Buisson, O. Sonmez, H. Mohamed, W. Lammers, and D. Epema. Scheduling Malleable Applications in Multicluster Systems. In CLUSTER '07: Proceedings of the 2007 IEEE International Conference on Cluster Computing, pages 372--381, 2007.
[3]
}}Community Climate System Model (CCSM). http://www.ccsm.ucar.edu.
[4]
}}W. Collins, C. Bitz, M. Blackmon, G. Bonan, C. Bretherton, J. Carton, P. Chang, S. Doney, J. Hack, T. Henderson, J. Kiehl, W. Large, D. McKenna, B. Santer, and R. Smith. The community climate system model: Ccsm3. 1998.
[5]
}}B. Howe, P. Lawson, R. Bellinger, E. Anderson, E. Santos, J. Freire, C. Scheidegger, A. Baptista, and C. Silva. End-to-End eScience: Integrating Workflow, Query, Visualization, and Provenance at an Ocean Observatory. In ESCIENCE '08: Proceedings of the 2008 Fourth IEEE International Conference on eScience, pages 127--134, 2008.
[6]
}}Y. Joshi and S. Vadhiyar. Analysis of DNA Sequence Transformations on Grids. Journal of Parallel and Distributed Computing, 69(1):80--90, 2009.
[7]
}}S. V. Kumar, P. Sadayappan, G. Mehta, K. Vahi, E. Deelman, V. Ratnakar, J. Kim, Y. Gil, M. Hall, T. Kurc, and J. Saltz. An Integrated Framework for Performance-based Optimization of Scientific Workflows. In HPDC '09: Proceedings of the 18th ACM international symposium on High performance distributed computing, pages 177--186, 2009.
[8]
}}U. Lublin and D. Feitelson. The Workload on Parallel Supercomputers: Modeling the Characteristics of Rigid Jobs. Journal of Parallel and Distributed Computing, 63(11):1105--1122, 2003.
[9]
}}N. Markatchev, C. Kiddle, and R. Simmonds. A Framework for Executing Long Running Jobs in Grid Environments. In HPCS '08: Proceedings of the 22nd International Symposium on High Performance Computing Systems and Applications, pages 69--75, 2008.
[10]
}}K. Nomura, R. Seymour, W. Weiqiang, H. Dursun, R. Kalia, A. Nakano, P. Vashishta, F. Shimojo, and L. Yang. A Metascalable Computing Framework for Large Spatiotemporal-scale Atomistic Simulations. In IPDPS '09: Proceedings of the 2009 IEEE International Symposium on Parallel and Distributed Processing, pages 1--10, 2009.
Index Terms
- Morco: middleware framework for long-running multi-component applications on batch grids
Recommendations
Adaptive Executions of Multi-Physics Coupled Applications on Batch Grids
Long running multi-physics coupled parallel applications have gained prominence in recent years. The high computational requirements and long durations of simulations of these applications necessitate the use of multiple systems of a Grid for execution. ...
Large improvements in application throughput of long-running multi-component applications using batch grids
Computational grids with multiple batch systems (batch grids) can be powerful infrastructures for executing long-running multi-component parallel applications. In this paper, we evaluate the potential improvements in throughput of long-running multi-...
Grids with multiple batch systems for performance enhancement of multi-component and parameter sweep parallel applications
In this work, we evaluate the benefits of using Grids with multiple batch systems to improve the performance of multi-component and parameter sweep parallel applications by reduction in queue waiting times. Using different job traces of different loads, ...
Comments
Information & Contributors
Information
Published In
Copyright © 2010 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
- University of Arizona: University of Arizona
- SIGARCH: ACM Special Interest Group on Computer Architecture
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 June 2010
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
HPDC '10
Sponsor:
- University of Arizona
- SIGARCH
HPDC '10: The 19th International Symposium on High Performance Distributed Computing
June 21 - 25, 2010
Illinois, Chicago
Acceptance Rates
Overall Acceptance Rate 166 of 966 submissions, 17%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 93Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 07 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in