Skip to main content
SpringerLink
Log in

Campus Grids Meet Applications: Modeling, Metascheduling and Integration

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Air Quality Forecasting (AQF) is a new discipline that attempts to reliably predict atmospheric pollution. An AQF application has complex workflows and in order to produce timely and reliable forecast results, each execution requires access to diverse and distributed computational and storage resources. Deploying AQF on Grids is one option to satisfy such needs, but requires the related Grid middleware to support automated workflow scheduling and execution on Grid resources. In this paper, we analyze the challenges in deploying an AQF application in a campus Grid environment and present our current efforts to develop a general solution for Grid-enabling scientific workflow applications in the GRACCE project. In GRACCE, an application’s workflow is described using GAMDL, a powerful dataflow language for describing application logic. The GRACCE metascheduling architecture provides the functionalities required for co-allocating Grid resources for workflow tasks, scheduling the workflows and monitoring their execution. By providing an integrated framework for modeling and metascheduling scientific workflow applications on Grid resources, we make it easy to build a customized environment with end-to-end support for application Grid deployment, from the management of an application and its dataset, to the automatic execution and analysis of its results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bose, A., Wickman, B., Wood, C.: MARS: A metascheduler for distributed resources in campus Grids, Proceedings of Fifth IEEE/ACM International Workshop on Grid Computing, 2004

  2. Chapman, B.M., Raghunath, P., Sundaram, B., Yan, Y.: Air quality prediction in a production quality Grid environment. In: Dongarra, J., et al. (eds.) Engineering the Grid: Status and Perspective. (Spring 2005)

  3. Chapman, B.M., Donepudi, H., He, J., Li, Y., Raghunath, P., Sundaram, B., Yan, Y.: Grid Environment with Web-Based Portal Access for Air Quality Modeling, Parallel and Distributed Scientific and Engineering Computing, Practice and Experience, 2003

  4. Chapman, B.M., Li, Y., B., Sundaram, He, J.: Computational Environment for Air Quality Modeling in Texas, Use of High Performance Computing in Meteorology. World Scientific (2003)

  5. Churches, D., Gombas, G., Harrison, A., Maassen, J., Robinson, C., Shields, M., Taylor, I., Wang, I.: Programming Scientific and Distributed Workflow with Triana Services, Special Issue of Concurrency and Computation: Practice and Experience, 2005

  6. Byun, D.W., Pleim, J., Tang, R., Bourgeois, A.: Meteorology-Chemistry Interface Processor (MCIP) for Models-3 Community Multiscale Air Quality (CMAQ) Modeling System. Environmental Protection Agency, Office of Research and Development, Washington, District of Columbia (1999)

    Google Scholar 

  7. Byun, D.W., Schere, K.: EPA’s third generation air quality modeling system: Description of the models-3 Community Multiscale Air Quality (CMAQ) Model. Journal of Mech. Review (2004)

  8. Deelman, E., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Patil, S., Su, M., Vahi, K., Livny, M.: Pegasus: Mapping Scientific Workflows onto the Grid, Across Grids Conference 2004, Nicosia, Cyprus

  9. Grell, G., Dudhia, J., Stauffer, D.: A Description of the Fifth-Generation Penn State/NCAR Mesoscale Model (MM5), NCAR Tech Notes, TN-398+STR

  10. Foster, I., Kesselman, C.: Globus: A metacomputing infrastructure toolkit. I. J. Supercomput. Appl. (Summer 1997)

  11. Foster, I., Kesselman, C., Lee, C., Lindell, R., Nahrstedt, K., Roy, A.: A distributed resource management architecture that supports advance reservations and co-allocation. Intl Workshop on Quality of Service, 1999

  12. Foster, I., Kesselman, C., Tsudik, G., Tuecke, S.: A security architecture for computational Grids, ACM Conference on Computers and Security, 83-91, 1998

  13. Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the Grid: Enabling scalable virtual organizations. Int. J. High Perform. Comput. Appl. 15(3), 200-222 (2001)

    Article  Google Scholar 

  14. Foster, I., Vockler, J., Wilde, M., Zhao, Y.: Chimera: A virtual data system for representing, querying, and automating data derivation, Proceedings of the 14th International Conference on Scientific and Statistical Database Management, pp. 37-46, 2002

  15. Cao, J., Jarvis, S.A., Saini, S., Nudd, G.R.: GridFlow: Workflow management for Grid computing, Proceedings of 3rd International Symposium on Cluster Computing and the Grid, at Tokyo, Japan, May 12-15, 2003

  16. Yu, J., Buyya, R.: A taxonomy of workflow management systems for Grid computing, Technical Report, GRIDS-TR-2005-1, University of Melbourne, Australia, March, 2005

  17. Czajkowski, K., Foster, I., Kesselman, C.: Resource co-allocation in computational Grids, Proceedings of the Eighth IEEE International Symposium on High Performance Distributed Computing (HPDC-8), pp. 219-228, 1999

  18. Czajkowski, K., Foster, I., Kesselman, C., Sander, V., Tuecke, S.: SNAP: A Protocol for negotiating service level agreements and coordinating resource management in distributed systems. Lect. Notes Comput. Sci. 2537, 153-183 (2002)

    Article  Google Scholar 

  19. Czajkowski, K., Foster, I., Karonis, N., Kesselman, C., Martin, S., Smith, W., Tuecke, S.: A resource management architecture for metacomputing systems, Workshop on Job Scheduling Strategies for Parallel Processing, pp. 62-82, 1998

  20. Wieczorek, M., Prodan, R., Fahringer, T.: Scheduling of scientific workflows in the ASKALON Grid environment. ACM SIGMOD Record Journal (2005)

  21. Buyya, R., Abramson, D., Giddy, J.: Nimrod/G: An architecture for a resource management and scheduling system in a global computational Grid, The 4th International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia 2000), May 2000

  22. Fahringer, T., Qin, J., Hainzer, S.: Specification of Grid workflow applications with AGWL: An abstract Grid workflow language, Proceedings of Cluster Computing and Grid, 2005

  23. Oinn, T., et al.: Taverna: Lessons in creating a workflow environment for the life sciences, Concurrency and Computation: Practice and Experience Grid Workflow Special Issue, 09, 2002

  24. Dabberdt, W.F., Carroll, M.A., Baumgardner, D., Carmichael, G., Cohen, R.: Meteorological research needs for improved air quality forecasting, The 11th Prospectus Development Team of the U.S. Weather Research Program, 2004

  25. Smith, W., Foster, I., Taylor, V.: Scheduling with advanced reservations, Proceedings of the 14th International Parallel and Distributed Processing Symposium (IPDPS’00), 2000

  26. Adelman, Z., Houyoux, M.: Processing the National Emissions Inventory 96 (NEI96) version 3.11 with SMOKE, The Emission Inventory Conference: One Atmosphere, One Inventory, Many Challenges, 1–3 May, Denver, Colorado Environmental Protection Agency, 2001

  27. BPEL4WS: Business Process Execution Language for Web Services v1.0, http://www.106.ibm.com/developerworks/webservices/library/wsbpel

  28. Community Scheduler Framework: http://www.platform.com/products/Globus

  29. DAGMan (Directed Acyclic Graph Manager): http://www.cs.wisc.edu/condor/dagman

  30. GRACCE: Grid Application Coordination, Collaboration and Execution: http://www.cs.uh.edu/~yanyh/gracce

  31. High Performance Computing Center, University of Houston: http://www.hpcc.uh.edu/

  32. Java CoG Kit Karajan/Gridant Workflow Guide: http://www.cogkit.org/release/4_0_a1/manual/workflow.pdf

  33. Job Submission Description Language, GGF: https://forge.Gridforum.org/projects/jsdl-wg

  34. Load Sharing Facility, Resource Management and Job Scheduling System: http://www.platform.com/products/HPC/

  35. Maui Moab Grid Scheduler (Silver): http://www.clusterresources.com/products/mgs

  36. Sun Grid Engine, Sun Microsystems: http://gridengine.sunsource.net/

  37. The Globus Resource Specification Language RSL v1.0: http://www-fp.globus.org/gram/rsl_spec1.html

  38. The University of Houston’s Sun Microsystems Center of Excellence in Geosciences: http://www.suncoe.uh.edu

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Houston, PGH 501, 4800 Calhoun Rd., Houston, TX, 77204, USA

    Yonghong Yan & Barbara M. Chapman

Authors
  1. Yonghong Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Barbara M. Chapman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yonghong Yan.

Additional information

The work has been performed as part of the University of Houston’s Sun Microsystems Center of Excellence in Geosciences [38].

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yan, Y., Chapman, B.M. Campus Grids Meet Applications: Modeling, Metascheduling and Integration. J Grid Computing 4, 159–175 (2006). https://doi.org/10.1007/s10723-006-9046-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-006-9046-4

Key words

Search

Navigation