Skip to main content
Springer Nature Link
Log in

From Event-Driven Workflows Towards a Posteriori Computing

  • Chapter
Future Generation Grids

Abstract

The integration of content-based event notification systems with workflow management is motivated by the need for dynamic, data-driven application systems which can dynamically discover, ingest data from, and interact with other application systems, including physical systems with online sensors and actuators. This requires workflows that can be dynamically reconfigured on-the-fly on the receipt of important events from both external physical systems and from other computational systems. Such a capability also supports fault tolerance, i.e., reconfiguring workflows on the receipt of failure events. When decentralized workflow management is considered, the need for a workflow agent framework becomes apparent. A key observation here is that systems providing truly autonomic, reconfigurable workflows cannot rely on any form of a priori knowledge, i.e., static information that is “compiled-in” to an application. Hence, applications will have to increasingly rely on a posteriori information that is discovered, understood, and ingested during run-time. In the most general case, this will require semantic analysis and planning to reach abstract goal states. These observations indicate that future generation grids could be pushed into more declarative programming methods and the use of artificial intelligence — topics that must be approached carefully to identify those problem domains where they can be successfully applied.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludäscher, and S. Mock:. Kepler: An extensible system for design and execution of scientific workflows. In 16th International Conference on Scientific and Statistical Database Management, 2004.

    Google Scholar 

  2. K. Amin et al. GridAnt: A Client-Controllable Grid Workflow System. In 37th Annual Hawaii International Conference on System Sciences, January 2004.

    Google Scholar 

  3. J. Blythe, E. Deelman, and Y. Gil. Automatically composed workflows for grid environments. IEEE Intelligent Systems, 19(4): 16–23, July/August 2004.

    Article  Google Scholar 

  4. J. Blythe et al. Task scheduling strategies for workflow-based applications in grids. In International Symposium on Cluster Computing and the Grid, 2005. To appear.

    Google Scholar 

  5. G. Bosilca et al. MPICH-V: Toward a Scalable Fault Tolerant MPI for Volatile Nodes. In IEEE/ACM Supercomputing’ 02, November 2002.

    Google Scholar 

  6. S. Brunett, D. Davis, T. Gottschalk, P. Messina, and C. Kesselman. Implementing distributed synthetic forces simulations in metacomputing environments. In Proceedings of the Heterogeneous Computing Workshop, March 1998.

    Google Scholar 

  7. M. Castro, P. Druschel, A. M. Kermarrec, and A. I. T. Rowstron. Scribe: a large-scale and decentralized application-level multicast infrastructure. IEEE Journal on Selected Areas in Communications, 20(8): 1489–99, 2002.

    Article  Google Scholar 

  8. A. Chervenak et al. Giggle: a framework for constructing scalable replica location services. In Supercomputing, 2002.

    Google Scholar 

  9. F. Darema. Grid Computing and Beyond: The Context of Dynamic, Data-Driven Application Systems. Proceedings of the IEEE, March 2005.

    Google Scholar 

  10. E. Deelman, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, S. Patil, M.-H. Su, K. Vahi, and M. Livny. Pegasus: Mapping scientific workflows onto the grid. In Across Grids Conference 2004, Nicosia, Cyprus, 2004.

    Google Scholar 

  11. E. Deelman et al. GriPhyN and LIGO, Building a Virtual Data Grid for Gravitational Wave Scientists. in 11th Intl Symposium on High Performance Distributed Computing. 2002.

    Google Scholar 

  12. E. Deelman et al. Mapping abstract complex workflows onto grid environments. Journal of Grid Computing, 1(1), 2003.

    Google Scholar 

  13. E. Deelman et al. Grid-based metadata services. In 16th International Conference on Scientific and Statistical Database Management, 2004.

    Google Scholar 

  14. E. Deelman, James Blythe, Yolanda Gil, and Carl Kesselman. The Grid Resource Management, chapter Workflow Management in GriPhyN. Kluwer, 2003.

    Google Scholar 

  15. E. Deelman et al. Grid-based galaxy morphology analysis for the national virtual observatory. In SC 2003, 2003.

    Google Scholar 

  16. E. Deelman et al. Pegasus and the pulsar search: From metadata to execution on the grid. In Applications Grid Workshop, PPAM 2003, Czestochowa, Poland, 2003.

    Google Scholar 

  17. I. Foster et al. Chimera: A Virtual Data System for Representing, Querying, and Automat-ing Data Derivation. in Scientific and Statistical Database Management, 2002.

    Google Scholar 

  18. I. Foster, N. Jennings, and C. Kesselman. Brain Meets Brawn: Why Grid and Agents Need Each Other. In AAMAS, July 19–23 2004.

    Google Scholar 

  19. I. Foster, C. Kesselman (Eds.) The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers, Inc., 1998.

    Google Scholar 

  20. I. Foster, C. Kesselman, C. Lee, B. Lindell, K. Nahrstedt, and A. Roy. A distributed resource management architecture that supports advance rservations and co-allocation. In 7th IEEE/IFIP International Workshop on Quality of Service, 1999.

    Google Scholar 

  21. G. Fox and S. Pallickara. An Event Service to Support Grid Computational Environments. Concurrency and Computation: Practice & Experience, 14(13–15): 1097–1129, 2002.

    Article  MATH  Google Scholar 

  22. P. Freeman et al. Cyberinfrastructure for Science and Engineering: Promises and Challenges. Proceedings of the IEEE, March 2005. Special issue on Grid Computing, M. Parashar and C. Lee, guest editors.

    Google Scholar 

  23. J. Frey, T. Tannenbaum, M. Livny, and S. Tuecke. Condor-G: A Computation Managament Agent for Multi-Institutional Grids. In Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC10), San Francisco, California, 2001.

    Google Scholar 

  24. M. Ghallab, D. Nau, and P. Traverso. Automated Planning: Theory and Practice. Morgan Kaufmann Publishers, 2004.

    Google Scholar 

  25. Y. Gil, E. Deelman, J. Blythe, C. Kesselman, and H. Tangmunarunkit. Artificial Intelligence and Grids: Workflow Planning and Beyond. IEEE Intelligent Systems, pages 26–33, Jan.–Feb. 2004.

    Google Scholar 

  26. M. Govindaraju et al. Merging the CCA Component Model with the OGSI Framework. In International Symposium on Cluster Computing and the Grid, May 2003.

    Google Scholar 

  27. Active Networks Working Group. Architectural framework for active networks. http://www.cc.gatech.edu/projects/canes/arch/arch-0-9.ps, 1999.

    Google Scholar 

  28. HLA Working Group. IEEE standard for modeling and simulation (M&S) high level architecture (HLA)-framework and rules. IEEE Standard 1516–2000, 2000.

    Google Scholar 

  29. J. C. Jacob et al. The montage architecture for grid-enabled science processing of large, distributed datasets. In Proceedings of the Earth Science Technology Conference (ESTC), 2004.

    Google Scholar 

  30. K. Kennedy et al. Toward a framework for preparing and executing adaptive grid programs. In Proceedings of NSF Next Generation Systems Program Workshop (International Parallel and Distributed Processing Symposium), April 2002.

    Google Scholar 

  31. J. Kim et al. A knowledge-based approach to interactive workflow composition. In Proceedings of Workshop: Planning and Scheduling for Web and Grid Services at the 14th International Conference on Automatic Planning and Scheduling (ICAPS 04), Whistler, Canada, 2004.

    Google Scholar 

  32. C. Lee. Topology-aware communication in wide-area message-passing. In Recent Advances in Parallel Virtual Machines and Message Passing Interface, pages 644–652, 2003. Springer-Verlag LNCS 2840.

    Google Scholar 

  33. C. Lee, E. Coe, B.S. Michel, I. Solis, J. Stepanek, J.M. Clark, and B. Davis. Using topology-aware communication services in grid environments. In Workshop on Grids and Advanced Networks, International Symposium on Cluster Computing and the Grid, May 2003.

    Google Scholar 

  34. C. Lee and S. Michel. The use of content-based routing to support events, coordination and topology-aware communication in wide-area grid environments. In D. Marinescu and C. A. Lee, editors, Process Coordination and Ubiquitous Computing, pages 99–118. CRC Press, 2003.

    Google Scholar 

  35. C. Lee and D. Talia. Grid programming models: Current tools, issues and directions. In Berman, Fox, and Hey, editors, Grid Computing: Making the Global Infrastructure a Reality, pages 555–578. Wiley, 2003.

    Google Scholar 

  36. B. Scott Michel. General-purpose Peer-to-Peer Infrastructure. PhD thesis, University of California, Los Angeles, 2004.

    Google Scholar 

  37. NIST. The secure hash algorithm (sha-1). Technical Report NIST FIPS PUB 180-1, National Institute of Standards and Technology, U.S. Department of Commerce, April 1995.

    Google Scholar 

  38. T. Oinn et al. Taverna: A tool for the composition and enactment of bioinformatics workflows. Bioinformatics Journal, 20(17):3045–3054, 2004.

    Article  Google Scholar 

  39. S. Pallickara and G. Fox. NaradaBrokering: A Middleware Framework and Architecture for Enabling Durable Peer-to-Peer Grids. In ACM/IFIP/USENIX International Middleware Conference, pages 41–61, 2003.

    Google Scholar 

  40. D. De Roure, N. Jennings, and N. Shadbolt. The Semantic Grid: Past, Present and Future. Proceedings of the IEEE, March 2005. Special issue on Grid Computing, M. Parashar and C. Lee, guest editors.

    Google Scholar 

  41. A. Rowstron and P. Druschel. Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems. In IFIP/ACM Intl Conf. on Distributed Systems Platforms (Middleware 2001), November 2001. Heidelberg, Germany.

    Google Scholar 

  42. A. Rowstron, A-M. Kermarrec, M. Castro, and P. Druschel. Scribe: The design of a large-scale event notification infrastructure. In Network Group Communication (NGC 2001), London, UK, 2001.

    Google Scholar 

  43. S. Graham et al. Publish-Subscribe Notification for Web Services. http://www-106.ibm.com/developerworks/library/ws-pubsub, 2004.

    Google Scholar 

  44. G. Singh et al. A Metadata Catalog Service for Data Intensive Applications. SC03, 2003.

    Google Scholar 

  45. D. Skillicorn and D. Talia. Models and languages for parallel computation. ACM Computing Surveys, 30(2), June 1998.

    Google Scholar 

  46. P. Stelling, C. DeMatteis, I. Foster, C. Kesselman, C. Lee, and G. von Laszewski. A fault detection service for wide area distributed computations. Cluster Computing, 2(2): 117–128, 1999. Special Issue on HPDC-7.

    Article  Google Scholar 

  47. I. Stoica, D. Adkins, S. Zhuang, S. Shenker, and S. Surana. Internet indirection infrastructure. ACM. Computer Communication Review, 32:73–86, 2002.

    Article  Google Scholar 

  48. I. Stoica, R. Morris, D. Karger, F. Kaashoek, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. ACM Computer Communication Review, 31(4):149–160, 2001.

    Article  Google Scholar 

  49. SUN Microsystems, Inc. Java Message Service API. http://java.sun.com/products/jms, 2002.

    Google Scholar 

  50. S.M. Sutton and L.J. Osterweil. The design of the next generation process language. In Joint 6th European Software Engineering Conference and the 5th ACM SIGSOET Symposium on Foundations of Software Engineering, pages 141–158, 1997.

    Google Scholar 

  51. I. Taylor, I. Wang, M. Shields, and S. Majithia. Distributed computing with Triana on the Grid. Concurrency and Computation:Practice and Experience, 17(1–18), 2005.

    Google Scholar 

  52. W.W. Terpstra, S. Behnel, L. Fiege, A. Zeidler, and A.P. Buchmann. A peer-to-peer approach to content-based publish/subscribe. In International Workshop on Distributed Event Based Systems (DEBS03), 2003.

    Google Scholar 

  53. The Object Management Group. CORBA 3 Release Information. http://www.omg.org/technology/corba/corba3releaseinfo.htm, 2000.

    Google Scholar 

  54. P. Triantafillou and I. Aekaterinidis. Content-based publish/subscribe systems over structured p2p networks. In International Workshop on Distributed Event Based Systems (DEBS04), May 2004.

    Google Scholar 

  55. M. Veloso et al. Integrating Planning and Learning: The PRODIGY Architecture. J. of Experimental and Theoretical Artificial Intelligence, 7(1), 1995.

    Google Scholar 

  56. S. Zabele et al. SANDS: Specialized Active Networking for Distributed Simulation. DARPA Active Network Conference and Exposition, pages 356–365, May 29–30 2002.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Systems Research Department, The Aerospace Corporation, El Segundo, CA, USA

    Craig A. Lee & B. Scott Michel

  2. Information Sciences Institute, University of Southern California, Marina del Rey, CA, USA

    Ewa Deelman & Jim Blythe

Authors
  1. Craig A. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. B. Scott Michel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ewa Deelman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jim Blythe
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Westminster, London, UK

    Vladimir Getov

  2. Information Science and Technologies Institute, Pisa, Italy

    Domenico Laforenza

  3. Zuse-Institut Berlin and Humboldt-Universitat zu Berlin, Germany

    Alexander Reinefeld

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer Science+Business Media, Inc.

About this chapter

Cite this chapter

Lee, C.A., Michel, B.S., Deelman, E., Blythe, J. (2006). From Event-Driven Workflows Towards a Posteriori Computing. In: Getov, V., Laforenza, D., Reinefeld, A. (eds) Future Generation Grids. Springer, Boston, MA . https://doi.org/10.1007/978-0-387-29445-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-0-387-29445-2_1

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-387-27935-0

  • Online ISBN: 978-0-387-29445-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics