Skip to main content
Springer Nature Link
Log in

Achieving Interoperation of Grid Data Resources via Workflow Level Integration

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Production Grids are becoming widely utilized by the e-Science community to run computation and data intensive experiments more efficiently. Unfortunately, different production Grid infrastructures are based on different middleware technologies, both for computation and for data access. Although there is significant effort from the Grid community to standardize the underlying middleware, solutions that allow existing non-standard tools to interoperate are one of the major concerns of Grid users today. This paper describes the generic requirements towards the interoperation of Grid data resources within computational workflows, and suggests integration techniques that allow workflow engines to access various heterogeneous data resources during workflow execution. Reference implementations of these techniques are presented and recommendations on their applicability and suitability are made.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. The EGEE—Enabling Grids for E-sciencE—website. http://www.eu-egee.org/

  2. The TeraGrid website. http://www.teraGrid.org

  3. The Open Science Grid website. http://www.openscienceGrid.org/

  4. The UK National Grid Service website. http://www.ngs.ac.uk/

  5. The D-Grid website. http://www.d-Grid.de/

  6. Open Grid Forum (OGF) Grid Interoperation Now Community Group (GIN-CG): https://forge.Gridforum.org/sf/projects/gin

  7. Riedel, M. (ed.): GIN—Data Management and Movement (GIN-DATA). Experiences from interoperation scenarios in production Grids, DRAFT version 1.00, 30 April 2007. http://forge.ogf.org/sf/docman/do/downloadDocument/projects.gin/docman.root.current_drafts.gin_data_experience_doc/doc14499;jsessionid=A21053D63FF203CBD617A8FA3BD0F984 (2007)

  8. Oinn, T., et al.: Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics J. 20(17), 3045–3054 (2004)

    Article  Google Scholar 

  9. Churches, D., et al.: Programming scientific and distributed workflow with Triana Services. Grid Workflow 2004. Concurrency Comput. Pract. Ex. 18(10), 1021–1037 (2006). ISSN 1532-0626

    Article  Google Scholar 

  10. Jones, M., Lee, E.A., Zhao, Y.: Scientific workflow management and the Kepler system. Concurrency Comput. Pract. Ex. 18(10), 1039–1065. ISSN 1532-0626 (2006)

    Article  Google Scholar 

  11. Kacsuk, P., Sipos, G.: Multi-grid, multi-user workflows in the P-GRADE Grid portal. J. Grid Computing 3(3–4), 221–238 (2005). Springer, 1570–7873. doi:10.1007/s10723-005-9012-6

    Article  Google Scholar 

  12. Kacsuk, P., Kiss, T., Sipos, G.: Solving the Grid interoperability problem by P-GRADE portal at workflow level. Future Gener. Comput. Syst.: Int. J. Grid Computing: Theor. Meth. Appl. 24(7), 744–751 (2008)

    Article  Google Scholar 

  13. Foster, I.: Globus Toolkit Version 4: software for service-oriented systems. J. Comput. Sci. Technol. 21(4), 513–520 (2006)

    Article  Google Scholar 

  14. The WestFocus Grid Alliance website, http://www.westfocus.org.uk/ICT/p54g12_About_the_project.aspx

  15. Laure, E., et al.: Programming the Grid with gLite. Comput. Methods Sci. Technol. 12(1), 33–45 (2006)

    Google Scholar 

  16. Kukla, T., Kiss, T., Terstyanszky, G., Kacsuk, P.: A general and scalable solution for heterogeneous workflow invocation and nesting. Accepted for the Workflows in Support of Large-Scale Science (WORKS08), to be held in conjunction with Supercomputing 08, Austin, USA, 17 November 2008

  17. Delaittre, T., Kiss, T., Goyeneche, A., Terstyanszky, G., Winter, S., Kacsuk, P.: GEMLCA: running legacy code applications as Grid services. J. Grid Computing 3(1–2), 75–90. Springer, 1570–7873 (2005)

    Article  Google Scholar 

  18. Sim, A., Soshani, A. (ed.): Storage Resource Manager Interface Specification version 2.2, 09 May 2007. http://www.ogf.org/Public_Comment_Docs/Documents/2007-10/OGF-GSM-SRM-v2.2.pdf (2007)

  19. Rajasekar, A., et al.: Storage resource broker—managing distributed data in a Grid. J. Comput. Soc. India 33(4), 42–54 (2003)

    Google Scholar 

  20. Allcock, W., Bester, J., Bresnahan, J., Chervenak, A., Liming, L., Tuecke, S.: GridFTP: Protocol Extension to FTP for the Grid, March 2001. http://www-fp.mcs.anl.gov/dsl/GridFTP-Protocol-RFC-Draft.pdf (2001)

  21. Open Grid Forum Data Access an Integration Services Working Group (DAIS). http://forge.ogf.org/sf/projects/dais-wg

  22. Antonioletti, M., et al.: The design and implementation of Grid database services in OGSA-DAI. Concurrency and Computation: Practice and Experience, vol. 17(2–4), pp. 357–376. Special Issue: Grids and Web Services for e-Science. Wiley, New York (2005)

    Google Scholar 

  23. Watson, P.: Databases in Grid applications: locality and distribution. In: Proceedings of the Database: Enterprise, Skills and Innovation. 22nd British National Conference on Databases, BNCOD 22, Sunderland, UK, 5–7 July 2005. Lecture Notes in Computer Science, vol. 3567, pp. 1–16. Springer, Heidelberg. ISBN 3-540-26973-8 (2005)

    Google Scholar 

  24. Introduction to JSR 168—The Java Portlet Specification, Sun Microsystems White Paper. http://developers.sun.com (2003)

  25. Christie, M., et al.: The LEAD Portal: a TeraGrid gateway and application service architecture. Concurrency and Computation: Practice and Experience, vol. 19(6), pp. 767–781, Oct 2006. Wiley, New York (2006)

    Google Scholar 

  26. Kodeboyina, D., Plale, B.: Experiences with OGSA-DAI: portlet access and benchmark. In: Global Grid Forum Workshop on Designing and Building Grid Services, 8 October 2003. Chicago, Illinois, USA (2003)

  27. Yang, X., et al.: A web portal for the national Grid service. In: Proceedings of the UK E-Science All Hands Meeting, 19–22 September 2005. Nottingham, UK. ISBN 1-904425-534 (2005)

  28. Baud, J.-P., et al.: LCG Data Management from EDG to EGEE. In: Proceedings of the UK E-Science All Hands Meeting, 19–22 September 2005. Nottingham, UK. ISBN 1-904425-534 (2005)

  29. Foster, I., et al.: A security architecture for computational Grids. In: Proc. of the 5th ACM conference on Computer and Communications Security, pp. 83–92 (1998)

  30. OGSA-DAI Release 3.0 Extension Pack 1. Available: http://www.ogsadai.org.uk/documentation/ogsadai3.0-extension-1/ (2008). Accessed: 29 August 2008

  31. Kiss, T., Kacsuk, P., Terstyanszky, G., Winter, S.: Workflow level interoperation of Grid data resources. In: Proceedings of the CCGrid 2008 8th IEEE International Symposium on Cluster Computing and the Grid, 19–22 May 2008, pp. 194–201. Lyon, France (2008)

  32. Kiss, T., Tudose, A., Kacsuk, P., Terstyanszky, G.: SRB data resources in computational Grid workflows. In: Proceedings of the UK e-Science All Hands Meeting 2007, Nottingham, UK, 10th–13th September 2007, pp. 643–650. ISBN 978-0-9553988-3-4 (2007)

  33. Delaitre, T., Goyeneche, A., Kiss, T., Terstyanszky, G.Z., Weingarten, N., Maselino, P., Gourgoulis, A., Winter, S.C.: Traffic simulation in P-grade as a Grid service. In: Conf. Proc. of the DAPSYS 2004 Conference, Budapest, Hungary, pp. 129–136. ISBN 0-387-23094-7 (2004)

  34. R-Development-Core-Team, R Language definition: Available: cran.r-project.org/doc/manuals/R-lang.pdf (1998). Accessed: 29 August 2008

  35. Riedel, M. (ed.): GIN—Information Services and Schema (GIN-INFO) experiences from interoperation scenarios in production Grids, DRAFT version 1.00, 02 October 2007. http://omii-europe.org/OMII-Europe/docs/2007-02-10_OGF_GIN-Info_v_1_00.pdf (2007)

  36. Boardman, R., Crouch, S., Mills, H., Newhouse, S., Papay, J.: Towards Grid interoperability. In: Proceedings of the UK e-Science All Hands Meeting 2007, Nottingham, UK, 10th–13th September 2007, pp. 127–134. ISBN 978-0-9553988-3-4 (2007)

  37. Specification, GFD-R.056, 7 November 2005. www.Gridforum.org/documents/GFD.56.pdf (2005)

  38. Foster, I., et al.: OGSA Basic Execution Service version 1.0, 17 March 2007. http://www.ogf.org/Public_Comment_Docs/Documents/Apr-2007/ogsa-bes-v33.pdf (2007)

  39. Dillaway, B., et al.: HPC Basic Profile version 1.0, 2 October 2007. http://www.ogf.org/Public_Comment_Docs/Documents/Feb-2007/HPC_Basic_Profile_v1.0.pdf (2007)

  40. Lee, W., McGough, A.S., Darlington, J.: Performance evaluation of the GridSAM job submission and monitoring system. In: Conf. Proc. of the UK e-Science All Hands Meeting, Nottingham, UK, pp. 915–922. ISBN 1-904425-534 (2005)

  41. Meredith, D., Maniopoulou, M., Richards, A., Mineter, M.: A JSDL application repository and artefact sharing portal for heterogeneous Grids and the NGS. In: Proceedings of the UK e-Science All Hands Meeting 2007, Nottingham, UK, 10th–13th September 2007, pp. 110–118. ISBN 978-0-9553988-3-4 (2007)

  42. Kertész, A., Rodero, I., Guim, F.: Data model for describing Grid resource broker capabilities. CoreGRID Workshop on Grid Middleware in conjunction with ISC’07 conference, Dresden, Germany, 25–26 June 2007

  43. Kertész, A., Sipos, G., Kacsuk, P.: Multi-grid brokering with the P-GRADE Portal. In: Proceedings of the Austrian Grid Symposium (AGS’06). OCG, Austria (2006)

    Google Scholar 

  44. The GRIP project web site, http://www.gloriad.org/gloriad/projects/project000032.html

  45. OMII Europe Project website. http://omii-europe.org/OMII-Europe/igiiw2007.html

  46. Harrison, A., Kelley, I., Mueller, K., Shields, M., Taylor, I.: Workflows hosted in portals. In: Proceedings of the UK e-Science All Hands Meeting 2007, Nottingham, UK, 10th–13th September 2007, pp. 32–39. ISBN 978-0-9553988-3-4 (2007)

  47. Ellis, J., et al.: JDBC 3.0 Specification. Sun Microsystems (2001)

  48. Demir, E., Chaussalet, T., Weingarten, N., Kiss, T.: A Grid implementation for profiling hospitals based on patient readmissions. Intelligent Patient Management. In: McClean, S., Millard, P., El-Darzi, E., Nugent, C.D. (eds.) Intelligent Patient Management. Studies in Computational Intelligence, vol. 189, pp. 127–146. Springer. ISBN 9783642001789 (2009)

  49. Kiss, T., et al.: Utilizing heterogeneous data sources in computational Grid workflows, appears in making Grids work. In: Danelutto, M., Fragopoulou P., Getov, V. (eds.) Proceedings of the CoreGrid Workshop on Programming Models, Grid and P2P System Architecture, Grid Systems, Tools and Environments, pp. 225–236. Springer, Heidelberg. ISBN: 978-0-387-78447-2 (2008)

  50. Kukla, T.: Integrating the OGSA-DAI to the P-GRADE portal. MSc thesis, Cranfield University, School of Engineering. https://dspace.lib.cranfield.ac.uk/bitstream/1826/2036/1/MSC_Thesis_Final_TKUKLA.pdf (2007)

Download references

Author information

Authors and Affiliations

  1. Centre for Parallel Computing, University of Westminster, 115 New Cavendish Street, London, W1W 6UW, UK

    Tamas Kiss & Tamas Kukla

Authors
  1. Tamas Kiss
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Tamas Kukla
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Tamas Kiss.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kiss, T., Kukla, T. Achieving Interoperation of Grid Data Resources via Workflow Level Integration. J Grid Computing 7, 355–374 (2009). https://doi.org/10.1007/s10723-009-9136-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-009-9136-1

Keywords