Skip to main content
SpringerLink
Log in

Enabling Grid Access to Mass Storage

Architecture and Design of the EDG Storage Element

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

The purpose of the EDG Storage Element (SE) is to provide a uniform Grid interface to mass storage systems. We needed an extensible and flexible architecture, and chose one based on message processing. Not only has this architecture provided a solid framework for meeting changing requirements, it also provides scope for building an extensible and scalable storage management system. While distributed SEs in a Grid are managed by higher level replication services, our request processing model opens possibilities for distributing a single SE across several nodes, allowing further opportunities for optimisation. In this paper, we discuss applications of distributed, transactional processing in mass storage, and its implications for optimisations based on access patterns.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. B. Allcock, J. Bester, J. Bresnahan, A.L. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnal and S. Tuecke, “Data Management and Transfer in High Performance Computational Grid Environments”, Parallel Computing Journal, Vol. 28, No. 5, pp. 749–771, 2002.

    Article  Google Scholar 

  2. K. Czajkowski, S. Fitzgerald, I. Foster and C. Kesselman, “Grid Information Services for Distributed Resource Sharing”, in Proc 10th IEEE Int'l Symposium on High-Performance Distributed Computing, 2001.

  3. I. Bird, B. Hess, A. Kowalski, et al., “Common Storage Resource Manager Operations”, 2001. Retrieved from http://sdm.lbl.gov/srm-wg/doc/srm.v1.0.pdf

  4. J. Gu, A. Sim and A. Shoshani, “The Storage Resource Manager Interface Specification”, version 2.1, 2003. Retrieved from http://sdm.lbl.gov/srm/documents/joint.docs/SRM.spec.v2.1._nal.doc

  5. CERN. “LHC Computing Grid Project”, 2003. Retrieved from http://lcg.web.cern.ch/LCG/

  6. D. Thain and M. Livny, “Multiple bypass: Interposition Agents for Distributed Computing,” Journal of Cluster Computing, Vol. 4, 39–47, 2001.

    Article  Google Scholar 

  7. D. Thain and M. Livny, “Bypass Seminar”, 2000. Retrieved from http://www.cs.wisc.edu/condor/bypass/bypass-seminar.ppt Presentation given at UWisc networking seminar.

  8. W.R. Stevens, Advanced Programming in the Unix Environment. Addison Wesley, 1993.

  9. D. Foster, D. Petravick and M. Ernst, “Grid File Access Proposal”, Technical Report, CERN, Fermilab, 2003. Retrieved from http://lcg.web.cern.ch/LCG/Documents/GDB/Documents 08.05.03 Foster.doc

  10. K. Czajkowski, D. Ferguson, I. Foster, J. Frey, S. Graham, T. Maguire, D. Snelling and S. Tuecke,“From Open Grid Services Infrastructure to WS-Resource Framework: Refactoring & Evolution”, Retrieved from http://www106.ibm.com/developerworks/library/ws-resource/ogsi to wsrf 1.0.pdf

  11. A. Shoshani, et al., “SRM Joint Functional Design”, Technical report, LBNL, JLAB, FNAL, CERN. Retrieved from http://sdm.lbl.gov/srm-g/doc/SRM.Joint.Functional.Design.Jan2002.pdf (SRM version 1), 2002.

  12. Sleepycat-Software, “Berkeley Database”, 2004. Retrieved from http://www.sleepycat.com/

  13. J. Gray and A. Reuter, Transaction Processing: Concepts and Techniques, Morgan Kaufmann, 1992.

  14. A. McNab, “GACL”, 2003. Retrieved from http://www.gridpp.ac.uk/gacl/

  15. L. Cornwall, J. Jensen, D. Kelsey and A. McNab, “EU DataGrid and GridPP Authorization and Access Control”, in Proceedings of the UK e-Science All Hands Conference, 2003. Retrieved from http://www.nesc.ac.uk/events/ahm2003/AHMCD/index authors.htm

  16. Sam, “SAM: Sequential Data Access Via Meta-Data”, 1998. Retrieved from http://d0db.fnal.gov/sam/

  17. A. Rajasekar, M. Wan, R. Moore, W. Schroeder, G. Kremenek, A. Jagatheesan, C. Cowart, B. Zhu, S.-Y. Chen and R. Olschanowsky, “Storage Resource Broker–Managing Distributed Data in a Grid,” Computer Society of India Journal Vol. 33, No. 4, 42–54, 2003.

    Google Scholar 

  18. M. Wan, A. Rajasekar, R. Moore and P. Andrew, “A Simple Mass Storage System for the SRB Data Grid,” In 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003.

  19. J. Bent, V. Venkataramani, N. LeRoy, A. Roy, J. Stanley, A.C. Arpaci-Dusseau, R.H. Arpaci-Dusseau and M. Livny, “Flexibility, Manageability, and Performance in a Grid Storage Appliance,” In Proceedings of the Eleventh IEEE Symposium on High Performance Distributed Computing, 2002. Retrieved from http://www.cs.wisc.edu/condor/nest/papers/nest-hpdc-02.ps

  20. DESY, “A Distributed Data Caching System,” 2001. Retrieved from http://dcache.desy.de/dCacheDesign.html

Download references

Author information

Authors and Affiliations

  1. Rutherford Appleton Laboratory, OX11 0QX, Chilton, Didcot, United Kingdom

    Jens G. Jensen, Tara Shah, Owen Synge, John Gordon, Glen Johnson & Regina Tam

Authors
  1. Jens G. Jensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tara Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Owen Synge
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. John Gordon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Glen Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Regina Tam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jens G. Jensen.

Additional information

This work is funded by EU contract IST-2000-25182.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jensen, J.G., Shah, T., Synge, O. et al. Enabling Grid Access to Mass Storage. J Grid Computing 3, 101–112 (2005). https://doi.org/10.1007/s10723-005-9000-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-005-9000-x

Key words

Search

Navigation