Skip to main content
SpringerLink
Log in

Seamless Access to Decentralized Storage Services in Computational Grids via a Virtual File System

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

This paper describes a novel technique for establishing a virtual file system that allows data to be transferred user-transparently and on-demand across computing and storage servers of a computational grid. Its implementation is based on extensions to the Network File System (NFS) that are encapsulated in software proxies. A key differentiator between this approach and previous work is the way in which file servers are partitioned: while conventional file systems share a single (logical) server across multiple users, the virtual file system employs multiple proxy servers that are created, customized and terminated dynamically, for the duration of a computing session, on a per-user basis. Furthermore, the solution does not require modifications to standard NFS clients and servers. The described approach has been deployed in the context of the PUNCH network-computing infrastructure, and is unique in its ability to integrate unmodified, interactive applications (even commercial ones) and existing computing infrastructure into a network computing environment. Experimental results show that: (1) the virtual file system performs well in comparison to native NFS in a local-area setup, with mean overheads of 1 and 18%, for the single-client execution of the Andrew benchmark in two representative computing environments, (2) the average overhead for eight clients can be reduced to within 1% of native NFS with the use of concurrent proxies, (3) the wide-area performance is within 1% of the local-area performance for a typical compute-intensive PUNCH application (SimpleScalar), while for the I/O-intensive application Andrew the wide-area performance is 5.5 times worse than the local-area performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A.D. Alexandrov, M. Ibel, K.E. Schauser and C.J. Scheiman, Ufo: A personal global file system based on user-level extensions to the operating system, ACM Transactions on Computer Systems 16(3) (August 1998) 207–233.

    Google Scholar 

  2. A. Bayucan, R.L. Henderson, C. Lesiak, B. Mann, T. Proett and D. Tweten, Portable batch system: External reference specification, Technical report, MRJ Technology Solutions (November 1999).

  3. D. Burger and T.M. Austin, The simplescalar tool set, version 2.0, Technical report 1342, Computer Sciences Department, University of Wisconsin at Madison (June 1997).

    Google Scholar 

  4. K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith and S. Teucke, A resource management architecture for metacomputing systems, in: Proceedings of the 4th Workshop on Job Scheduling Strategies for Parallel Processing (1998). Held in conjunction with the International Parallel and Distributed Processing Symposium.

  5. A.S. Grimshaw, W.A. Wulf et al., The legion vision of a worldwide virtual computer, Communications of the ACM 40(1) (1997).

  6. R.L. Henderson and D. Tweten, Portable batch system: Requirement specification, Technical report, NAS Systems Division, NASA Ames Research Center (August 1998).

  7. J.H. Howard, M.L. Kazar, S.G. Menees, D.A. Nichols, M. Satyanarayanan, R.N. Sidebotham and M.J. West, Scale and performance of a distributed file system, ACM Transactions on Computer Systems 6(1) (February 1988) 51–81.

    Google Scholar 

  8. N.H. Kapadia, R.J.O. Figueiredo and J.A.B. Fortes, PUNCH:Web portal for running tools, IEEE Micro (May-June 2000) 38–47.

  9. N.H. Kapadia, R.J.O. Figueiredo and J.A.B. Fortes, Enhancing the scalability and usability of computational grids via logical user accounts and virtual file systems, in: Proceedings of the Heterogeneous Computing Workshop (HCW) at the International Parallel and Distributed Processing Symposium (IPDPS), San Francisco, CA (April 2001).

  10. N.H. Kapadia and J.A.B. Fortes, PUNCH: An architecture for webenabled wide-area network-computing, Cluster Computing: The Journal of Networks, Software Tools and Applications 2(2), Special Issue on High Performance Distributed Computing (September 1999) 153–164.

    Google Scholar 

  11. M. Litzkow, M. Livny and M.W. Mutka, Condor – a hunter of idle workstations, in: Proceedings of the 8th International Conference on Distributed Computing Systems (June 1988) pp. 104–111.

  12. D. Mazières, M. Kaminsky, M.F. Kaashoek and E. Witchel, Separating key management from file system security, in: Proceedings of the 17th ACM Symposium on Operating Systems Principles (SOSP), Kiawah Island, SC (December 1999).

  13. J.H. Morris, M. Satyanarayanan, M.H. Conner, J.H. Howard, D.S. Rosenthal and F.D. Smith, Andrew: A distributed personal computing environment, Communications of the ACM 29(3) (1986) 184–201.

    Google Scholar 

  14. B. Pawlowski, C. Juszczak, P. Staubach, C. Smith, D. Lebel and D. Hitz, NFS version 3 design and implementation, in: Proceedings of the USENIX Summer Technical Conference (1994).

  15. H.C. Rao and L.L. Peterson, Accessing files on the internet: The jade file system, IEEE Transactions on Software Engineering 19(6) (1993) 613–625.

    Google Scholar 

  16. D. Royo, N.H. Kapadia, J.A.B. Fortes and L. Diaz de Cerio, Active yellow pages: A pipelined resource management architecture for wide-area network computing, in: Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing (HPDC'01), San Francisco, CA (August 2001).

  17. A.Z. Spector and M.L. Kazar, Wide area file service and the AFS experimental system, Unix Review 7(3) (1989).

  18. D. Thain, J. Basney, S.-C. Son and M. Livny, The kangaroo approach to data movement on the grid, in: Proceedings of the 2001 IEEE International Conference on High-Performance Distributed Computing (HPDC) (August 2001) pp. 325–333.

  19. B.S. White, A.S. Grimshaw and A. Nguyen-Tuong, Grid-based file access: The legion I/O model, in: Proceedings of the 9th IEEE International Symposium on High Performance Distributed Computing (HPDC'00), Pittsburgh, PA (August 2000) pp. 165–173.

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, 32611, USA

    Renato J. Figueiredo

  2. Capital One Services, Inc., Glen Allen, VA, 23060, USA

    Nirav Kapadia

  3. Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, 32611, USA

    José A.B. Fortes

Authors
  1. Renato J. Figueiredo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nirav Kapadia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. José A.B. Fortes
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Figueiredo, R.J., Kapadia, N. & Fortes, J.A. Seamless Access to Decentralized Storage Services in Computational Grids via a Virtual File System. Cluster Computing 7, 113–122 (2004). https://doi.org/10.1023/B:CLUS.0000018561.64239.d3

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:CLUS.0000018561.64239.d3

Search

Navigation