Skip to main content
SpringerLink
Log in

Storage Management Systems for Organizationally Distributed Environments PLGrid PLUS Case Study

  • Conference paper
  • First Online:
Parallel Processing and Applied Mathematics (PPAM 2013)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 8384))

Abstract

With the increasing amount of data the research community is facing problems with methods of effectively accessing, storing, and processing data in large scale and geographically distributed environments. This paper addresses major data management issues, in particular use cases and scenarios (on the basis of Polish research community organized around the PLGrid PLUS Project) and discusses architectures of data storage management systems available in both PL-Grid and other similar federated environments. On that basis, a concept of a new meta storage system, named VeilFS, is presented. The proposed system unifies file access methods for geographically distributed large scale systems and hides complexity of data access and management in such environments. However, it should be emphasized that the main purpose of this article is identification and discussion about users’ requirements and existing solutions. The VeilFS system will be described in detail in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. FUSE: Filesystem in Userspace. http://fuse.sourceforge.net/ (2013). Accessed 21 April 2013 (Online)

  2. PLATON Storage Service U4. http://www.storage.pionier.net.pl/ (2013). Accessed 21 April 2013 (Online)

  3. Tachyon Project. http://tachyon-project.org/ (2013). Accessed 21 April 2013

  4. Atkinson, M., et al.: Data-intensive research workshop report. Technical Report, e-Science Institute. http://research.nesc.ac.uk/files/DIRWS.pdf (2010)

  5. Baud, J.P.B., Caey, J., Lemaitre, S., Nicholson, C., Smith, D., Stewart, G.: LCG data management: from EDG to EGEE. In: UK e-Science All Hands Meeting, Nottingham, UK (2005)

    Google Scholar 

  6. Benedyczak, K., Rekawek, T., Rybicki, J., Schuller, B.: UNICORE data management: recent advancements. In: Romberg, M., Bala, P., Mller-Pfefferkorn, R., Mallmann, D. (eds.) UNICORE Summit 2011 Proceedings, Torun, Poland, 7–8 July 2011. IAS Series, vol. 9, pp. 24–27, Forschungszentrum Jülich (2011)

    Google Scholar 

  7. Braam, P.J., Schwan, P.: Lustre: the intergalactic file system. In: Ottawa Linux Symposium, June 2002

    Google Scholar 

  8. Dutka, Ł., Kitowski, J.: Application of component-expert technology for selection of data-handlers in CrossGrid. In: Kranzlmüller, D., Kacsuk, P., Dongarra, J., Volkert, J. (eds.) PVM/MPI 2002. LNCS, vol. 2474, pp. 25–32. Springer, Heidelberg (2002)

    Google Scholar 

  9. Grid File Access Library 2.0 official page. https://svnweb.cern.ch/trac/lcgutil/wiki/gfal2 (2013). Accessed 14 April 2013

  10. Hey, A., Tansley, S., Tolle, K.: The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, Redmond (2009)

    Google Scholar 

  11. Hunich, D., Muller-Pfefferkorn, R.: Managing large datasets with iRODS: a performance analysis. In: Proceedings of the 2010 International Multiconference on Computer Science and Information Technology (IMCSIT), pp. 647–654 (2010)

    Google Scholar 

  12. Kitowski, J., et al.: Polish computational research space for international scientific collaborations. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2011, Part I. LNCS, vol. 7203, pp. 317–326. Springer, Heidelberg (2012)

    Google Scholar 

  13. Kitowski, J., Turała, M., Wiatr, K., Dutka, Ł.: Pl-grid: foundations and perspectives of national computing infrastructure. In: Bubak, M., Szepieniec, T., Wiatr, K. (eds.) PL-Grid 2011. LNCS, vol. 7136, pp. 1–14. Springer, Heidelberg (2012)

    Google Scholar 

  14. Lustre. http://www.whamcloud.com/lustre/ (2013). Accessed 10 January 2013

  15. Mills, S., Lucas, S., Irakliotis, L., Rappa, M., Carlson, T., Perlowitz, B.: DEMYSTIFYING BIG DATA: a practical guide to transforming the business of Government. Technical report. http://www.ibm.com/software/data/demystifying-big-data/ (2012)

  16. OpenStack Object Storage (“Swift”). https://wiki.openstack.org/wiki/Swift (2013). Accessed 14 April 2013

  17. PLGrid Plus project. http://www.plgrid.pl/en#section-1t (2013). Accessed 14 April 2013

  18. Roblitz, T.: Towards implementing virtual data infrastructures a case study with iRODS. Comput. Sci. 13(4), 21–33 (2012). http://journals.agh.edu.pl/csci/article/view/43

  19. Shafer, J., Rixner, S., Cox, A.L.: The Hadoop distributed filesystem: balancing portability and performance. In: ISPASS, pp. 122–133, March 2010

    Google Scholar 

  20. Słota, R., Nikolow, D., Skitał, Ł., Kitowski, J.: Implementation of replication methods in the grid environment. In: Sloot, P.M.A., Hoekstra, A.G., Priol, T., Reinefeld, A., Bubak, M. (eds.) EGC 2005. LNCS, vol. 3470, pp. 474–484. Springer, Heidelberg (2005)

    Google Scholar 

  21. Słota, R.: Storage QOS provisioning for execution programming of data-intensive applications. Sci. Program. 20(1), 69–80 (2012)

    Google Scholar 

  22. Słota, R., Król, D., Skałkowski, K., Orzechowski, M., Nikolow, D., Kryza, B., Wrzeszcz, M., Kitowski, J.: A toolkit for storage QOS provisioning for data-intensive applications. Comput. Sci. 13(1), 63–73 (2012). http://journals.agh.edu.pl/csci/article/view/26

  23. Słota, R., Nikolow, D., Kitowski, J., Król, D., Kryza, B.: FiVO/QStorMan semantic toolkit for supporting data-intensive applications in distributed environments. Comput. Inform. 31(5), 1003–1024 (2012)

    Google Scholar 

  24. Stewart, G.A., Cameron, D., Cowan, G.A., McCance, G.: Storage and data management in EGEE. In: Proceedings of the fifth Australasian Symposium on ACSW frontiers, ACSW’07, Australia, vol. 68, pp. 69–77. Australian Computer Society Inc, Darlinghurst (2007)

    Google Scholar 

  25. Thain, D., Livny, M.: Parrot: an application environment for data-intensive computing. J. Parallel Distrib. Comput. Pract. 6(3), 9–18 (2005)

    Google Scholar 

  26. Worldwide LHC Computing Grid. http://wlcg.web.cern.ch/ (2013). Accessed 10 April 2013

  27. Zhou, K., Wang, H., Li, C.: Cloud storage technology and its application. ZTE Commun. 16(4), 24–27 (2010)

    MathSciNet  Google Scholar 

Download references

Acknowledgments

This research is supported partly by the European Regional Development Fund program no. POIG.02.03.00-00-096/10 as part of the PLGrid PLUS project and AGH-UST grants no. 11.11.230.015 and 15.11.230.097.

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Computer Science, Electronics and Telecommunications, AGH University of Science and Technology, Al. Mickiewicza 30, 30–059, Krakow, Poland

    Renata Słota, Darin Nikolow & Jacek Kitowski

  2. ACC Cyfronet AGH, AGH University of Science and Technology, Ul. Nawojki 11, 30–950, Krakow, Poland

    Łukasz Dutka, Michał Wrzeszcz, Bartosz Kryza, Dariusz Król & Jacek Kitowski

Authors
  1. Renata Słota
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Łukasz Dutka
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michał Wrzeszcz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bartosz Kryza
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Darin Nikolow
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dariusz Król
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jacek Kitowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Renata Słota .

Editor information

Editors and Affiliations

  1. Institute of Computer and Information Science, Czestochowa University of Technology, Czestochowa, Poland

    Roman Wyrzykowski

  2. University of Tennessee, Department of Computer Science, Knoxville, Tennessee, USA

    Jack Dongarra

  3. Institute of Computer and Information Science, Czestochowa University of Technology, Czestochowa, Poland

    Konrad Karczewski

  4. Technical University of Denmark Informatics and Mathematical Modelling, Kongens Lyngby, Denmark

    Jerzy Waśniewski

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Słota, R. et al. (2014). Storage Management Systems for Organizationally Distributed Environments PLGrid PLUS Case Study. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds) Parallel Processing and Applied Mathematics. PPAM 2013. Lecture Notes in Computer Science(), vol 8384. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-55224-3_68

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-55224-3_68

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-55223-6

  • Online ISBN: 978-3-642-55224-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation