Skip to main content
SpringerLink
Log in
Book cover

International Conference on Parallel Processing and Applied Mathematics

PPAM 2017: Parallel Processing and Applied Mathematics pp 471–480Cite as

Consistency Models for Global Scalable Data Access Services

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10777))

Abstract

Developing and deploying a global and scalable data access service is a challenging task. We assume that the globalization is achieved by creating and maintaining appropriate metadata while the scalability is achieved by limiting the number of entities taking part in keeping the metadata consistency. In this paper, we present different consistency and synchronization models for various metadata types chosen for implementation of global and scalable data access service.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Baud, J.P.B., Casey, J., Lemaitre, S., Nicholson, C., Smith, D., Stewart, G.: LCG data management: from EDG to EGEE. In: UK e-Science All Hands Meeting, Nottingham, UK (2005). http://www.allhands.org.uk/2005/proceedings/papers/475.pdf

  2. Drago, I., Mellia, M., Munafo, M., Sperotto, A., Sadre, R., Pras, A.: Inside dropbox: understanding personal cloud storage services. In: Proceedings of the 2012 ACM Conference on Internet Measurement, IMC 2012, pp. 481–494. ACM, New York (2012)

    Google Scholar 

  3. Dutka, Ł., Wrzeszcz, M., Lichoń, T., Słota, R., Zemek, K., Trzepla, K., Opiola, Ł., Słota, R., Kitowski, J.: Onedata - a step forward towards globalization of data access for computing infrastructures. In: Proceedings of the International Conference on Computational Science, ICCS 2015, Computational Science at the Gates of Nature, Reykjavík, Iceland, pp. 2843–2847, 1–3 June 2015

    Google Scholar 

  4. Hünich, D., Müller-Pfefferkorn, R.: Managing large datasets with iRODS - a performance analysis. In: Proceedings of the 2010 International Multiconference on Computer Science and Information Technology (IMCSIT), pp. 647–654. IEEE (2010)

    Google Scholar 

  5. Pacheco, L., Halalai, R., Schiavoni, V., Pedone, F., Riviere, E., Felber, P.: GlobalFS: a strongly consistent multi-site file system. In: Proceedings of the 35th IEEE Symposium on Reliable Distributed Systems, SRDS 2016, Budapest, Hungary, 26–29 September 2016, pp. 147–156. IEEE Computer Society (2016). http://dblp.uni-trier.de/db/conf/srds/srds2016.html#PachecoHSPRF16

  6. Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), MSST 2010, pp. 1–10. IEEE Computer Society, Washington (2010). https://doi.org/10.1109/MSST.2010.5496972

  7. Slota, R., Nikolow, D., Skalkowski, K., Kitowski, J.: Management of data access with quality of service in PL-Grid environment. Comput. Inf. 31(2), 463–479 (2012). http://www.cai.sk/ojs/index.php/cai/article/view/950

    Google Scholar 

  8. Słota, R., Nikolow, D., Skitał, Ł., Kitowski, J.: Implementation of replication methods in the grid environment. In: Sloot, P.M.A., Hoekstra, A.G., Priol, T., Reinefeld, A., Bubak, M. (eds.) EGC 2005. LNCS, vol. 3470, pp. 474–484. Springer, Heidelberg (2005). https://doi.org/10.1007/11508380_49

    Chapter  Google Scholar 

  9. Słota, R., Skitał, Ł., Nikolow, D., Kitowski, J.: Algorithms for automatic data replication in grid environment. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Waśniewski, J. (eds.) PPAM 2005. LNCS, vol. 3911, pp. 707–714. Springer, Heidelberg (2006). https://doi.org/10.1007/11752578_85

    Chapter  Google Scholar 

  10. Thain, D., Livny, M.: Parrot: an application environment for data-intensive computing. Scalable Comput. Pract. Exp. 6(3), 9–18 (2005)

    Google Scholar 

  11. Viotti, P., Vukolić, M.: Consistency in non-transactional distributed storage systems. ACM Comput. Surv. 49(1), 19:1–19:34 (2016). https://doi.org/10.1145/2926965

    Article  Google Scholar 

  12. Wang, F., Oral, S., Shipman, G., Drokin, O., Wang, T., Huang, I.: Understanding Lustre Filesystem Internals. Technical report ORNL/TM-2009/117, Oak Ridge National Lab., National Center for Computational Sciences (2009)

    Google Scholar 

  13. Weil, S.A., Brandt, S.A., Miller, E.L., Long, D.D.E., Maltzahn, C.: Ceph: a scalable, high-performance distributed file system. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI), pp. 307–320 (2006)

    Google Scholar 

  14. Weil, S.A., Brandt, S.A., Miller, E.L., Maltzahn, C.: CRUSH: controlled, scalable, decentralized placement of replicated data. In: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, SC 2006, Tampa. ACM, New York (2006). https://doi.org/10.1145/1188455.1188582

  15. Wrzeszcz, M., Trzepla, K., Słota, R., Zemek, K., Lichoń, T., Opioła, Ł., Nikolow, D., Dutka, Ł., Słota, R., Kitowski, J.: Metadata organization and management for globalization of data access with Onedata. In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K., Kitowski, J., Wiatr, K. (eds.) PPAM 2015. LNCS, vol. 9573, pp. 312–321. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32149-3_30

    Chapter  Google Scholar 

  16. Zhang, J., Wu, Y., Chung, Y.C.: PROAR: a weak consistency model for Ceph. In: 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS), pp. 347–353 (2016)

    Google Scholar 

Download references

Acknowledgements

This research is supported partly by the European Regional Development Fund program no. POIG.02.03.00-12-137/13 as part of the PLGrid Core. R. G. Słota, D. Nikolow and J. Kitowski acknowledge AGH-UST statutory Grant no. 11.11.230.337. Support by IndigoDC project no. RIA 653549 is also acknowledged by Ł. Dutka, M. Wrzeszcz. T. Lichoń acknowledges Polish Ministry of Science and Higher Education under AGH University of Science and Technology, Faculty of Computer Science, Electronics and Telecommunications statutory project no. 15.11.230.317, and owes special thanks to ACK Cyfronet AGH computing center for providing computing and storage infrastructure for this research.

Author information

Authors and Affiliations

  1. Academic Computer Centre Cyfronet AGH, AGH University of Science and Technology, Krakow, Poland

    Michał Wrzeszcz, Tomasz Lichoń, Łukasz Dutka & Jacek Kitowski

  2. Department of Computer Science, Faculty of Computer Science, Electronics and Telecommunications, AGH University of Science and Technology, Krakow, Poland

    Michał Wrzeszcz, Darin Nikolow, Tomasz Lichoń, Rafał Słota, Renata G. Słota & Jacek Kitowski

Authors
  1. Michał Wrzeszcz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Darin Nikolow
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomasz Lichoń
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rafał Słota
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Łukasz Dutka
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Renata G. Słota
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jacek Kitowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michał Wrzeszcz .

Editor information

Editors and Affiliations

  1. Czestochowa University of Technology, Czestochowa, Poland

    Roman Wyrzykowski

  2. University of Tennessee, Knoxville, Tennessee, USA

    Jack Dongarra

  3. University of Southern California, Marina Del Rey, California, USA

    Ewa Deelman

  4. Czestochowa University of Technology, Czestochowa, Poland

    Konrad Karczewski

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wrzeszcz, M. et al. (2018). Consistency Models for Global Scalable Data Access Services. In: Wyrzykowski, R., Dongarra, J., Deelman, E., Karczewski, K. (eds) Parallel Processing and Applied Mathematics. PPAM 2017. Lecture Notes in Computer Science(), vol 10777. Springer, Cham. https://doi.org/10.1007/978-3-319-78024-5_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-78024-5_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-78023-8

  • Online ISBN: 978-3-319-78024-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation