Abstract
In this article, the authors present a solution to the problem of accessing data in organizationally distributed environments, such as Grids and Clouds, in a uniform and efficient manner. An overview of existing storage solutions is described, in particular high-performance filesystems and data management systems, with regard to the provided functionality, scalability and configuration elasticity. Next, a novel solution, called VeilFS, is described in terms of objectives to attain, its architecture and current implementation status. In particular, the mechanisms used for achieving a desired level of performance and fault-tolerance are discussed and preliminary overhead tests are presented.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ceph Filesystem web site, http://ceph.com/docs/next/cephfs/
Dropbox web site, https://www.dropbox.com/
GlusterFS community web site, http://www.gluster.org/about/
Nfs version 3 protocol specification, http://tools.ietf.org/html/rfc1813
Scality web site, http://www.scality.com/products/what-is-ring/
Sysbench: a system performance benchmark, http://sysbench.sourceforge.net/index.html
Gantz, J., Reinsel, D.: The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East (2012), http://www.emc.com/leadership/digital-universe/index.htm
Hunich, D., Muller-Pfefferkorn, R.: Managing Large Datasets with iRODS: a Performance Analysis. In: Proceedings of the 2010 International Multiconference on Computer Science and Information Technology (IMCSIT), pp. 647–654 (2010)
Kitowski, J., Dutka, Ł., Mosurska, Z., Pająk, R., Sterzel, M., Szepieniec, T.: Development of Polish Infrastructure for Advanced Scientific Research – Status and Current Achievements. In: Proc. of IEEE Conf. 12th Inter. Symposium on Parallel and Distributed Computing (ISPDC 2013), Bucharest, Romania, pp. 34–41 (2013)
Kryza, B., Król, D., Wrzeszcz, M., Dutka, Ł., Kitowski, J.: Interactive cloud data farming environment for military mission planning support. Computer Science 13(3), 89–100 (2012), https://journals.agh.edu.pl/csci/article/view/19
Mills, S., Lucas, S., Irakliotis, L., Rappa, M., Carlson, T., Perlowitz, B.: DEMYSTIFYING BIG DATA: A Practical Guide to Transforming the Business of Government. Technical report (2012), http://www.ibm.com/software/data/demystifying-big-data/
Roblitz, T.: Towards Implementing Virtual Data Infrastructures – a Case Study with iRODS. Computer Science 13(4) (2012), http://journals.agh.edu.pl/csci/article/view/43
Słota, R., Dutka, Ł., Wrzeszcz, M., Kryza, B., Nikolow, D., Król, D., Kitowski, J.: Storage Systems for Organizationally Distributed Environments – PLGrid Plus Case Study. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasśniewski, J. (eds.) PPAM 2013, Part I. LNCS, pp. 724–733. Springer, Heidelberg (2013)
Słota, R., Król, D., Skałkowski, K., Kryza, B., Nikołow, D., Orzechowski, M., Kitowski, J.: A Toolkit for Storage QoS Provisioning for Data-Intensive Applications. In: Bubak, M., Szepieniec, T., Wiatr, K. (eds.) PL-Grid 2011. LNCS, vol. 7136, pp. 157–170. Springer, Heidelberg (2012)
Słota, R., Nikolow, D., Kitowski, J., Król, D., Kryza, B.: FiVO/QStorMan Semantic Toolkit for Supporting Data-Intensive Applications in Distributed Environments. Computing and Informatics 31(5), 1003–1024 (2012), http://dblp.uni-trier.de/db/journals/cai/cai31.html#SlotaNK0K12
Szepieniec, T., Tomanek, M., Radecki, M., Szopa, M., Bubak, M.: Implementation of Service Level Management in PL-Grid Infrastructure. In: Bubak, M., Szepieniec, T., Wiatr, K. (eds.) PL-Grid 2011. LNCS, vol. 7136, pp. 171–181. Springer, Heidelberg (2012)
Thain, D., Livny, M.: Parrot: an Application Environment for Data-Intensive Computing. Journal of Parallel and Distributed Computing Practices, 9–18 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Dutka, Ł., Słota, R., Wrzeszcz, M., Król, D., Kitowski, J. (2014). Uniform and Efficient Access to Data in Organizationally Distributed Environments. In: Bubak, M., Kitowski, J., Wiatr, K. (eds) eScience on Distributed Computing Infrastructure. Lecture Notes in Computer Science, vol 8500. Springer, Cham. https://doi.org/10.1007/978-3-319-10894-0_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-10894-0_13
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10893-3
Online ISBN: 978-3-319-10894-0
eBook Packages: Computer ScienceComputer Science (R0)