Abstract
Nowadays, data-intensive scientific research needs storage capabilities that enable efficient data sharing. This is of great importance for many scientific domains such as the Virtual Physiological Human. In this paper, we introduce a solution that federates a variety of systems ranging from file servers to more sophisticated ones used in clouds or grids. Our solution follows a client-centric approach that loosely couples a variety of data resources that may use different technologies such as Openstack-Swift, iRODS, GridFTP, and may be geographically distributed. It is implemented as a lightweight service which does not require installation of a software on the resources it uses. In this way we are able to efficiently use heterogeneous storage resources, reduce the usage complexity of multiple storage resources, and avoid vendor lock-in in case of cloud storage. To demonstrate the usability of our approach we performed a number of experiments that assess the performance and functionality of the developed system.
Chapter PDF
Similar content being viewed by others
References
CERN: Worldwide LHC Computing Grid (March 2012)
Begeman, K., Belikov, A.N., Boxhoorn, D.R., Dijkstra, F., Holties, H., Meyer-Zhao, Z., Renting, G.A., Valentijn, E.A., Vriend, W.J.: Lofar information system. Future Gener. Comput. Syst. 27(3), 319–328 (2011)
Hey, T., Tansley, S., Tolle, K.: The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft (2009)
Benkner, S., Bisbal, J., Engelbrecht, G., Hose, R.D., Kaniovskyi, Y., Koehler, M., Pedrinaci, C., Wood, S.: Towards collaborative data management in the VPH-share project. In: Alexander, M., et al. (eds.) Euro-Par 2011, Part I. LNCS, vol. 7155, pp. 54–63. Springer, Heidelberg (2012)
Belloum, A., Inda, M., Vasunin, D., Korkhov, V., Zhao, Z., Rauwerda, H., Breit, T., Bubak, M., Hertzberger, L.: Collaborative e-science experiments and scientific workflows. IEEE Internet Computing 15(4), 39–47 (2011)
Kurze, T., Klems, M., Bermbach, D., Lenk, A., Tai, S., Kunze, M.: Cloud federation. In: Proceedings of the 2nd International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011), IARIA (September 2011)
Neokleous, K., Dikaiakos, M., Fragopoulou, P., Markatos, E.: Grid reliability: A study of failures on the egee infrastructure. In: Gorlatch, S., Bubak, M., Priol, T. (eds.) Proceedings of the CoreGRID Integration Workshop 2006, pp. 165–176 (October 2006)
Foster, I.: The virtual data grid: a new model and architecture for data-intensive collaboration. In: Proceedings of the 15th International Conference on Scientific and Statistical Database Management, SSDBM 2003, p. 11. IEEE Computer Society, Washington, DC (2003)
Rajasekar, A., Wan, M., Moore, R., Kremenek, G., Guptil, T.: Data grids, collections, and grid bricks. In: Proceedings of the 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies (MSST 2003), pp. 2–9 (2003)
Sánchez, A., Pérez, M.S., Karasavvas, K., Herrero, P., Pérez, A.: Mapfs-dai, an extension of ogsa-dai based on a parallel file system. Future Generation Computer Systems 23(1), 138–145 (2007)
Abu-Libdeh, H., Princehouse, L., Weatherspoon, H.: Racs: a case for cloud storage diversity. In: Proceedings of the 1st ACM Symposium on Cloud Computing, SoCC 2010, pp. 229–240. ACM, New York (2010)
Broberg, J., Buyya, R., Tari, Z.: MetaCDN: Harnessing Storage Clouds’ for high performance content delivery. Journal of Network and Computer Applications 32(5), 1012–1022 (2009) Next Generation Content Networks
Testi, D., Quadrani, P., Viceconti, M.: PhysiomeSpace: digital library service for biomedical data. Physical and Engineering Sciences 368(1921), 2853–2861 (2010)
Rajasekar, A., Wan, M., Moore, R., Schroeder, W.: A prototype rule-based distributed data management system. In: HPDC Workshop on Next Generation Distributed Data Management (2006)
Wan, M., Moore, R., Rajasekar, A.: Integration of cloud storage with data grids. In: Proceedings of ICVCI 2009, 3rd International Conference on the Virtual Computing Initiative, Research Triangle Park, NC, USA (2009)
Nowakowski, P., Bartynski, T., Gubala, T., Harezlak, D., Kasztelnik, M., Meizner, M.M.J., Bubak, M.: Cloud platform for vph applications. In: 8th International Conference on eScience 2012 (October 2012)
VL-e: VBrowser web site (March 2012), http://www.vl-e.nl/vbrowser
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Koulouzis, S., Vasyunin, D., Cushing, R., Belloum, A., Bubak, M. (2014). Cloud Data Federation for Scientific Applications. In: an Mey, D., et al. Euro-Par 2013: Parallel Processing Workshops. Euro-Par 2013. Lecture Notes in Computer Science, vol 8374. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-54420-0_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-54420-0_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-54419-4
Online ISBN: 978-3-642-54420-0
eBook Packages: Computer ScienceComputer Science (R0)