A classification of file placement and replication methods on grids

https://doi.org/10.1016/j.future.2013.02.006Get rights and content

Abstract

This paper presents a classification of file placement and replication methods on grids. The study is motivated by file transfer issues encountered in the Virtual Imaging Platform deployed on the European Grid Infrastructure. Approaches proposed in the last 6 years are classified using taxonomies of replication process, replication optimization, file models, resource models and replication validation. Most existing approaches implement file replication as a middleware service, using dynamic strategies. Production approaches are slightly different than works evaluated in simulation or in controlled conditions which (i) mostly assumes simplistic file models (undistinguished read-only files), (ii) rely on elaborated access patterns, (iii) assume clairvoyance of the infrastructure parameters and (iv) study file availability less than other metrics but insist on cost.

Highlights

► We propose a classification of file replication and placement methods on grids. ► We review works published during the past 6 years. ► We highlight singularities of approaches deployed in production.

Introduction

Data management is a critical component of distributed systems and in particular grids [1], [2], [3]. Usually, applications do not seek only for computing power but also have stringent requirements concerning data sharing, storage and transfers. For instance, medical image analysis often involves pipelines or workflows retrieving images from an indexed database, staging-in image files to the computing resources and storing results. This requires that file storage and transfers are secure, reliable and efficient.

Among data management issues, file placement and replication were studied since the early ages of grid computing.1 However, applications running on production grids still report low transfer reliability and performance. In [4], data transfers between grid sites account for 86% of job failures and [5], [6] mention that data-related errors have serious impact on the performance. Such data transfer issues originate in the unavailability of storage machines and in network connectivity issues between sites. They not only impact computing jobs but also end-users in their transfer of input and output data. Groups of users, a.k.a. Virtual Organizations (VOs), also have to properly deal with file placement in order (i) to control available space on storage machines and (ii) to reduce the impact of storage decommissioning.

File replication is a common solution to improve the reliability and performance of data transfers. It consists in storing several instances of the same file on different resources. File placement aims at choosing these resources appropriately. For instance, [7] reports significant improvements obtained by replication on metrics such as makespan, job running time and consumed bandwidth. Yet, the overhead of file replication also has to be considered [8].

Many file management strategies were proposed but none was adopted in large-scale production infrastructures. Although the European Grid Infrastructure (EGI) reports average availability and reliability ranging from 84% to 96% with a target of 90%2 (which clearly motivates the need for efficient data management strategies), deployed middleware systems still offer very low-level data management facilities, putting file placement and replication decisions in the hands of the application or VO managers. For instance, high-energy physics experiments3 still do not expect the middleware to provide any data placement system due to the complexity and variety of application use-cases [9]. In practice, they rely on substantial manpower to implement data placement policies: system administrators constantly monitor the status of the datasets and storage systems, triggering transfers when needed [10].

Such manual operations are not affordable for smaller user communities. Instead, automatic file placement methods should be made available to enable reliable use of grid infrastructures. This paper aims at providing a structured outline of the existing file placement and replication methods. It extends the taxonomies of replication architecture and strategy presented in [11]. Our study is motivated by the practical example of the Virtual Imaging Platform (VIP) [12], a science-gateway deployed on EGI. On this platform, file transfer performance and reliability are a real issue, and we are looking into strategies to improve them. Section 2 describes file transfer issues VIP. Section 3 then formalizes the problem and Section 4 presents a taxonomy of the approaches. Existing works are classified according to this taxonomy in Section 5, and production systems are discussed in Section 6. This paper focuses on grids as aggregations of computing and storage clusters: desktop grids, volunteer computing, clouds, peer-to-peer systems and parallel file systems used for clusters (NFS,4 Lustre,5 PVFS [13], Hadoop [14], GFS [15]) are out of scope.

Section snippets

File transfers in the Virtual Imaging Platform

Applications in VIP are described as workflows generating jobs distributed to EGI sites. When they reach a computing resource, jobs download their input files from storage elements (SEs), run the application, and upload results to storage elements. To motivate our study of file placement and replication, we studied the performance and reliability of job file transfers in VIP on a data-set of 489 workflow executions started in November 2012 on EGI’s biomed6 Virtual

Problem formalization

We define the file placement problem as follows. Given a set of n files and p storage elements, a storage configuration is defined by a matrix S of n×p boolean values such that Si,j=1 if and only if file i is stored on storage element j. The set of storage matrices is noted S.

Implementing a storage matrix S means performing all the required file transfers and replications so that the status of the infrastructure is described by S. This has a cost defined by the migration function: ϕ:S×SR+S,Pϕ(

Taxonomies of file placement

To create the taxonomies we use the method proposed and used in [16] for a review of multi-criteria grid workflow scheduling. It is described as follows (see beginning of Section 3 in [16]):

[To analyze the file placement problem], several important facets of the problem are considered. Each facet describes the [file placement] problem from a different perspective. For every facet we propose a certain taxonomy which classifies different [file placement] approaches into different possibleclasses

Classification of existing methods

The taxonomies presented above extend the taxonomies of replication architecture and strategy in the survey of data grids presented in [11]: a new validation facet is introduced, clairvoyance predicate is added to the resource model, optimization target is added to the optimization method, replication actor is added to the process implementation, dynamic methods are classified as online and infrastructure-adapting, two new classes are added to the file type predicate and three new classes are

File replication methods in production systems

Most high-performance computing infrastructures (HPC) use parallel file systems which completely hide data management operations. In these file systems, data replication is automatic, but it cannot exploit application-level information such as file type or access pattern. Conversely, high-throughput computing (HTC) infrastructures delegate data management to middleware or application-level services. File replication policies can be defined according to applications’ characteristics, but they

Conclusion

We presented a formalization of the file placement and replication problem on grids and we proposed a classification of recent approaches based on taxonomies of the file model, the resource model, the replication process, the optimization method and the replication validation. These taxonomies were described using an RDF-like subject–predicate–object notation as described in [16]. A total of 45 classes were identified.

A clear gap was identified between production approaches and the ones staying

Acknowledgments

This work is supported by the National Natural Science Foundation of China (60777004) and International S&T Cooperation Project of China (2007DFB30320). It is also supported by the French National Grid Initiative, “France-Grilles”.15

Jianwei Ma is a Ph.D. candidate of Harbin Institute of Technology. He obtain his master’s degree in 2003 on Automatic control theory and engineering from Heilongjiang University. His research is focusing on medical data management in data grid.

References (58)

  • Dagmar Krefting, Ralf Luetzkendorf, Kathrin Peter, Johannes Bernarding, Performance analysis of diffusion tensor...
  • H. Sato, S. Matsuoka, T. Endo, N. Maruyama, Access-pattern and bandwidth aware file replication algorithm in a grid...
  • L. Tuura et al.

    Scaling cms data transfer system for lhc start-up

    Journal of Physics: Conference Series

    (2008)
  • J. Rehn, T. Barrass, D. Bonacorsi, J. Hernandez, I. Semeniouk, L. Tuura, Y. Wu, Phedex high-throughput data transfer...
  • Srikumar Venugopal et al.

    A taxonomy of data grids for distributed data sharing, management, and processing

    ACM Computing Surveys

    (2006)
  • T. Glatard et al.

    A virtual imaging platform for multi-modality medical image simulation

    IEEE Transactions on Medical Imaging

    (2013)
  • Philip H. Carns et al.

    Pvfs: a parallel file system for linux clusters

  • Konstantin Shvachko, Hairong Kuang, Sanjay Radia, Robert Chansler, The hadoop distributed file system. in: Mass Storage...
  • Sanjay Ghemawat et al.

    The google file system

  • Domenico Talia et al.

    Taxonomies of the multi-criteria grid workflow scheduling problem

  • Gurmeet Singh et al.

    Optimizing workflow data footprint

    Scientific Programming

    (2007)
  • W.H. Bell, D.G. Cameron, R. Carvajal-Schiaffino, A.P. Millar, K. Stockinger, F. Zini, Evaluation of an economy-based...
  • Kavitha Ranganathan, Ian Foster, Identifying dynamic replication strategies for a high-performance data grid, in: Craig...
  • M.M. Deris, J.H. Abawajy, H.M. Suzuri, An efficient replicated data access approach for large-scale distributed...
  • Ali H. Elghirani, Riky Subrata, Albert Y. Zomaya, A proactive non-cooperative game-theoretic framework for data...
  • Ali Elghirani, Riky Subrata, Albert Y. Zomaya, Intelligent scheduling and replication in datagrids: a synergistic...
  • M. Carman, F. Zini, L. Serafini, K. Stockinger, Towards an economy-based optimisation of file access and replication on...
  • William H. Bell et al.

    Optorsim: a grid simulator for studying dynamic data replication strategies

    International Journal of High Performance Computing Applications

    (2003)
  • Susan V. Vrbsky, Ming Lei, Karl Smith, Jeff Byrd, Data replication and power consumption in data grids, in: IEEE...
  • Cited by (0)

    Jianwei Ma is a Ph.D. candidate of Harbin Institute of Technology. He obtain his master’s degree in 2003 on Automatic control theory and engineering from Heilongjiang University. His research is focusing on medical data management in data grid.

    Wanyu Liu is leading the HIT-INSA Sino-French Research Center for Biomedical Imaging at the Harbin Institute of Technology in Harbin, China.

    Tristan Glatard obtained a Ph.D. in grid computing applied to medical image analysis from the University of Nice Sophia-Antipolis in 2007. He was a post-doc at the University of Amsterdam in 2008. He is now a researcher at CNRS Creatis in Lyon, working on distributed systems for medical imaging applications.

    View full text