Skip to main content
SpringerLink
Log in

File replication, maintenance, and consistency management services in data grids

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Data replication and consistency refer to the same data being stored in distributed sites, and kept consistent when one or more copies are modified. A good file maintenance and consistency strategy can reduce file access times and access latencies, and increase download speeds, thus reducing overall computing times. In this paper, we propose dynamic services for replicating and maintaining data in grid environments, and directing replicas to appropriate locations for use. To address a problem with the Bandwidth Hierarchy-based Replication (BHR) algorithm, a strategy for maintaining replicas dynamically, we propose the Dynamic Maintenance Service (DMS). We also propose a One-way Replica Consistency Service (ORCS) for data grid environments, a positive approach to resolving consistency maintenance issues we hope will strike a balance between improving data access performance and replica consistency. Experimental results show that our services are more efficient than other strategies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Allcock B, Bester J, Bresnahan J, Chervenak A, Foster I, Kesselman C, Meder S, Nefedova V, Quesnel D, Tuecke S (2002) Data management and transfer in high-performance computational grid environments. Parallel Comput 28(5):749–771

    Article  Google Scholar 

  2. Allcock B, Bester J, Bresnahan J, Chervenak A, Foster I, Kesselman C, Meder S, Nefedova V, Quesnel D, Tuecke S (2001) Secure, efficient data transport and replica management for high-performance data-intensive computing. In: Proceedings of the eighteenth IEEE symposium on mass storage systems and technologies, pp 13–28

  3. CERN. http://public.web.cern.ch/Public/Welcome.html

  4. Chang RS, Chang JS (2006) Adaptable replica consistency service for data grids. In: Proceeding of the third international conference of information technology (ITNG’06), pp 646–651

  5. Chervenak A, Foster I, Kesselman C, Salisbury C, Tuecke S (2001) The data grid: towards an architecture for the distributed management and analysis of large scientific datasets. J Netw Comput Appl 23:187–200

    Article  Google Scholar 

  6. Czajkowski K, Fitzgerald S, Foster I, Kesselman C (2001) Grid information services for distributed resource sharing, In: Proceedings of the tenth IEEE international symposium on high-performance distributed computing (HPDC-10’01), August 2001, pp 181–194

  7. Domenici A, Donno F, Pucciani G, Stockinger H, Stockinger K (2004) Replica consistency in a data grid. Nucl Instr Methods Phys Res A 534(1–2):24–28

    Article  Google Scholar 

  8. Düllmann D, Hoschek W, Martinez JJ, Segal B (2001) Models for replica synchronisation and consistency in a data grid. In: Proceedings of the 10th IEEE international symposium on high performance distributed computing (HPDC-10’01), October 2001, pp 67

  9. Fathali J (2006, to appear) A genetic algorithm for the p-median problem with pos/neg weights. Appl Math Comput 8 (August)

  10. Fisher ML (1981) The Lagrangian relaxation method for solving integer programming problems. Manag Sci 27:1–18

    Article  MATH  Google Scholar 

  11. Foster I (2002) The grid: a new infrastructure for 21st century science. Phys Today 55(2):42–47

    Article  Google Scholar 

  12. Foster I, Kesselman C (1999) The grid 2: blueprint for a new computing infrastructure, 2nd edn. Morgan Kaufmann, San Mateo (Elsevier series in grid computing)

    Google Scholar 

  13. Foster I, Kesselman C (1997) Globus: a metacomputing infrastructure toolkit. Int J Supercomput Appl High Perform Comput 11(2):115–128

    Article  Google Scholar 

  14. Huang CQ, Xu FY, Hu XY (2006) Massive data oriented replication algorithms for consistency maintenance in data grids. ICCS 2006, Part I, LNCS 3991, pp 838–841

  15. Hu JF, Xiao N, Zhao YJ, Fu W (2005) An asynchronous replica consistency model in data grid. In: Parallel and distributed processing and applications (ISPA 2005 workshops), pp 475–484

  16. Jackson LE, Rouskas GN, Stallmann MFM (2007) The directional p-median problem: definition, complexity, and algorithms. Eur J Oper Res 179:1097–1108. http://people.engr.ncsu.edu/mfms/Publications/2007-EJOR-Jackson.pdf

    Article  MATH  MathSciNet  Google Scholar 

  17. Java CoG. http://www-unix.globus.org/cog/

  18. NWS. http://nws.cs.ucsb.edu/

  19. OptorSim. http://edg-wp2.web.cern.ch/edg-wp2/optimization/optorsim.html

  20. Park SM, Kim JH, Ko YB, Yoon W-S (2003) Dynamic data grid replication strategy based on Internet hierarchy. In: The second international workshop on grid and cooperative computing (GCC2003), pp 838–846

  21. Park SM, Kim JH (2003) Chameleon: a resource scheduler in a data grid environment. In: Proceedings of third international symposium on cluster computing and the grid, p. 258. http://portal.acm.org/citation.cfm?id=792481

  22. Rahman RM, Barker K, Alhajj R (2006) Replica placement design with static optimality and dynamic maintainability. In: Proceedings of the sixth IEEE international symposium on cluster computing and the grid (CCGRID’06), pp 434–437

  23. Rahman RM, Barker K, Alhajj R (2006) Effective dynamic replica maintenance algorithm for the grid environment. In: Proceedings of advances in grid and pervasive computing, vol 3947: Grid and pervasive computing 2006 (GPC2006), pp 336–345

  24. Ranganathan K, Foster I Design and evaluation of dynamic replication strategies for a high performance data grid. In: Proceedings of international conference on computing in high energy and nuclear physics

  25. Shi XH, Jin H, Qiang WZ, Zou DQ (2003) An adaptive meta-scheduler for data-intensive applications. In: Proceedings of grid and cooperative computing (GCC’03), pp 830–837

  26. Stockinger H, Samar A, Allcock B, Foster I, Holtman K, Tierney B (2002) File and object replication in data grids. J Cluster Comput 5(3):305–314

    Article  Google Scholar 

  27. The Globus Alliance. http://www.globus.org/

  28. Vazhkudai S, Tuecke S, Foster I (2001) Replica selection in the globus data grid. In: Proceedings of the 1st international symposium on cluster computing and the grid (CCGRID 2001), pp 106–113

  29. Venugopal S, Buyya R, Ramamohanarao K (2006) A taxonomy of data grids for distributed data sharing, management, and processing. ACM computing surveys, vol  38, Article 3, March 2006

  30. Yang CT, Yang IH, Li KC, Wang SY (2007) Improvements on dynamic adjustment mechanism in co-allocation data grid environments. J Supercomput 40(3):269–280

    Article  MathSciNet  Google Scholar 

  31. Yang CT, Wang SY, Fu CP (2007) A dynamic adjustment mechanism for data transfer in data grids. In: Network and parallel computing: IFIP international conference, NPC 2007. Lecture notes in computer science, vol 4672. Springer, Berlin, pp 61–70. ISSN 1611-3349

    Google Scholar 

  32. Yang CT, Yang MF, Chiang WC (2008) Implementation of a cyber transformer for parallel download in co-allocation data grid environments. In: Proceedings of the 7th international conference on grid and cooperative computing (GCC2008) and second EchoGRID conference, October 24–26, 2008 in Shenzhen, Guangdong, China, pp 242–253

  33. Yang CT, Yang IH, Chen CH, Wang SY (2006) Implementation of a dynamic adjustment mechanism with efficient replica selection in co-allocation data grid environments. In: Proceedings of the 21st annual ACM symposium on applied computing (SAC 2006) – distributed systems and grid computing (DSGC) track, vol 1, pp 797–804, Dijon, France, April 23–27, 2006

  34. Yang CT, Yang IH, Wang SY, Li KC, Hsu CH (2009) A recursively-adjusting co-allocation scheme with cyber-transformer in data grids. Future Gener Comput Syst 25(7):695–703

    Article  Google Scholar 

  35. Ganglia. http://ganglia.info/

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Tunghai University, Taichung, 40704, Taiwan, ROC

    Chao-Tung Yang & Chun-Pin Fu

  2. Department of Computer Science and Information Engineering, Chung Hua University, Hsinchu, 30013, Taiwan, ROC

    Ching-Hsien Hsu

Authors
  1. Chao-Tung Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chun-Pin Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ching-Hsien Hsu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao-Tung Yang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yang, CT., Fu, CP. & Hsu, CH. File replication, maintenance, and consistency management services in data grids. J Supercomput 53, 411–439 (2010). https://doi.org/10.1007/s11227-009-0302-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-009-0302-9

Keywords

Search

Navigation