Skip to main content
SpringerLink
Log in

Optimizing write operation on replica in data grid

  • Research Papers
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Replication technique can improve access performance and data availability of data grid systems. So far, most related work on data replication in data grid systems focuses on read-only dataset. Though write operations have been widely used in practice, its optimization problem has not been sufficiently studied. In this paper, we show that write requests from users can be processed by a nearest site, regardless of the location of latest written replicas. The policy of write nearby instead of write to the latest replica can improve the performance of write operations, while inconsistency is caused between multiple replicas. Therefore, we propose two metadata management algorithms to make sure that users can retrieve up-to-date datasets at any time. We extend OptorSim to support the implementation and evaluation of our scheme. The results of simulation show that our scheme achieves performance gains up to 3.3 compared with WriteMaster scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Chervenak A, Foster I, Kesselman C, et al. The data grid: Towards architecture for the distributed management and analysis of large scientific datasets. J Netw Comput Appl, 2000, 23: 187–200

    Article  Google Scholar 

  2. Hoschek W, Jaen-Martinez J, Samar A. Data management in an international data grid project. In: Grid 2000, London, UK, 2000. 333–361

  3. Stockinger H, Samar A, Allcock A, et al. File and object replication in data grids. J Clust Comput, 2002, 5: 305–314

    Article  Google Scholar 

  4. Venugopal S, Buyya R, Ramamohanarao K. A taxonomy of data grids for distributed data sharing, management, and processing. ACM Comput Surv, 2006, 38: 1–53

    Article  Google Scholar 

  5. Ranganathan K, Foster I. Identifying dynamic replication strategies for a high performance data grid. In: GRID 2001, Denver, 2001. 75–86

  6. Chervenak A, Deelman E, Foster I, et al. Giggle: A framework for constructing scalable replica location services. In: SC 2002, Baltimore, Maryland, USA, 2002. 1–17

  7. Samar A, Stockinger H. Grid data management pilot (GDMP): A tool for wide area replication. In: IASTED International Conference on Applied Informatics, Innsbruck, Austria, 2001

  8. Rajasekar A, Wan M, Moore R, et al. Storage resource broker-managing distributed data in a grid. Comput Soc India J, 2003, 33: 42–54

    Google Scholar 

  9. Foster I, Kesselman C, Tuecke S. The anatomy of the grid: Enabling scalable virtual organizations. Int J Supercomput Appl, 2001, 15: 200–222

    Article  Google Scholar 

  10. Yang C T, Lai C L. Apply cluster and grid computing on parallel 3D rendering. In: 2004 IEEE International Conference on Multimedia and Expo. Taipei, 2004. 859–862

  11. Cameron J, Casey L, Guy P, et al. Replica management services in the European Data Grid Project UK. In: E-Science All Hands Conference, Nottingham, 2004

  12. Domenici A, Donno F, Pucciani G, et al. Relaxed data consistency with CONStanza. In: CCGRID 2006, Singapore, 2006. 425–429

  13. Ghemawat S, Gobioff H, Leung S T. The Google file system. In: SOSP 2003, New York, USA, 2003. 29–43

  14. Bell W H, Cameron D G, Capozza L, et al. OptorSim: a grid simulator for studying dynamic data replication strategies. Int J High Perform Comput Appl, 2003, 17: 403–416

    Article  Google Scholar 

  15. Cameron D G, Millar A P, Nicholson C, et al. Analysis of scheduling and replica optimisation strategies for data grids using OptorSim. J Grid Comput, 2004, 2: 57–69

    Article  Google Scholar 

  16. Lin Y F, Liu P, Wu J J. Optimal placement of replica in data grid environments with locality assurance. In: The 12th International Conference on Parallel and Distributed Systems, Minneapolis, USA, 2006

  17. Dullmann W H, Jaen-Martinez J, Segal B, et al. Models for replica synchronisation and consistency in a data grid. In: HPDC 2001, San Francisco, CA, USA, 2001. 67–75

  18. Domenici A, Donno F, Pucciani G, et al. Replica consistency in a data grid. In: International Workshop on Advanced Computing and Analysis Techniques in Physics Research, Tsukuba, Japan, 2003. 534: 24–28

    Google Scholar 

  19. Anne B, Veronika R, Yves R. Impact of QoS on replica placement in tree networks. In: The 7th International Conference on Computational Science 2007, Beijing, China, 2007. 366–373

  20. Yuan Y L, Wu Y W, Yang G W, et al. Dynamic data replication based on local optimization principle in data grid. In: The 6th International Conference on Grid and Cooperative Computing, Urumchi, China, 2007. 815–822

  21. Bell W H, Cameron D G, Carvajal-Schiaffino R, et al. Evaluation of an economy-based file replication strategy for a data grid. In: CCGrid 2003, Tokyo, Japan, 2003. 661–668

  22. Chang R, Chang J. Adaptable replica consistency service for data grids. In: The 3th International Conference on Information Technology: New Generations, Las Vegas, Nevada, USA, 2006. 646–651

  23. Zhong M, Shen K, Seiferas J. Replication degree customization for high availability. In: EuroSys 2008, Glasgow, Scotland, 2008. 55–68

  24. Zhong M, Lu P, Shen K, et al. Optimizing data popularity conscious bloom filters. In: PODC 2008, Toronto, Canada, 2008

  25. Lei M, Vrbsky S V, Hong X. An on-line replication strategy to increase availability in data grids. Future Gener Comput Syst, 2008, 24: 85–98

    Article  MATH  Google Scholar 

  26. Raouf B, Karsten M, Young M. A heuristic for fair correlation-aware resource placement. Exper Algor, 2009, 5526: 89–100

    Article  Google Scholar 

  27. Zhong M, Shen K, Seiferas J. Correlation-aware object placement for multi-object operations. In: ICDCS 2008, Beijing, China, 2008. 512–521

  28. Maccormick J, Thekkath C A, Jager M, et al. Niobe: A practical replication protocol. ACM Trans Stor, 2008, 3: 1–43

    Article  Google Scholar 

  29. Cameron D G, Carvajal-Schiaffino R, Millar P, et al. Evaluating scheduling and replica optimisation strategies in OptorSim. In: Grid 2003, Phoenix, Arizona, USA, 2003. 52–59

  30. Belalem G, Slimani Y. Consistency management for data grid in optorSim simulator. In: 2007 International Conference on Multimedia and Ubiquitous Engineering, Seoul, Korea, 2007. 554–560

  31. Lefebure V, Wildish T. The Spring 2002 DAQ TDR Production. CMS Internal Note, Geneva, Switzerland, 2005

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    PengZhi Xu, YongWei Wu, XiaoMeng Huang, GuangWen Yang & WeiMin Zheng

Authors
  1. PengZhi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. YongWei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. XiaoMeng Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. GuangWen Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. WeiMin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to YongWei Wu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xu, P., Wu, Y., Huang, X. et al. Optimizing write operation on replica in data grid. Sci. China Inf. Sci. 54, 1–11 (2011). https://doi.org/10.1007/s11432-010-4153-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-010-4153-z

Keywords

Search

Navigation