Skip to main content
SpringerLink
Log in

A Quantitative Analysis of Redundancy Schemes for Peer-to-Peer Storage Systems

  • Conference paper
  • First Online:
Stabilization, Safety, and Security of Distributed Systems (SSS 2010)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 6366))

Included in the following conference series:

  • 626 Accesses

Abstract

Fully decentralized peer-to-peer (P2P) storage systems lack the reliability guarantees that centralized systems can give. They need to rely on the system’s statistical properties, only. Nevertheless, such probabilistic guarantees can lead to highly reliable systems. Moreover, their statistical nature makes P2P storage systems an ideal supplement to centralized storage systems, because they fail in entirely different circumstances than centralized systems.

In this paper, we investigate the behavior of different replication and erasure code schemes as peers fail. We calculate the data loss probability and the repairing delay, which is caused by the peers’ limited bandwidth. Using a Weibull model to describe peer behavior, we show that there are four different loss processes that affect the availability and durability of the data: initial loss, diurnal loss, early loss, and longterm loss. They need to be treated differently to obtain optimal results. Based on this insight we give general recommendations for the design of redundancy schemes in P2P storage systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Acedański, S., Deb, S., Médard, M., Kötter, R.: How good is random linear coding based distributed networked storage. In: Proc. of the 1st Workshop on Network Coding, Theory and Applications (NetCod), Riva del Garda, Italy (April 2005)

    Google Scholar 

  2. Ahlswede, R., Cai, N., Li, S.y.R., Yeung, R.W., Member, S.: Network information flow. IEEE Transactions on Information Theory 46, 1204–1216 (2000)

    Article  MathSciNet  Google Scholar 

  3. Batten, C., Barr, K., Saraf, A., Trepetin, S.: pStore: A secure peer-to-peer backup system. Technical Memo MIT-LCS-TM-632, Massachusetts Institute of Technology Laboratory for Computer Science (October 2002)

    Google Scholar 

  4. Bhagwan, R., Tati, K., Cheng, Y.C., Savage, S., Voelker, G.M.: Total recall: System support for automated availability management. In: Proc. of the 1st USENIX Symposium on Networked Systems Design and Implementation, NSDI 2004 (2004)

    Google Scholar 

  5. Blake, C., Rodrigues, R.: High availability, scalable storage, dynamic peer networks: pick two. In: Proc. of the 9th USENIX Conference on Hot Topics in Operating Systems (HOTOS 2003), Lihue, Hawaii (2003)

    Google Scholar 

  6. Chun, B.G., Dabek, F., Haeberlen, A., Sit, E., Weatherspoon, H., Kaashoek, M.F., Kubiatowicz, J., Morris, R.: Efficient replica maintenance for distributed storage systems. In: Proc. of the 3rd USENIX Symposium on Networked Systems Design and Implementation, NSDI 2006 (2006)

    Google Scholar 

  7. Dabek, F., Li, J., Sit, E., Robertson, J., Kaashoek, M.F., Morris, R.: Designing a DHT for low latency and high throughput. In: Proc. of the 1st USENIX Symposium on Networked Systems Design and Implementation, NSDI 2004 (2004)

    Google Scholar 

  8. Datta, A., Aberer, K.: Internet-scale storage systems under churn – a study of the steady-state using markov models. In: Proc. of the 6th IEEE Intl. Conference on Peer-to-Peer Computing (P2P 2006), Washington, DC (2006)

    Google Scholar 

  9. Dimakis, R.G., Godfrey, P.B., Wainwright, M.J., Ramch, K.: Network coding for distributed storage systems. In: Proc. of the 26th Annual Joint Conference of the IEEE Computer and Communications Societies, INFOCOM 2007 (2007)

    Google Scholar 

  10. Duminuco, A., Biersack, E.: Hierarchical codes: How to make erasure codes attractive for peer-to-peer storage systems. In: Proc. of the 8th International Conference on Peer-to-Peer Computing (P2P 2008), Aachen, Germany (September 2008)

    Google Scholar 

  11. Duminuco, A., Biersack, E., En-Najjary, T.: Proactive replication in distributed storage systems using machine availability estimation. In: Proc. of the 3rd ACM Intl. Conference on Emerging Networking Experiments and Technologies (CoNEXT 2007), New York, NY (2007)

    Google Scholar 

  12. Gkantsidis, C., Rodriguez, P.R.: Network coding for large scale content distribution. In: Proc. of the 24th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM 2005), vol. 4, pp. 2235–2245 (March 2005)

    Google Scholar 

  13. Ho, T., Mdard, M., Koetter, R., Karger, D.R., Member, A., Effros, M., Member, S., Member, S., Member, S., Shi, J., Leong, B.: A random linear network coding approach to multicast. IEEE Trans. Inform. Theory 52, 4413–4430 (2006)

    Article  MathSciNet  Google Scholar 

  14. Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S., Eaton, P., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Weimer, W., Wells, C., Zhao, B.: OceanStore: An architecture for global-scale persistent storage. In: Proc. of the 9th Intl. Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2000 (2000)

    Google Scholar 

  15. Lin, W.K., Chiu, D.M., Lee, Y.B.: Erasure code replication revisited. In: Proc. of the 4th International Conference on Peer-to-Peer Computing (P2P 2004), Zurich, Switzerland (August 2004)

    Google Scholar 

  16. Ramabhadran, S.: Analysis of long-running replicated systems. In: Proc. of the 25th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM 2006), Barcelona, Spain (2006)

    Google Scholar 

  17. Rodrigues, R., Liskov, B.: High availability in DHTs: Erasure coding vs. replication. In: Castro, M., van Renesse, R. (eds.) IPTPS 2005. LNCS, vol. 3640, pp. 226–239. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  18. Sit, E., Haeberlen, A., Dabek, F.,, B.: g. Chun, H. Weatherspoon, R. Morris, M.F. Kaashoek, J. Kubiatowicz. Proactive replication for data durability. In: Proc. of the 5th Intl. Workshop on Peer-to-Peer Systems (IPTPS 2006), Santa Barbara, CA (February 2006)

    Google Scholar 

  19. Steiner, M., En-Najjary, T., Biersack, E.W.: Long term study of peer behavior in the KAD DHT. IEEE/ACM Transactions on Networking 17(5) (October 2009)

    Google Scholar 

  20. Utard, G., Vernois, A.: Data durability in peer to peer storage systems. In: Proc. of the 4th IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2004), Chicago, Illinois (April 2004)

    Google Scholar 

  21. Weatherspoon, H., Kubiatowicz, J.: Erasure Coding vs. Replication: A Quantitative Comparison. In: Druschel, P., Kaashoek, M.F., Rowstron, A. (eds.) IPTPS 2002. LNCS, vol. 2429, p. 328. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Technische Universität München, Germany

    Yaser Houri, Johanna Amann & Thomas Fuhrmann

Authors
  1. Yaser Houri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Johanna Amann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Fuhrmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, Ben-Gurion University of the Negev,Beer-Sheva, 84105, Israel

    Shlomi Dolev

  2. Department of Computer Science, The University of Texas at Dallas, TX 75083-0688, Richardson, USA

    Jorge Cobb

  3. Department of Computer Science, Yale University, 51 Prospect Street, CT 06511, New Haven, USA

    Michael Fischer

  4. Department of Computer Science, Columbia University, 10027, New York, NY, USA

    Moti Yung

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Houri, Y., Amann, J., Fuhrmann, T. (2010). A Quantitative Analysis of Redundancy Schemes for Peer-to-Peer Storage Systems. In: Dolev, S., Cobb, J., Fischer, M., Yung, M. (eds) Stabilization, Safety, and Security of Distributed Systems. SSS 2010. Lecture Notes in Computer Science, vol 6366. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16023-3_40

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-16023-3_40

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-16022-6

  • Online ISBN: 978-3-642-16023-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation