Abstract
Fully decentralized peer-to-peer (P2P) storage systems lack the reliability guarantees that centralized systems can give. They need to rely on the system’s statistical properties, only. Nevertheless, such probabilistic guarantees can lead to highly reliable systems. Moreover, their statistical nature makes P2P storage systems an ideal supplement to centralized storage systems, because they fail in entirely different circumstances than centralized systems.
In this paper, we investigate the behavior of different replication and erasure code schemes as peers fail. We calculate the data loss probability and the repairing delay, which is caused by the peers’ limited bandwidth. Using a Weibull model to describe peer behavior, we show that there are four different loss processes that affect the availability and durability of the data: initial loss, diurnal loss, early loss, and longterm loss. They need to be treated differently to obtain optimal results. Based on this insight we give general recommendations for the design of redundancy schemes in P2P storage systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Acedański, S., Deb, S., Médard, M., Kötter, R.: How good is random linear coding based distributed networked storage. In: Proc. of the 1st Workshop on Network Coding, Theory and Applications (NetCod), Riva del Garda, Italy (April 2005)
Ahlswede, R., Cai, N., Li, S.y.R., Yeung, R.W., Member, S.: Network information flow. IEEE Transactions on Information Theory 46, 1204–1216 (2000)
Batten, C., Barr, K., Saraf, A., Trepetin, S.: pStore: A secure peer-to-peer backup system. Technical Memo MIT-LCS-TM-632, Massachusetts Institute of Technology Laboratory for Computer Science (October 2002)
Bhagwan, R., Tati, K., Cheng, Y.C., Savage, S., Voelker, G.M.: Total recall: System support for automated availability management. In: Proc. of the 1st USENIX Symposium on Networked Systems Design and Implementation, NSDI 2004 (2004)
Blake, C., Rodrigues, R.: High availability, scalable storage, dynamic peer networks: pick two. In: Proc. of the 9th USENIX Conference on Hot Topics in Operating Systems (HOTOS 2003), Lihue, Hawaii (2003)
Chun, B.G., Dabek, F., Haeberlen, A., Sit, E., Weatherspoon, H., Kaashoek, M.F., Kubiatowicz, J., Morris, R.: Efficient replica maintenance for distributed storage systems. In: Proc. of the 3rd USENIX Symposium on Networked Systems Design and Implementation, NSDI 2006 (2006)
Dabek, F., Li, J., Sit, E., Robertson, J., Kaashoek, M.F., Morris, R.: Designing a DHT for low latency and high throughput. In: Proc. of the 1st USENIX Symposium on Networked Systems Design and Implementation, NSDI 2004 (2004)
Datta, A., Aberer, K.: Internet-scale storage systems under churn – a study of the steady-state using markov models. In: Proc. of the 6th IEEE Intl. Conference on Peer-to-Peer Computing (P2P 2006), Washington, DC (2006)
Dimakis, R.G., Godfrey, P.B., Wainwright, M.J., Ramch, K.: Network coding for distributed storage systems. In: Proc. of the 26th Annual Joint Conference of the IEEE Computer and Communications Societies, INFOCOM 2007 (2007)
Duminuco, A., Biersack, E.: Hierarchical codes: How to make erasure codes attractive for peer-to-peer storage systems. In: Proc. of the 8th International Conference on Peer-to-Peer Computing (P2P 2008), Aachen, Germany (September 2008)
Duminuco, A., Biersack, E., En-Najjary, T.: Proactive replication in distributed storage systems using machine availability estimation. In: Proc. of the 3rd ACM Intl. Conference on Emerging Networking Experiments and Technologies (CoNEXT 2007), New York, NY (2007)
Gkantsidis, C., Rodriguez, P.R.: Network coding for large scale content distribution. In: Proc. of the 24th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM 2005), vol. 4, pp. 2235–2245 (March 2005)
Ho, T., Mdard, M., Koetter, R., Karger, D.R., Member, A., Effros, M., Member, S., Member, S., Member, S., Shi, J., Leong, B.: A random linear network coding approach to multicast. IEEE Trans. Inform. Theory 52, 4413–4430 (2006)
Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S., Eaton, P., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Weimer, W., Wells, C., Zhao, B.: OceanStore: An architecture for global-scale persistent storage. In: Proc. of the 9th Intl. Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2000 (2000)
Lin, W.K., Chiu, D.M., Lee, Y.B.: Erasure code replication revisited. In: Proc. of the 4th International Conference on Peer-to-Peer Computing (P2P 2004), Zurich, Switzerland (August 2004)
Ramabhadran, S.: Analysis of long-running replicated systems. In: Proc. of the 25th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM 2006), Barcelona, Spain (2006)
Rodrigues, R., Liskov, B.: High availability in DHTs: Erasure coding vs. replication. In: Castro, M., van Renesse, R. (eds.) IPTPS 2005. LNCS, vol. 3640, pp. 226–239. Springer, Heidelberg (2005)
Sit, E., Haeberlen, A., Dabek, F.,, B.: g. Chun, H. Weatherspoon, R. Morris, M.F. Kaashoek, J. Kubiatowicz. Proactive replication for data durability. In: Proc. of the 5th Intl. Workshop on Peer-to-Peer Systems (IPTPS 2006), Santa Barbara, CA (February 2006)
Steiner, M., En-Najjary, T., Biersack, E.W.: Long term study of peer behavior in the KAD DHT. IEEE/ACM Transactions on Networking 17(5) (October 2009)
Utard, G., Vernois, A.: Data durability in peer to peer storage systems. In: Proc. of the 4th IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2004), Chicago, Illinois (April 2004)
Weatherspoon, H., Kubiatowicz, J.: Erasure Coding vs. Replication: A Quantitative Comparison. In: Druschel, P., Kaashoek, M.F., Rowstron, A. (eds.) IPTPS 2002. LNCS, vol. 2429, p. 328. Springer, Heidelberg (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Houri, Y., Amann, J., Fuhrmann, T. (2010). A Quantitative Analysis of Redundancy Schemes for Peer-to-Peer Storage Systems. In: Dolev, S., Cobb, J., Fischer, M., Yung, M. (eds) Stabilization, Safety, and Security of Distributed Systems. SSS 2010. Lecture Notes in Computer Science, vol 6366. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16023-3_40
Download citation
DOI: https://doi.org/10.1007/978-3-642-16023-3_40
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16022-6
Online ISBN: 978-3-642-16023-3
eBook Packages: Computer ScienceComputer Science (R0)