Skip to main content
SpringerLink
Log in

Towards a Scalable and Robust DHT

  • Published:
Theory of Computing Systems Aims and scope Submit manuscript

Abstract

The problem of scalable and robust distributed data storage has recently attracted a lot of attention. A common approach in the area of peer-to-peer systems has been to use a distributed hash table (or DHT). DHTs are based on the concept of virtual space. Peers and data items are mapped to points in that space, and local-control rules are used to decide, based on these virtual locations, how to interconnect the peers and how to map the data to the peers.

DHTs are known to be highly scalable and easy to update as peers enter and leave the system. It is relatively easy to extend the DHT concept so that a constant fraction of faulty peers can be handled without any problems, but handling adversarial peers is very challenging. The biggest threats appear to be join-leave attacks (i.e., adaptive join-leave behavior by the adversarial peers) and attacks on the data management level (i.e., adaptive insert and lookup attacks by the adversarial peers) against which no provably robust mechanisms are known so far. Join-leave attacks, for example, may be used to isolate honest peers in the system, and attacks on the data management level may be used to create a high load-imbalance, seriously degrading the correctness and scalability of the system.

We show, on a high level, that both of these threats can be handled in a scalable manner, even if a constant fraction of the peers in the system is adversarial, demonstrating that open systems for scalable distributed data storage that are robust against even massive adversarial behavior are feasible.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alt, H., Hagerup, T., Mehlhorn, K., Preparata, F.P.: Deterministic simulation of idealized parallel computers on more realistic ones. SIAM J. Comput. 16, 808–835 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  2. Aspnes, J., Shah, G.: Skip graphs. In: Proc. of the 14th ACM Symp. on Discrete Algorithms (SODA), pp. 384–393 (2003)

  3. Awerbuch, B., Scheideler, C.: Group spreading: A protocol for provably secure distributed name service. In: Proc. of the 31st International Colloquium on Automata, Languages and Programming (ICALP), 2004

  4. Awerbuch, B., Scheideler, C.: Robust random number generation for peer-to-peer systems. In: 10th Int. Conference on Principles of Distributed Systems (OPODIS), pp. 275–289 (2006)

  5. Borodin, A., Hopcroft, J.E.: Routing, merging, and sorting on parallel models of computation. J. Comput. Syst. Sci. 30, 130–145 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  6. Castro, M., Druschel, P., Ganesh, A., Rowstron, A., Wallach, D.: Security for structured peer-to-peer overlay networks. In: Proc. of the 5th Usenix Symp. on Operating Systems Design and Implementation (OSDI), 2002

  7. Castro, M., Liskov, B.: Practical Byzantine fault tolerance. In: Proc. of the 2nd Usenix Symp. on Operating Systems Design and Implementation (OSDI), 1999

  8. Crosby, S., Wallach, D.: Denial of service via algorithmic complexity attacks. In: Usenix Security, 2003

  9. Dietzfelbinger, M., Meyer auf der Heide, F.: Simple, efficient shared memory simulations. In: Proc. of the 5 ACM Symp. on Parallel Algorithms and Architectures (SPAA), pp. 110–119 (1993)

  10. Douceur, J.R.: The Sybil attack. In: Proc. of the 1st International Workshop on Peer-to-Peer Systems (IPTPS), 2002

  11. Druschel, P., Rowstron, A.: Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. In: Proc. of the 18th IFIP/ACM International Conference on Distributed Systems Platforms (Middleware 2001), 2001

  12. Dubhashi, D., Panconesi, A.: Concentration of measure for the analysis of randomized algorithms. Unpublished manuscript, accessible via http://www.cs.unibo.it/~pancones/papers.html (20 October 1998)

  13. Fiat, A., Saia, J., Young, M.: Making chord robust to Byzantine attacks. In: Proc. of the European Symposium on Algorithms (ESA), 2005

  14. Gennaro, R., Ishai, Y., Kushilevitz, E., Rabin, T.: The round complexity of verifiable secret sharing and secure multicast. In: Proc. of the 33rd ACM Symp. on Theory of Computing (STOC), pp. 580–589 (2001)

  15. Herley, K.T., Bilardi, G.: Deterministic simulations of PRAMs on bounded degree networks. SIAM J. Comput. 23, 276–292 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  16. Karger, D., Lehman, E., Leighton, T., Levine, M., Lewin, D., Panigrahi, R.: Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the World Wide Web. In: Proc. of the 29th ACM Symp. on Theory of Computing (STOC), pp. 654–663 (1997)

  17. Kuhn, F., Schmid, S., Wattenhofer, R.: A self-repairing peer-to-peer system resilient to dynamic adversarial churn. In: Proc. of the 4th International Workshop on Peer-to-Peer Systems (IPTPS), 2005

  18. Luccio, F., Pietracaprina, A., Pucci, G.: A new scheme for the deterministic simulation of PRAMs in VLSI. Algorithmica 5, 529–544 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  19. McDiarmid, A.: Concentration. In: Habib, M., McDiarmid, C., Ramirez-Alfonsin, J., Reed, B. (eds.) Probabilistic Methods for Algorithmic Discrete Mathematics, pp. 195–247. Springer, Berlin (1998)

    Google Scholar 

  20. Mehlhorn, K., Vishkin, U.: Randomized and deterministic simulations of PRAMs by parallel machines with restricted granularity of parallel memories. Acta Inf. 21, 339–374 (1984)

    MATH  MathSciNet  Google Scholar 

  21. Naor, M., Wieder, U.: Novel architectures for P2P applications: the continuous-discrete approach. In: Proc. of the 15th ACM Symp. on Parallel Algorithms and Architectures (SPAA), 2003

  22. Nielson, S., Crosby, S., Wallach, D.: Kill the messenger: A taxonomy of rational attacks. In: Proc. of the 4th International Workshop on Peer-to-Peer Systems (IPTPS), 2005

  23. Padmanabhan, V.N., Sripanidkulchai, K.: The case for cooperative networking. In: Proc. of the 1st International Workshop on Peer-to-Peer Systems (IPTPS), 2002

  24. Pagh, R., Rodler, F.: Cuckoo hashing. J. Algorithms 51, 122–144 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  25. Plaxton, G., Rajaraman, R., Richa, A.W.: Accessing nearby copies of replicated objects in a distributed environment. In: Proc. of the 9th ACM Symp. on Parallel Algorithms and Architectures (SPAA), pp. 311–320 (1997)

  26. Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable content-addressable network. In: Proc. of the ACM SIGCOMM ’01, 2001

  27. Rhea, S., Geels, D., Roscoe, T., Kubiatowicz, J.: Handling churn in a DHT. In: USENIX Annual Technical Conference, 2004

  28. Saia, J., Fiat, A., Gribble, S., Karlin, A., Saroiu, S.: Dynamically fault-tolerant content addressable networks. In: Proc. of the 1st International Workshop on Peer-to-Peer Systems (IPTPS), 2002

  29. Scheideler, C.: How to spread adversarial nodes? Rotate! In: Proc. of the 37th ACM Symp. on Theory of Computing (STOC), pp. 704–713 (2005)

  30. Singh, A., Castro, M., Rowstron, A., Druschel, P.: Defending against Eclipse attacks on overlay networks. In: Proc. of the 11th ACM SIGOPS European Workshop, 2004

  31. Sit, E., Morris, R.: Security considerations for peer-to-peer distributed hash tables. In: Proc. of 1st International Workshop on Peer-to-Peer Systems (IPTPS), 2002

  32. Srivatsa, M., Liu, L.: Vulnerabilities and security threats in structured overlay networks: A quantitative analysis. In: Proc. of the 20th IEEE Computer Security Applications Conference (ACSAC), 2004

  33. Stading, T., Maniatis, P., Baker, M.: Peer-to-peer caching schemes to address flash crowds. In: Proc. of the 1st International Workshop on Peer-to-Peer Systems (IPTPS), 2002

  34. Stavron, A., Rubenstein, D., Sahn, S.: A lightweight robust P2P system to handle flash crowds. In: Proc. of the IEEE Int. Conf. on Network Protocols (ICNP), 2002

  35. Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: A scalable peer-to-peer lookup service for Internet applications. In: Proc. of the ACM SIGCOMM ’01, 2001; see also http://www.pdos.lcs.mit.edu/chord/

  36. Upfal, E., Wigderson, A.: How to share memory in a distributed system. J. ACM 34, 116–127 (1987)

    MATH  MathSciNet  Google Scholar 

  37. Valiant, L.G.: A scheme for fast parallel communication. SIAM J. Comput. 2(11), 350–361 (1982)

    Article  MathSciNet  Google Scholar 

  38. Zhao, B.Y., Kubiatowicz, J., Joseph, A.: Tapestry: An infrastructure for fault-tolerant wide-area location and routing. Technical report, UCB/CSD-01-1141, University of California at Berkeley (2001); see also http://www.cs.berkeley.edu/~ravenben/tapestry

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Johns Hopkins University, Baltimore, MD, 21218, USA

    Baruch Awerbuch

  2. Institute for Computer Science, Technical University of Munich, Boltzmannstr. 3, 85748, Garching, Germany

    Christian Scheideler

Authors
  1. Baruch Awerbuch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Scheideler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Scheideler.

Additional information

Supported by NSF-ANIR 0240551, NSF-CCF 0515080, and NSF-CCR 0311795.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Awerbuch, B., Scheideler, C. Towards a Scalable and Robust DHT. Theory Comput Syst 45, 234–260 (2009). https://doi.org/10.1007/s00224-008-9099-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00224-008-9099-9

Keywords

Search

Navigation