Skip to main content
SpringerLink
Log in

Improving peer connectivity in wide-area overlays of virtual workstations

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Self-configuring virtual networks rely on structured P2P routing to provide seamless connectivity among nodes through overlay routing of virtual IP packets, support decentralized hole-punching to establish bi-directional communication links among nodes behind network address translators, and dynamic configuration of virtual IP addresses. Our experiences with deployments of virtual networks in support of wide-area overlays of virtual workstations (WOWs) reveal that connectivity constraints imposed by symmetric NATs and by Internet route outages often hinder P2P overlay structure maintenance and routability, subsequently limiting the ability of WOWs to deliver high-throughput computing through aggregation of resources in different domains.

In this paper, we describe and evaluate two novel approaches which are generally applicable and fully decentralized, and show that they improve routability of structured P2P networks in such connectivity constrained environments: (1) a fault-tolerant routing algorithm based on simulated annealing from optimization theory, and (2) tunneling of connections between adjacent nodes (in the P2P identifier space) over common neighbors when direct communication is not possible. Simulation-based analyses show that (1) when pairs of nodes only have 70% chance of being able to communicate directly, the described approaches improve all-to-all routability of the network from 90% to 99%, and (2) even when only 70% of the nodes are behind NATs that include symmetric NATs, these techniques improve the all-to-all connectivity of the network from less than 95% to more than 99%. We have implemented these techniques in the IP-over-P2P (IPOP) virtual network and have conducted experiments with a 180-node WOW Condor pool, demonstrating that, at 81% probability of establishing a pair-wise connection, annealing and tunneling combined allow all nodes to be connected to the pool, compared to only 160 nodes in the absence of these techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Anderson, D.P., Cobb, J., Korpella, E., Lebofsky, M., Werthimer, D.: Seti@home: an experiment in public-resource computing. Commun. ACM 11(45), 56–61 (2002)

    Article  Google Scholar 

  2. Aspnes, J., Diamadi, Z., Shah, G.: Fault-tolerant routing in peer-to-peer systems. In: Proc. of the Symp. on Principles of Distributed Computing (PODC), Monterey, CA, July 2002

  3. The bamboo dht—introduction. http://bamboo-dht.org/

  4. Calder, B., Chien, A.A., Wang, J., Yang, D.: The entropia virtual machine for desktop grids. In: CSE Technical Report CS2003-0773, University of California, San Diego, San Diego, CA, Oct. 2003

  5. Castro, M., Costa, M., Rowstron, A.: Performance and dependability of structured peer-to-peer overlays. In: Proc. of the Conf. on Dependable Systems and Networks, June 2004

  6. Castro, M., Druschel, P., Hu, Y.C., Rowstron, A.: Topology-aware routing in structured peer-to-peer overlay networks. In: Microsoft Research MSR-TR-2002-82, Sep. 2002

  7. Chen, W., Liu, X.: Enforcing routing consistency in structured peer-to-peer overlays: should we and could we? In: Proc. of the Workshop on Peer-to-Peer Systems (IPTPS), Santa Barbara, CA, Feb. 2006

  8. Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., Bowman, M.: Planetlab: an overlay testbed for broad-coverage services. ACM SIGCOMM Comput. Commun. Rev. 33(3), 3–12 (2003)

    Article  Google Scholar 

  9. Ford, B.: Unmanaged Internet Protocol: taming the edge network management crisis. In: Proc. of the Workshop on Hot Topics in Networks (HotNets), Cambridge, MA, Nov. 2003

  10. Ford, B., Srisuresh, P., Kegel, D.: Peer-to-peer communication across network address translators. In: Proc. of the USENIX Annual Technical Conference, Anaheim, California, April 2005

  11. Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the grid: enabling scalable virtual organizations. Int. J. Supercomput. Appl. 15(3), 200–222 (2001)

    Article  Google Scholar 

  12. Freedman, M.J., Lakshminarayanan, K., Rhea, S., Stoica, I.: Non-transitive connectivity and DHTs. In: Proc. of the USENIX WORLDS, San Francisco, CA, Dec. 2005

  13. Ganguly, A., Agrawal, A., Boykin, P.O., Figueiredo, R.J.: IP over P2P: enabling self-configuring virtual IP networks for grid computing. In: Proc. of Intl. Parallel and Distributed Processing Symp. (IPDPS), Rhodes, Greece, April 2006

  14. Ganguly, A., Agrawal, A., Boykin, P.O., Figueiredo, R.J.: Wow: self-organizing wide area overlay networks of virtual workstations. In: Proc. of Intl. Symp. on High Performance Distributed Computing, Paris, France, June 2006

  15. Ganguly, A., Wolinsky, D., Boykin, P.O., Figueiredo, R.J.: Decentralized dynamic host configuration in wide-area overlays of virtual workstations. In: Proc. of the Workshop on Desktop Grids and Volunteer Computing Systems, with IPDPS, Long Beach, CA, March 2006

  16. Gerding, S., Stribling, J.: Examining the trade-offs of structured overlays in dynamic non-transitive network, Dec. 2003. http://pdos.lcs.mit.edu/~strib/doc/networking_fall2003.ps

  17. Gummadi, K., Gummadi, R., Gribble, S., Ratnasamy, S., Shenker, S., Stoica, I.: The impact of DHT routing geometry on resilience and proximity. In: Proc. of ACM SIGCOMM, Karlsruhe, Germany, Aug. 2003

  18. Kleinberg, J.: Nature 406, 845 (2000)

    Article  Google Scholar 

  19. Li, J., Stribling, J., Morris, R., Kaashoek, M.F., Gil, T.M.: A performance vs. cost framework for evaluating DHT design tradeoffs under churn. In: Proc. IEEE INFOCOM, 2005

  20. Liang, J., Kumar, R., Ross, K.: The fasttrack overlay: a measurement study. In: Computer Networks (Special Issue on Overlays), 2005

  21. Litzkow, M., Livny, M., Mutka, M.: Condor—a hunter of idle workstations. In: Proc. of Intl. Conference on Distributed Computing Systems, June 1988

  22. Lo, V., Zappala, D., Zhou, D., Liu, Y., Zhao, S.: Cluster computing on the fly: P2P scheduling of idle cycles in the internet. In: Proc. of the 3rd Intl. Workshop on Peer-to-Peer Systems (IPTPS), San Diego, CA, Feb. 2004

  23. Maassen, J., Bal, H.E.: Smartsockets: solving the connectivity problems in grid computing. In: Proc. of Symp. on High Performance Distributed Computing Symposium, Monterey Bay, CA, June 2007

  24. Maymounkov, P., Mazières, D.: Kademlia: a peer-to-peer information system based on the xor metric. In: Proc. of the Workshop on Peer-to-Peer Systems (IPTPS), Cambridge, MA, March 2002

  25. Mislove, A., Druschel, P.: Providing administrative control and autonomy in structured peer-to-peer overlays. In: Proc. of the Workshop on Peer-to-Peer Systems, San Diego, CA, Feb. 2004

  26. Mislove, A., Post, A., Haeberlen, A., Druschel, P.: Experiences in building and operating epost, a reliable peer-to-peer application. In: Proc. of European Conf. on Computer Systems, Leuven, Belgium, April 2006

  27. Rhea, S., Geels, D., Roscoe, T., Kubiatowicz, J.: Handling churn in a DHT. In: Proc. of USENIX Technical Conference, June 2004

  28. Rhea, S., Godfrey, B., Karp, B., Kubiatowicz, J., Ratnasamy, S., Shenker, S., Stoica, I., Opendht, H.Yu.: A public DHT service and its uses. In: Proc. of ACM SIGCOMM, Philadelphia, PA, Aug. 2005

  29. Rowstron, A., Druschel, P.: Pastry: scalable, decentralized object location and routing for large-scale peer-to-peer systems. In: Proc. of the Intl. Conf. on Distributed Systems Platforms (Middleware), Heidelberg, Germany, Nov. 2001

  30. Stoica, I., Morris, R., Liben-Nowell, D., Karger, D.R., Kaashoek, M.F., Dabek, F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup protocol for internet applications. IEEE/ACM Trans. Netw. 11(1), 17–32 (2003)

    Article  Google Scholar 

  31. Universal plug and play in windows xp. http://technet.microsoft.com/en-us/library/bb457049.aspx

  32. Wolinsky, D., Agrawal, A., Boykin, P.O., Davis, J., Ganguly, A., Paramygin, V., Sheng, P., Figueiredo, R.: On the design of virtual machine sandboxes for distributed computing in wide area overlays of virtual workstations. In: Proc. of the Workshop on Virtualization Technology in Distributed Computing, with Supercomputing, Tampa, FL, Nov. 2006

Download references

Author information

Authors and Affiliations

  1. Advanced Computing and Information Systems Laboratory, University of Florida, Gainesville, FL, 32611, USA

    Arijit Ganguly, P. Oscar Boykin, David I. Wolinsky & Renato J. Figueiredo

Authors
  1. Arijit Ganguly
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. Oscar Boykin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David I. Wolinsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Renato J. Figueiredo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arijit Ganguly.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ganguly, A., Boykin, P.O., Wolinsky, D.I. et al. Improving peer connectivity in wide-area overlays of virtual workstations. Cluster Comput 12, 239–256 (2009). https://doi.org/10.1007/s10586-009-0075-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-009-0075-1

Keywords

Search

Navigation