Abstract
Self-configuring virtual networks rely on structured P2P routing to provide seamless connectivity among nodes through overlay routing of virtual IP packets, support decentralized hole-punching to establish bi-directional communication links among nodes behind network address translators, and dynamic configuration of virtual IP addresses. Our experiences with deployments of virtual networks in support of wide-area overlays of virtual workstations (WOWs) reveal that connectivity constraints imposed by symmetric NATs and by Internet route outages often hinder P2P overlay structure maintenance and routability, subsequently limiting the ability of WOWs to deliver high-throughput computing through aggregation of resources in different domains.
In this paper, we describe and evaluate two novel approaches which are generally applicable and fully decentralized, and show that they improve routability of structured P2P networks in such connectivity constrained environments: (1) a fault-tolerant routing algorithm based on simulated annealing from optimization theory, and (2) tunneling of connections between adjacent nodes (in the P2P identifier space) over common neighbors when direct communication is not possible. Simulation-based analyses show that (1) when pairs of nodes only have 70% chance of being able to communicate directly, the described approaches improve all-to-all routability of the network from 90% to 99%, and (2) even when only 70% of the nodes are behind NATs that include symmetric NATs, these techniques improve the all-to-all connectivity of the network from less than 95% to more than 99%. We have implemented these techniques in the IP-over-P2P (IPOP) virtual network and have conducted experiments with a 180-node WOW Condor pool, demonstrating that, at 81% probability of establishing a pair-wise connection, annealing and tunneling combined allow all nodes to be connected to the pool, compared to only 160 nodes in the absence of these techniques.
Similar content being viewed by others
References
Anderson, D.P., Cobb, J., Korpella, E., Lebofsky, M., Werthimer, D.: Seti@home: an experiment in public-resource computing. Commun. ACM 11(45), 56–61 (2002)
Aspnes, J., Diamadi, Z., Shah, G.: Fault-tolerant routing in peer-to-peer systems. In: Proc. of the Symp. on Principles of Distributed Computing (PODC), Monterey, CA, July 2002
The bamboo dht—introduction. http://bamboo-dht.org/
Calder, B., Chien, A.A., Wang, J., Yang, D.: The entropia virtual machine for desktop grids. In: CSE Technical Report CS2003-0773, University of California, San Diego, San Diego, CA, Oct. 2003
Castro, M., Costa, M., Rowstron, A.: Performance and dependability of structured peer-to-peer overlays. In: Proc. of the Conf. on Dependable Systems and Networks, June 2004
Castro, M., Druschel, P., Hu, Y.C., Rowstron, A.: Topology-aware routing in structured peer-to-peer overlay networks. In: Microsoft Research MSR-TR-2002-82, Sep. 2002
Chen, W., Liu, X.: Enforcing routing consistency in structured peer-to-peer overlays: should we and could we? In: Proc. of the Workshop on Peer-to-Peer Systems (IPTPS), Santa Barbara, CA, Feb. 2006
Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., Bowman, M.: Planetlab: an overlay testbed for broad-coverage services. ACM SIGCOMM Comput. Commun. Rev. 33(3), 3–12 (2003)
Ford, B.: Unmanaged Internet Protocol: taming the edge network management crisis. In: Proc. of the Workshop on Hot Topics in Networks (HotNets), Cambridge, MA, Nov. 2003
Ford, B., Srisuresh, P., Kegel, D.: Peer-to-peer communication across network address translators. In: Proc. of the USENIX Annual Technical Conference, Anaheim, California, April 2005
Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the grid: enabling scalable virtual organizations. Int. J. Supercomput. Appl. 15(3), 200–222 (2001)
Freedman, M.J., Lakshminarayanan, K., Rhea, S., Stoica, I.: Non-transitive connectivity and DHTs. In: Proc. of the USENIX WORLDS, San Francisco, CA, Dec. 2005
Ganguly, A., Agrawal, A., Boykin, P.O., Figueiredo, R.J.: IP over P2P: enabling self-configuring virtual IP networks for grid computing. In: Proc. of Intl. Parallel and Distributed Processing Symp. (IPDPS), Rhodes, Greece, April 2006
Ganguly, A., Agrawal, A., Boykin, P.O., Figueiredo, R.J.: Wow: self-organizing wide area overlay networks of virtual workstations. In: Proc. of Intl. Symp. on High Performance Distributed Computing, Paris, France, June 2006
Ganguly, A., Wolinsky, D., Boykin, P.O., Figueiredo, R.J.: Decentralized dynamic host configuration in wide-area overlays of virtual workstations. In: Proc. of the Workshop on Desktop Grids and Volunteer Computing Systems, with IPDPS, Long Beach, CA, March 2006
Gerding, S., Stribling, J.: Examining the trade-offs of structured overlays in dynamic non-transitive network, Dec. 2003. http://pdos.lcs.mit.edu/~strib/doc/networking_fall2003.ps
Gummadi, K., Gummadi, R., Gribble, S., Ratnasamy, S., Shenker, S., Stoica, I.: The impact of DHT routing geometry on resilience and proximity. In: Proc. of ACM SIGCOMM, Karlsruhe, Germany, Aug. 2003
Kleinberg, J.: Nature 406, 845 (2000)
Li, J., Stribling, J., Morris, R., Kaashoek, M.F., Gil, T.M.: A performance vs. cost framework for evaluating DHT design tradeoffs under churn. In: Proc. IEEE INFOCOM, 2005
Liang, J., Kumar, R., Ross, K.: The fasttrack overlay: a measurement study. In: Computer Networks (Special Issue on Overlays), 2005
Litzkow, M., Livny, M., Mutka, M.: Condor—a hunter of idle workstations. In: Proc. of Intl. Conference on Distributed Computing Systems, June 1988
Lo, V., Zappala, D., Zhou, D., Liu, Y., Zhao, S.: Cluster computing on the fly: P2P scheduling of idle cycles in the internet. In: Proc. of the 3rd Intl. Workshop on Peer-to-Peer Systems (IPTPS), San Diego, CA, Feb. 2004
Maassen, J., Bal, H.E.: Smartsockets: solving the connectivity problems in grid computing. In: Proc. of Symp. on High Performance Distributed Computing Symposium, Monterey Bay, CA, June 2007
Maymounkov, P., Mazières, D.: Kademlia: a peer-to-peer information system based on the xor metric. In: Proc. of the Workshop on Peer-to-Peer Systems (IPTPS), Cambridge, MA, March 2002
Mislove, A., Druschel, P.: Providing administrative control and autonomy in structured peer-to-peer overlays. In: Proc. of the Workshop on Peer-to-Peer Systems, San Diego, CA, Feb. 2004
Mislove, A., Post, A., Haeberlen, A., Druschel, P.: Experiences in building and operating epost, a reliable peer-to-peer application. In: Proc. of European Conf. on Computer Systems, Leuven, Belgium, April 2006
Rhea, S., Geels, D., Roscoe, T., Kubiatowicz, J.: Handling churn in a DHT. In: Proc. of USENIX Technical Conference, June 2004
Rhea, S., Godfrey, B., Karp, B., Kubiatowicz, J., Ratnasamy, S., Shenker, S., Stoica, I., Opendht, H.Yu.: A public DHT service and its uses. In: Proc. of ACM SIGCOMM, Philadelphia, PA, Aug. 2005
Rowstron, A., Druschel, P.: Pastry: scalable, decentralized object location and routing for large-scale peer-to-peer systems. In: Proc. of the Intl. Conf. on Distributed Systems Platforms (Middleware), Heidelberg, Germany, Nov. 2001
Stoica, I., Morris, R., Liben-Nowell, D., Karger, D.R., Kaashoek, M.F., Dabek, F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup protocol for internet applications. IEEE/ACM Trans. Netw. 11(1), 17–32 (2003)
Universal plug and play in windows xp. http://technet.microsoft.com/en-us/library/bb457049.aspx
Wolinsky, D., Agrawal, A., Boykin, P.O., Davis, J., Ganguly, A., Paramygin, V., Sheng, P., Figueiredo, R.: On the design of virtual machine sandboxes for distributed computing in wide area overlays of virtual workstations. In: Proc. of the Workshop on Virtualization Technology in Distributed Computing, with Supercomputing, Tampa, FL, Nov. 2006
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ganguly, A., Boykin, P.O., Wolinsky, D.I. et al. Improving peer connectivity in wide-area overlays of virtual workstations. Cluster Comput 12, 239–256 (2009). https://doi.org/10.1007/s10586-009-0075-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10586-009-0075-1