Skip to main content
SpringerLink
Log in

WOW: Self-organizing Wide Area Overlay Networks of Virtual Workstations

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

This paper describes WOW, a distributed system that combines virtual machine, overlay networking and peer-to-peer techniques to create scalable wide-area networks of virtual workstations for high-throughput computing. The system is architected to: facilitate the addition of nodes to a pool of resources through the use of system virtual machines (VMs) and self-organizing virtual network links; to maintain IP connectivity even if VMs migrate across network domains; and to present to end-users and applications an environment that is functionally identical to a local-area network or cluster of workstations. We describe IPOP, a network virtualization technique that builds upon a novel, extensible user-level decentralized technique to discover, establish and maintain overlay links to tunnel IP packets over different transports (including UDP and TCP) and across firewalls. We evaluate latency and bandwidth overheads of IPOP and also time taken for a new node to become fully-routable over the virtual network. We also report on several experiments conducted on a testbed WOW deployment with 118 P2P router nodes over PlanetLab and 33 VMware-based VM nodes distributed across six firewalled domains. Experiments show that the testbed delivers good performance for two unmodified, representative benchmarks drawn from the life-sciences domain. We also demonstrate that the system is capable of seamlessly maintaining connectivity at the virtual IP layer for typical client/server applications (NFS, SSH, PBS) when VMs migrate across a WAN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. distributed.net. http://distributed.net/

  2. fightaids@home. http://fightaidsathome.scripps.edu/

  3. Open science Grid web site. http://www.opensciencegrid.org

  4. Openpbs web site. http://www.openpbs.org

  5. Adabala, S., Chadha, V., Chawla, P., Figueiredo, R.J., Fortes, J.A.B., et al.: From virtualized resources to virtual computing Grids: The In-VIGO system. Future Generation Computing Systems, special issue on Complex Problem-Solving Environments for Grid Computing 21(6) (Apr 2005)

  6. Al-Dmour, N.A., Teahan, W.J.: Parcop: A Parcop: A decentralized peer-to-peer computing system. In: Proc. of the 3rd Intl. Symp. on Algorithms, Models and Tools for Parallel Computing on Heterogenous Networks (Jul 2004)

  7. Anderson, D.P., Cobb, J., Korpella, E., Lebofsky, M., Werthimer, D.: Seti@home: An experiment in public-resource computing. Commun. ACM 11(45), 56–61 (2002)

    Article  Google Scholar 

  8. Anderson, T.E., Culler, D.E., Patterson, D.A. et al.: A case for network of workstations: NOW. IEEE Micro 15, 54–64 (February 1995)

    Article  Google Scholar 

  9. Andrade, N., Costa, L., Germoglio, G., Cirne, W.: Peer-to-peer Grid computing with the ourgrid community. In: Proc. of the 23rd Brazilian Symp. on Computer Networks (May 2005)

  10. Aumage, O., Hofman, R.F.H., Bal, H.E.: Netibis: An efficient and dynamic communication system for heterogenous Grids. In: In Proc. of CCGrid 2005, Cardiff, UK (May 2005)

  11. Bailey, T., Elkan, C.: Fitting a mixture model by expectation maximization to discover motifs in biopolymers. In: Proc. of the Second Intl. Conference on Intelligent Systems for Molecular Biology, pp. 28–36, AAAI, Menlo Park, CA (1994)

    Google Scholar 

  12. Barham, P., Dragovic, B., Fraser, K., Hand, S., et.al.: Xen and the art of virtualization. In: Proc. of the 19th ACM symposium on Operating systems principles, pp. 164–177, Bolton Landing, NY (2003)

  13. Becker, D., Sterling, T., Savarese, D., Dorband, J., Ranawake, U., Packer, C.: Beowulf: A parallel workstation for scientific computation. In: Proc. of the Intl. Conference on Parallel Processing (ICPP) (1995)

  14. Boykin, P.O., Bridgewater, J.S.A., Kong, J., Lozev, K., Rezaei, B.: Brunet software library. http://brunet.ee.ucla.edu/brunet

  15. Calder, B., Chien, A.A., Wang, J., Yang, D.: The entropia virtual machine for desktop Grids. In: CSE technical report CS2003-0773, University of California, San Diego, San Diego, CA (Oct 2003)

  16. Cao, J., Kwong, O.M.K., Wang, X., Cai, W.: A peer-to-peer approach to task scheduling in computation Grid. Intl. J. Grid and Utility Computing 1(1) (2005)

  17. Chakravarti, A.J., Baumgartner, G., Lauria, M.: The organic Grid: Self-organizing computation on a peer-to-peer network. IEEE Trans. Syst. Man Cybern. 35(3) (May 2005)

  18. Cheema, A.S., Muhammad, M., Gupta, I.: Peer-to-peer discovery of computational resources for Grid applications. In: Proc. of the 6th IEEE/ACM Workshop on Grid Computing, Seattle, WA (November 2005)

  19. Clark, C., Fraser, K., Hand, S., Hansen, J., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proc. 2nd Symp. on Networked Systems Design and Implementation (NSDI), Boston, MA (May 2005)

  20. Denis, A., Aumage, O., Hofman, R., Verstoep, K., Kielmann, T., Bal, H.E.: Wide-area communication for Grids: An integrated solution to connectivity, performance and security problems. In: Proc. of the 13th Intl. Symp. on High Performance Distributed Computing, Honolulu, Hawaii (Jun 2004)

  21. Bal, H.E., et al.: The distributed asci supercomputer project. In: ACM Special Interest Group, Operating Systems Review, vol 34, no. 4 (Oct 2000)

  22. Figueiredo, R.J., Dinda, P., Fortes, J.A.B.: A case for Grid computing on virtual machines. In: Proc. of the 23rd IEEE Intl. Conference on Distributed Computing Systems (ICDCS), Providence, Rhode Island (May 2003)

    Google Scholar 

  23. Ford, B., Srisuresh, P., Kegel, D.: Peer-to-peer communication across network address translators. In: Proc. of the 2005 USENIX Annual Technical Conference (USENIX ’05), Anaheim, California (Apr 2005)

  24. Foster, I., Iamnitchi, A.: On death, taxes, and the convergence of peer-to-peer and Grid computing. In: Proc. of the 2nd Intl. Workshop on Peer-to-Peer Systems (IPTPS), Berkeley, CA (Feb 2003)

  25. Foster, I., Kesselman, C.: Globus: A metacomputing infrastructure toolkit. Int. J. Supercomput. Appl. 11(2), 115–128 (1997)

    Article  Google Scholar 

  26. Ganguly, A., Agrawal, A., Boykin, P.O., Figueiredo, R.J.: IP over P2P: Enabling self-configuring virtual IP networks for Grid computing. In: Proc. of the IEEE Intl. Parallel and Distributed Processing Symp. (IPDPS), Rhodes, Greece (Apr 2006)

  27. Grimshaw, A.S., Wulf, W.A.: Legion: Flexible support for wide-area computing. In: Proc. of the 7th ACM SIGOPS European Workshop, Ireland (1996)

  28. Guha, S., Takeda, Y., Francis, P. NUTSS: a SIP based approach to UDP and TCP connectivity. In: Proc. of Special Interest Group on Data Communications (SIGCOMM) Workshops, Portland, OR, pp. 43–48 (Aug 2004)

  29. Iamnitchi, A., Foster, I., Nurmi, D.C.: A peer-to-peer approach to resource location in Grid environments. In: Proc. of the 11th Symp. on High Performance Distributed Computing, Edinburgh, UK (Aug 2002)

  30. Jiang, X., Xu, D.: Violin: Virtual internetworking on overlay infrastructure. In: Proc. of the 2nd Intl. Symp. on Parallel and Distributed Processing and Applications (Dec 2004)

  31. Kannan, J., Kubota, A., Lakshminarayan, K.: Supporting legacy applications over i3. Computer Science Division, University of California, Berkeley (Jun 2004) Technical report no. UCB/CSD-04-1342

  32. Keahey, K., Foster, I., Freeman, T., Zhang, X., Galron, D.: Virtual workspaces in the Grid. In: Proc. of Europar, Lisbon, Portugal (Sep 2005)

  33. Kleinberg, J.: Nature 406, 845 (2000)

    Article  Google Scholar 

  34. Kozuch, M., Satyanarayanan, M.: Internet suspend/resume. In: 4th IEEE Workshop on Mobile Computing Systems and Applications (WMCSA 2002), 20-21 June 2002, Callicoon, NY, USA, p. 40. IEEE Computer Society (2002)

  35. Krsul, I., Ganguly, A., Zhang, J., Fortes, J.A.B., Figueiredo, R.J.: Vmplants:providing and managing virtual machine execution environments for Grid computing. In: Proc. of SC 2004, Pittsburgh, PA (Nov 2004)

  36. Litzkow, M., Livny, M., Mutka, M.: Condor - a hunter of idle workstations. In: Proc. 8th IEEE Intl. Conference on Distributed Computing Systems (ICDCS) (Jun 1988)

  37. Lo, V., Zappala, D., Zhou, D., Liu, Y., Zhao, S.: Cluster computing on the fly: P2P scheduling of idle cycles in the internet. In: Proc. of the Third Intl. Workshop on Peer-to-Peer Systems (IPTPS), San Diego, CA (Feb 2004)

  38. Olsen, G., Matsuda, H., Haggstrom, R., Overbeek, R.: fastdnaml: a tool for construction of phyllogenetic trees of dna sequences using maximum likelihood. Comput. Appl. Biosci. 10, 41–48 (1994)

    Google Scholar 

  39. Raman, R., Livny, M., Solomon, M.: Matchmaking: Distributed resource management for high throughput computing. In: Proc. 7th IEEE Intl. Symp. on High Performance Distributed Computing (HPDC), Chicago, IL (Jul 1998)

  40. Rezmerita, A., Morlier, T., Neri, V., Cappello, F.: Private virtual cluster: Infrastructure and protocol for instant Grids. In: In Proc. of the European Conf. on Parallel Computing, Dresden, Germany (Aug 2006)

  41. Rosenberg, J., Weinberger, J., Huitema, C., Mahy, R.: Rfc 3489 - stun - simple traversal of user data protocol through network address translators (2003) http://www.ietf.org/rfc/rfc3489.txt

  42. Santhanam, S., Elango, P., Arpaci-Dusseau, A., Livny, M.: Deploying virtual machines as sandboxes for the Grid. In: Proc., Second USENIX Workshop on Real, Large Distributed Systems (WORLDS), pp. 7–12, San Francisco, CA (Dec 2005)

  43. Sapuntzakis, C., Chandra, R., Pfaff, B., Chow, J., Lam, M., Rosenblum, M.: Optimizing the migration of virtual computers. In: Proc. of USENIX Operating System Design and Implementation (OSDI) (2002)

  44. Sapuntzakis, C., Brumley, D., Chandra, R., Zeldovich, N., Chow, J., Lam, M.S., Rosenblum, M.: Virtual appliances for deploying and maintaining software. In: Proceedings of the Seventeenth Large Installation Systems Administration Conference (LISA 2003) (October 2003)

  45. Shoykhet, A., Lange, J., Dinda, P.: A system for virtual machine. Northwestern University (Jul 2004) Technical Report NWU-CS-04-39

  46. Smith, J., Nair, R.: Virtual Machines: Versatile Platforms for Systems and Processes. Morgan Kaufmann, San Mateo, CA (2005)

    MATH  Google Scholar 

  47. Son, S., Allcock, B., Livny, M.: Codo: Firewall traversal by cooperative on-demand opening. In: Proc. of 14th Intl. Symp. on High Performance Distributed Computing (HPDC) (2005)

  48. Son, S., Livny, M.: Recovering internet symmetry in distributed computing. In: Proc. of the 3rd Intl. Symp. on Cluster COmputing and the Grid (May 2003)

  49. Stewart, C., Hart, D., Berry, D., Olsen, G., Wernert, E., Fischer, W.: Parallel implementation and performance of fastdnaml – a program for maximum likelihood phylogenetic inference. In: Proc. of IEEE/ACM Supercomputing Conference (SC01) (2001)

  50. Stoica, I., Adkins, D., Zhuang, S., Shenker, S., Surana, S.: Internet indirection infrastructure. IEEE/ACM Trans. Netw. 12(2) (Apr 2004)

  51. Sugerman, J., Venkitachalan, G., Lim, B.H.: Virtualizing i/o devices on vmware workstation’s hosted virtual machine monitor. In: Proc. of the USENIX Annual Technical Conference (Jun 2001)

  52. Sundararaj, A., Dinda, P.: Towards virtual networks for virtual machine Grid computing. In: Phroc. of the 3rd USENIX Virtual Machine Research and Technology Symp., San Jose, CA (May 2004)

  53. Sundararaj, A., Gupta, A., Dinda, P. Dynamic topology adaptation of virtual networks of virtual machines. In: Proc. Seventh Workshop on Languages, Compilers and Run-time Support for Scalable Systems (LCR) (Oct 2004)

  54. Sunderam, V., Dongarra, J., Geist, A., Manchek, R.: The pvm concurrent computing system: Evolution, experiences, and trends. Parallel Comput. 20(4), 531–547 (Apr 1994)

    Article  MATH  Google Scholar 

  55. Tannenbaum, T., Wright, D., Miller, K., Livny, M.: Beowulf Cluster Computing with Linux, Chapter Condor – A Distributed Job Scheduler. MIT, Cambridge, MA (2002)

    Google Scholar 

  56. Therning, N., Bengtsson, L.: Jalapeno-decentralized Grid computing using peer-to-peer technology. In: In Proc. of the 2nd Conference on Computing Frontiers, Ischia, Italy (2005)

  57. Tsugawa, M., Fortes, J.A.B. A virtual network (ViNe) architecture for Grid computing. In: Proc. of the IEEE Intl. Parallel and Distributed Processing Symp. (IPDPS), Rhodes, Greece (Jun 2006)

  58. Xu, D., Ruth, P., Rhee, J., Kennel, R., Goasguen, S.: Autonomic adaptation of virtual distributed environments in a multi-domain infrastructure. In: Proc. of IEEE Intl Symp. on High-Performance Distributed Computing (HPDC), Hot Topics Session, Paris, France (Jun 2006)

Download references

Author information

Authors and Affiliations

  1. Advanced Computing and Information Systems Laboratory, University of Florida, Gainesville, FL, 32601, USA

    A. Ganguly, A. Agrawal, P. O. Boykin & R. J. Figueiredo

Authors
  1. A. Ganguly
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Agrawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. O. Boykin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. J. Figueiredo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Ganguly.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ganguly, A., Agrawal, A., Boykin, P.O. et al. WOW: Self-organizing Wide Area Overlay Networks of Virtual Workstations. J Grid Computing 5, 151–172 (2007). https://doi.org/10.1007/s10723-007-9076-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-007-9076-6

Keywords

Search

Navigation