Abstract
Current advances in high-speed networks such as ATM and fiber-optics, and software technologies such as the JAVA programming language and WWW tools, have made network-based computing a cost-effective, high-performance distributed computing environment. Metacomputing, a special subset of network-based computing, is a well-integrated execution environment derived by combining diverse and distributed resources such as MPPs, workstations, mass storage, and databases that show a heterogeneous nature in terms of hardware, software, and organization. In this paper we present the Virtual Distributed Computing Environment (VDCE), a metacomputing environment currently being developed at Syracuse University. VDCE provides an efficient web-based approach for developing, evaluating, and visualizing large-scale distributed applications that are based on predefined task libraries on diverse platforms. The VDCE task libraries relieve end-users of tedious task implementations and also support reusability. The VDCE software architecture is described in terms of three modules: (a) the Application Editor, a user-friendly application development environment that generates the Application Flow Graph (AFG) of an application; (b) the Application Scheduler, which provides an efficient task-to-resource mapping of AFG; and (c) the VDCE Runtime System, which is responsible for running and managing application execution and for monitoring the VDCE resources. We present experimental results of an application execution on the VDCE prototype for evaluating the performance of different machine and network configurations. We also show how the VDCE can be used as a problem-solving environment on which large-scale, network-centric applications can be developed by a novice programmer rather than by an expert in low-level details of parallel programming languages.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
T.L. Adam, K. Chandy and J. Dickson, A comparison of list scheduling for parallel processing systems, Communication of ACM 17 (1974) 685-690.
C. Angalano, J. Schopf, R. Wolski and F. Berman, Zoom: a hierarchical representation for heterogeneous applications, Technical Report cs95-451, Computer Science Department, University of California at San Diego (1995).
A. Beguelin, J. Dongara, A. Geist, R. Manchek and V. Sunderam, User Guide to PVM, Oak Ridge National Laboratory and Department of Mathematics and Computer Science, Emory University (1993).
F. Berman, R. Wolski, S. Figueira, J. Schopf and G. Shao, Application-level scheduling on distributed heterogeneous networks, in: Proceedings of Supercomputing' 96(1996).
J.C. Browne, S. Hyder, J. Dongarra, K. Moore and P. Newton, Visual programming and debugging for parallel computing, IEEE Parallel and Distributed Technology 3(1) (1995) 75-83.
R. Butler and E. Lusk, User's guide to the p4 programming system, Mathematics and Computer Science Division, Argonne National Laboratory.
H. Casanova and J. Dongarra, Netsolve: a network server for solving computational science problems, in: Proceedings of Supercomputing' 96(1996).
K. Dincer, World-wide virtual machine: A metacomputing environment integrating World-Wide Web and high performance computing and communication technologies, Ph.D. thesis, Syracuse University (1997).
H. El-Rewini, H. Ali and T. Lewis, Task scheduling in multiprocessing systems, IEEE Computer 28(12) (1995) 27-37.
I. Foster and C. Kesselman, Globus: a metacomputing infrastructure toolkit, in: Proceedings of the Workshop on Environment and Tools for Parallel Scientific Computing(1996).
J. Gehring and A. Reinefeld, MARS - A framework for minimizing the job execution time in a metacomputing environment, Future Generation Computing Systems (1996).
A. Grimshaw and W. Wulf, Legion - A view from 50,000 feet, in: Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing(1996) pp. 89-99.
Y. Kwok and I. Ahmad, Dynamic critical-path scheduling: an effective technique for allocating task graphs to multiprocessors, IEEE Transactions on Parallel and Distributed Systems 7 (1996) 506-521.
M. Litzkow, Miron Livny experience with the condor distributed batch system, in: IEEE Workshop on Experimental Distributed Systems(1990).
Message Passing Interface Forum, MPI: A message-passing interface standard, version 1.0 (May 1994).
P. Newton and J.C. Browne, The CODE 2.0 graphical parallel programming language, in: Proceedings of ACM International Conference on Supercomputing(1992).
S. Park, S. Hariri, Y. Kim, J.S. Harris and R. Yadav, NYNET communication system (NCS): a multithreaded message passing tool over ATM network, in: Proceedings of the 5th IEEE International Symposium on High Performance Distributed Computing(1996) pp. 460-469.
L. Smarr and C. Catlett, Metacomputing, Communications of the ACM 35(6) (June 1992) 44-52.
H. Topcuoglu and S. Hariri, A global computing environment for networked resources, in: Proceedings of International Conference on Parallel Processing(1997) pp. 493-496.
H. Topcuoglu, S. Hariri, W. Furmanski, J. Valente, I. Ra, D. Kim, Y. Kim, X. Bing and B. Ye, The software architecture of a virtual distributed computing environment, in: Proceedings of 6th IEEE International Symposium on High Performance Distributed Computing(1997) pp. 40-49.
J. Weissman and A. Grimshaw, A federated model for scheduling in wide-area-systems, in: Proceedings of the 5th IEEE International Symposium on High Performance Distributed Computing(1996) pp. 542-550.
R. Wolski, C. Anglano, J. Schopf and F. Berman, Developing heterogeneous applications using Zoom and HeNCE, in: Proceedings of the 4th Heterogeneous Computing Workshop(1995).
Y. Yan and X. Zhang, An effective and practical performance prediction model for parallel computing on non-dedicated heterogeneous NOW, Journal of Parallel and Distributed Computing 38 (1996) 63-80.
M. Zaki, W. Li and M. Cierniak, Performance impact of processor and memory heterogeneity in a network of machines, in: Proceedings of the 4th Heterogeneous Computing Workshop(1995).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Topcuoglu, H., Hariri, S., Kim, D. et al. The design and evaluation of a virtual distributed computing environment. Cluster Computing 1, 81–93 (1998). https://doi.org/10.1023/A:1019069012307
Issue Date:
DOI: https://doi.org/10.1023/A:1019069012307