Abstract
Cluster/distributed computing has become a popular, cost-effective alternative to high-performance parallel computers. Many parallel programming languages and related programming models have become widely accepted on clusters. However, the high communication overhead is a major shortcoming of running parallel applications on cluster/distributed computing environments. To reduce the communication overhead and thus the completion time of a parallel application, this paper introduces and evaluates an efficient Key Message (KM) approach to support parallel computing on cluster computing environments. In this paper, we briefly present the model and algorithm, and then analytical and simulation methods are adopted to evaluate the performance of the algorithm. It demonstrates that when network background load increases or the computation to communication ratio decreases, the analysis results show better improvement on communication of a parallel application over the system which does not use the KM approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
T. E. Anderson, D. E. Culler, D. A. Patterson, and the NOW team. A case for NOW (Networks of Workstations). IEEE Micro, 15:54–64, 1995.
T. E. Anderson, S. S. Owicki, J. B. Saxe, and C. P. Thacker. High speed switch scheduling for local area networks. ACM Transactions on Computer Systems, 11(4):319–352, 1993.
D. Bertsekas and R. Gallager. Data Network, 2nd ed. Prentice-Hall, International Editions, 1992.
R. M. Bethea, B. S. Duran, and T. L. Boullion. Statistical Methods for Enginners and Scientists, 2nd ed. Marcel Dekker, 1984.
A. D. Birrell and B. J. Nelson. Implementing remote procedure calls. ACM Transactions on Computer Systems, 2(1):39–59, 1984.
P. H. Chretienne. Complexity of tree scheduling with interprocessor communication delays. Technical Report, M.A.S.I. 90.5, Universite Pierre et Marie Curie, France, 1990.
Y. C. Chung and S. Ranka. An optimization approach for static scheduling of directed-acyclic graphs on distributed memory multiprocessors. Technical report, Syracuse Univ, USA, 1992.
D. D. Clark, M. L. Lambert, and L. Zhang. NETBLT: A high throughput transport protocol. In Proceedings of the 1987 SIGCOMM Symposium on Communications Architectures and Protocols, pp. 353–359, Sept. 1987.
IBM Corp. Introduction to parallel processing and SP systems 9076 SP1 and 9076 SP2. IBM Technical Report GG24-4344-00, available from http://www.rs6000.ibm.com/resource.
R. Cytron, M. Hind, and W. Hsieh. Automatic generation of DAG parallelism. In Proc. of SIGPLAN Conf. on Programming Language Design and Implementations, pp. 54–68, 1989.
Y. F. Dong, X. Du, and X. D. Zhang. Characterizing and scheduling communication interactions of parallel and local jobs on networks of workstations. Computer Communications, 21(5):470–484, 1998.
H. El-Rewini, T. G. Lewis, and H. H. Ali. Task Scheduling in Parallel and Distributed Systems. PTR Prentice Hall, 1994.
A. Gerasoulis, S. Venugopal, and T. Yang. Clustering task graphs for message passing architectures. In 1990 Proc. of ACM Inter. Conf. on Supercomputing, Amsterdam, pp. 447–456, 1990.
GM, available from http://www.myrinet.com/.
K. Hwang. Advanced Computer Architecture: Parallelism, Scalability, Programmability. McGraw-Hill, 1993.
L. V. Kale, B. Ramkumar, V. Saletore, and A. B. Sinha. Prioritization in parallel symbolic computing. Technical Report 06, Univ. of Iowa and Oregon State Univ, 1993.
K. M. Khalil, K. Q. Luc, and D. V. Wilson. LAN traffic analysis and workload characterization. In Proc. of 15th Conference on Local Computer Networks, pp. 112 –122, 1990.
Myrinet. available from http://www.myri.com.
S. Pakin, V. Karamcheti, and A. A. Chien. Fast Messages: efficient, portable communication for workstation clusters and MPPs. IEEE Concurrency, 5(2):60–73, 1997.
G. Reinman, B. Calder, and T. Austin. Optimizations enabled by a decoupled front end architecture. IEEE Transactions on Computers, 50(4):338–355, 2001.
A. Sinha and L. V. Kale. A load balancing strategy for prioritized execution of tasks. In International Parallel Processing Symposium, Newport Beach, CA, pp. 230–237, 1993.
C. A. Thekkath and H. M. Levy. Limits to low-latency communication on high-speed networks. ACM Transactions on Computer Systems, 11(2):179–203, 1993.
Tuma and J. Jan. Engineering Mathematics Handbook. McGraw-Hill, New York, c1998.
A. Veres and M. Boda. The chaotic nature of tcp congestion control. In The IEEE INFOCOM, 2000.
T. von Eicken. Active messages: An efficient communication architecture for multiprocessors. PhD thesis, U. C. Berkeley, Dept. of Computer Science, USA, 1993.
T. S. Woon. Fast task allocation in distributed systems. Technical Report H17-94, Nanyang Technological University, 1994.
T. Yang, P. Diniz, A. Gerasoulis, and V. Sarkar. Scheduling iterative task computation on message-passing machines. In Proc. of 7th SIAM Conference on Parallel Processing for Scientific Computing, CA, pp. 581–586, Feb. 1995.
M. Zhu, W. Cai, and B.-S. Lee. Key message approach to optimize communication of parallel applications on clusters. Journal of Cluster Computing, 6:253–265, 2003.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhu, M., Cai, W., Lee, BS. et al. A Hybrid Analysis of an Optimization Approach for Cluster Applications. J Supercomput 32, 191–215 (2005). https://doi.org/10.1007/s11227-005-0157-7
Issue Date:
DOI: https://doi.org/10.1007/s11227-005-0157-7