Skip to main content
SpringerLink
Log in

On Exploiting Heterogeneity for Cluster Based Parallel Multithreading Using Task Duplication

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Triggered by the ever increasing advancements in processor and networking technology, a cluster of PCs connected by a high-speed network has become a viable and cost-effective platform for the execution of computation intensive parallel multithreaded applications. However, there are two research issues to be tackled in the scheduling problem for PC cluster computing: (1) how to reduce the communication overhead of executing a multithreaded application on the cluster; (2) how to exploit the heterogeneity, which is unavoidable in an evolving PC cluster, for the application. In this paper, we propose to use a duplication based approach in scheduling tasks/threads to a heterogeneous cluster of PCs. In duplication based scheduling, critical tasks are redundantly scheduled to more than one machine, in order to reduce the number of inter-task communication operations. The start times of the succeeding tasks are also reduced. The task duplication process is guided given the system heterogeneity in that the critical tasks are scheduled or replicated in faster machines. The algorithm has been implemented in our experimental application parallelization system for generating multithreaded parallel code executable on a cluster of Pentium PCs. Our experiments, using three numerical applications and one protocol processing kernel (multithreading per request), have indicated that heterogeneity of PC cluster is indeed useful for optimizing the execution of parallel multithreaded programs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. I. Ahmad, Y.-K. Kwok, M.-Y. Wu, and W. Shu. CASCH: a tool for computer aided scheduling. IEEE Concurrency, 8(4):21–33, 2000.

    Google Scholar 

  2. I. Ahmad and Y.-K. Kwok. On exploiting task duplication in parallel program scheduling. IEEE Transactions on Parallel and Distributed Systems, 9(9):872–892, 1998.

    Google Scholar 

  3. CCGrid Web Site, http://www.ccgrid.org/.

  4. M. Cosnard and M. Loi. Automatic task graphs generation techniques. Parallel Processing Letters, 5(4):527–538, 1995.

    Google Scholar 

  5. M. Cosnard, M. Marrakchi, Y. Robert, and D. Trystam. Parallel Gaussian elimination on An MIMD computer. Parallel Computing, 6:275–296, 1988.

    Google Scholar 

  6. S. M. Ehandarkar and H. R. Arabnia. Parallel computer vision on a reconfigurable multiprocessor network. IEEE Transactions on Parallel and Distributed Systems, 8(3):292–309, 1997.

    Google Scholar 

  7. T. Fahringer. Compile-time estimation of communication costs for data parallel programs. Journal of Parallel and Distributed Computing, 39:46–65, 1996.

    Google Scholar 

  8. M. Hughes, M. Schoffner, and D. Hamner, Java Network Programming, 2nd ed. Manning Publications Co., Greenwich, CT, 1999.

    Google Scholar 

  9. Y.-K. Kwok and I. Ahmad. Benchmarking and comparison of the task graph scheduling algorithms. Journal of Parallel and Distributed Computing, 59(3):381–422, 1999.

    Google Scholar 

  10. Y.-K. Kwok and I. Ahmad. Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Computing Surveys, 31(4):406–471, 1999.

    Google Scholar 

  11. M. A. Palis, J.-C. Liou, and D. S. L. Wei. Task clustering and scheduling for distributed memory parallel architectures. IEEE Transactions on Parallel and Distributed Systems, 7(1):46–55, 1996.

    Google Scholar 

  12. C. Papadimitriou and M. Yannakakis. Toward an architecture independent analysis of parallel algorithms. SIAM Journal on Computing, 19(2):322–328, 1990.

    Google Scholar 

  13. G. F. Pfister. In Search of Clusters, 2nd ed. Prentice Hall, Englewood Cliffs, NJ, 1998.

    Google Scholar 

  14. B. Shirazi, H. Chen, and J. Marquis. Comparative study of task duplication static scheduling versus clustering and non-clustering techniques. Concurrency: Practice and Experience, 7(5):371–390, 1995.

    Google Scholar 

  15. H. J. Siegel, H. G. Dietz, and J. K. Antonio. Software support for heterogeneous computing. ACM Computing Surveys, 28(1):237–239, 1996.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong

    Yu-Kwong Kwok

Authors
  1. Yu-Kwong Kwok
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kwok, YK. On Exploiting Heterogeneity for Cluster Based Parallel Multithreading Using Task Duplication. The Journal of Supercomputing 25, 63–72 (2003). https://doi.org/10.1023/A:1022856623227

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1022856623227

Search

Navigation