Skip to main content
SpringerLink
Log in

A comparative study of real workload traces and synthetic workload models for parallel job scheduling

  • Conference paper
  • First Online:
Job Scheduling Strategies for Parallel Processing (JSSPP 1998)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1459))

Included in the following conference series:

Abstract

Two basic approaches are taken when modeling workloads in simulation-based performance evaluation of parallel job scheduling algorithms: (1) a carefully reconstructed trace from a real supercomputer can provide a very realistic job stream, or (2) a flexible synthetic model that attempts to capture the behavior of observed workloads can be devised. Both approaches require that accurate statistical observations be made and that the researcher be aware of the applicability of a given trace for his or her experimental goals.

In this paper, we compare a number of real workload traces and synthetic workload models currently used to evaluate job scheduling and allocation strategies. Our results indicate that the choice of workload model alone — real workload trace versus synthetic workload models — did not significantly affect the relative performance of the algorithms in this study (two scheduling algorithms and three static processor allocation algorithms). Almost all traces and models gave the same ranking of algorithms from best to worst. However, two specific workload characteristics were found to significantly affect algorithm performance: (a) proportion of power-of-two job sizes and (b) degree of correlation between job size and job runtime. When used in the experimental evaluation of resource management algorithms, workloads differing in these two characteristics may lead to discrepant conclusions.

This research was sponsored by NSF grant MIP-9108528.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. R. H. Arpaci, A. C. Dusseau, A. M. Vahdat, L. T. Liu, T. E. Anderson, and D. A. Patterson. The interaction of parallel and sequential workloads on a network of workstations. In Proceedings SIGMETRICS'95, 1995.

    Google Scholar 

  2. S. H. Chiang, R. K. Mansharamini, and M. K. Vernon. Use of application characteristics and limited preemption for run-to-completion parallel processor scheduling policies. In Proceedings SIGMETRICS'94, 1994.

    Google Scholar 

  3. P. J. Chuang and N. F. Tzeng. An efficient submesh allocation strategy for mesh computer systems. In Proceedings IEEE International Conference on Distributed Computer Systems, 1991.

    Google Scholar 

  4. Intel Corp. Paragon Network Queuing System manual. October 1993.

    Google Scholar 

  5. A. B. Downey. A parallel workload model and its implications for processor allocation. In Proceedings HPDC'97, 1997.

    Google Scholar 

  6. . B. Downey. Predicting queue times on space-sharing parallel computers. In Proceedings of the 3rd Workshop on Job Scheduling Strategies for Parallel Processing, IPPS '97, 1997.

    Google Scholar 

  7. D. Feitelson. Packing schemes for gang scheduling. In Proceedings of the 2nd Workshop on Job Scheduling Strategies for Parallel Processing, IPPS '96, 1996.

    Google Scholar 

  8. D. G. Feitelson. A survey of scheduling in multiprogrammed parallel systems. Technical Report RC 19790 (87657), IBM Research Division, T.J. Watson Research Center, Yorktown Heights, NY 10598, October 1994.

    Google Scholar 

  9. D. G. Feitelson and B. Nitzberg. Job characteristics of a production parallel scientific workload on the NASA Ames iPSC/860. In Proceedings of the 1st. Workshop on Job Scheduling Strategies for Parallel Processing, IPPS '95, April 1995.

    Google Scholar 

  10. D. G. Feitelson and L. Rudolph, editors. Job Scheduling Strategies for Parallel Processing. Springer Lecture Notes in Computer Science, 1995–1997.

    Google Scholar 

  11. S. Hotovy. Workload evolution on the Cornell Theory Center IBM SP2. In Proceedings of the 2nd Workshop on Job Scheduling Strategies for Parallel Processing, IPPS '96, 1996.

    Google Scholar 

  12. http://science.nas.nasa.gov/Softwaxe/PBS/pbshome.html. PBS portable batch system.

    Google Scholar 

  13. http://www.llnl.gov/liv_comp/dpcs/. LLNL distributed production control system.

    Google Scholar 

  14. http://www.tc.cornell.edu/Papers/abdullah.jul96/index.html. Extensible Argonne scheduler system (EASY).

    Google Scholar 

  15. J. Jones. NASA Ames NAS, Personal communication, 1997.

    Google Scholar 

  16. P. Krueger, T. Lai, and V. A. Dixit-Radiya. Job scheduling is more important than processor allocation for hypercube computers. IEEE Transactions on Parallel and Distributed Systems, 5(5):488–497, May 1994.

    Article  Google Scholar 

  17. V. M. Lo, K. Windisch, W. Liu, and B. Nitzberg. Non-contiguous processor allocation algorithms for mesh-connected multicomputers. IEEE Transactions on Parallel and Distributed Systems, 8(7):712–726, July 1997.

    Article  Google Scholar 

  18. J. Mache and V. Lo. Minimizing message-passing contention in fragmentation-free processor allocation. In Proceedings of the 10th International Conference on Parallel and Distributed Computing Systems, 1997.

    Google Scholar 

  19. J. Subhlok, T. Gross, and T. Suzuoka. Impacts of job mix on optimizations for space sharing schedulers. In Proceedings of Supercomputing '96, 1996.

    Google Scholar 

  20. M. Wan, R. Moore, G. Kremenek, and K. Steube. A batch scheduler for the Intel Paragon MPP system with a non-contiguous node allocation algorithm. In Proceedings of the 2nd Workshop on Job Scheduling Strategies for Parallel Processing, IPPS '96, 1996.

    Google Scholar 

  21. K. Windisch, V. Lo, D. Feitelson, B. Nitzberg, and R. Moore. A comparison of workload traces from two production parallel machines. In Proceedings of the Sixth Symposium on the Frontiers of Massively Parallel Computation, 1996.

    Google Scholar 

  22. K. Windisch, V. M. Lo, and B. Bose. Contiguous and non-contiguous processor allocation algorithms for k-ary n-cubes. In Proceedings of the International Conference on Parallel Processing, 1995.

    Google Scholar 

  23. K. Windisch, J. V. Miller, and V. M. Lo. Procsimity: an experimental tool for processor allocation and scheduling in highly parallel systems. In Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computation, February 1995.

    Google Scholar 

  24. Y. Zhu. Efficient processor allocation strategies for mesh-connected parallel computers. Journal of Parallel and Distributed Computing, 16:328–337, 1992.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Oregon, 97403, Eugene, OR

    Virginia Lo, Jens Mache & Kurt Windisch

Authors
  1. Virginia Lo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jens Mache
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kurt Windisch
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Dror G. Feitelson Larry Rudolph

Rights and permissions

Reprints and permissions

Copyright information

© 1998 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Lo, V., Mache, J., Windisch, K. (1998). A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In: Feitelson, D.G., Rudolph, L. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 1998. Lecture Notes in Computer Science, vol 1459. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0053979

Download citation

  • DOI: https://doi.org/10.1007/BFb0053979

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-64825-3

  • Online ISBN: 978-3-540-68536-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation