Skip to main content
SpringerLink
Log in
Book cover

International Conference on High-Performance Computing

HiPC 2001: High Performance Computing — HiPC 2001 pp 58–67Cite as

On Job Scheduling for HPC-Clusters and the dynP Scheduler

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2228))

Abstract

Efficient job-scheduling strategies are important to improve the performance and usability of HPC-clusters. In this paper we evaluate job-scheduling strategies (FCFS, SJF, and LJF)used in the resource management system CCS (Computing Center Software). As input for our simulations we use two job sets that are generated from trace files of CCS. Based on the evaluation we introduce the dynP scheduler which combines the three scheduling strategies and dynamically changes between them online. The average estimated runtime of jobs in the waiting queue and two bounds are used as the criterion whether or not dynP switches to a new policy. Obviously the performance of dynP depends on the setting of the two bounds. Diverse parameter pairs were evaluated, and with the proper setting dynP achieves an equal or even better (+9%) performance.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A. Mu’alem and D. Feitelson. Utilization, Predictability, Workloads, and User Runtime Estimates in Scheduling the IBM SP2 with Backfilling. In IEEE Trans. Parallel & Distributed Systems 12(6), pages 529–543, June 2001.

    Article  Google Scholar 

  2. A. Streit. On Job Scheduling for HPC-Clusters and the dynP Scheduler. TR-001-01, PC2-Paderborn Center for Parallel Computing, Paderborn University, July 2001.

    Google Scholar 

  3. D. Feitelson and A. Weil. Utilization and Predictability in Scheduling the IBM SP2 with Backfilling. In Proceedings ofthe 1st Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing (IPPS/SPDP-98), pages 542–547, Los Alamitos, March 1998. IEEE Computer Society.

    Google Scholar 

  4. D.G. Feitelson and L. Rudolph. Metrics and Benchmarking for Parallel Job Scheduling. Lecture Notes in Computer Science, 1459:1–24, 1998.

    Google Scholar 

  5. D.G. Feitelson and M. Naaman. Self-Tuning Systems. In IEEE Software 16(2), pages 52–60, April/Mai 1999.

    Article  Google Scholar 

  6. F. Ramme and T. Romke and K. Kremer. A Distributed Computing Center Software for the Efficient Use of Parallel Computer Systems. Lecture Notes in Computer Science, 797:129–136, 1994.

    Google Scholar 

  7. H. Franke and J. Jann and J. Moreira and P. Pattnaik and M. Jette. An Evaluation of Parallel Job Scheduling for ASCI Blue-Pacific. In Proceedings of SC’ 99, Portland, Oregon, pages 11–18. ACM Press and IEEE Computer Society Press, 1999.

    Google Scholar 

  8. J. Gehring and F. Ramme. Architecture-Independent Request-Scheduling with Tight Waiting-Time Estimations. Lecture Notes in Computer Science, 1162:65–80, 1996.

    Google Scholar 

  9. A. Keller, M. Brune, and A. Reinefeld. Resource Management for High-Performance PC Clusters. Lecture Notes in Computer Science, 1593:270–281, 1999.

    Google Scholar 

  10. A. Keller and A. Reinefeld. CCS Resource Management in Networked HPC Systems. In Proc. of Heterogenous Computing Workshop HCW’98 at IPPS, Orlando, 1998; IEEE Computer Society Press, pages 44–56, 1998.

    Google Scholar 

  11. D.A. Lifka. The ANL/IBM SP Scheduling System. Lecture Notes in Computer Science, 949:295–303, 1995.

    Google Scholar 

  12. J.E. Moreira, H. Franke, W. Chan,and L.L. Fong. A Gang-Scheduling System for ASCI Blue-Pacific. Lecture Notes in Computer Science, 1593, 1999.

    Google Scholar 

  13. J. Skovira, W. Chan, H. Zhou, and D. Lifka. The EASY-LoadLeveler API Project. Lecture Notes in Computer Science, 1162:41–47, 1996.

    Google Scholar 

  14. D. Talby and D.G. Feitelson. Supporting Priorities and Improving Utilization of the IBM SP2 Scheduler Using Slack-Based Backfilling. TR 98-13, Hebrew University, Jerusalem, April 1999.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. PC2 Paderborn Center for Parallel Computing, 33102, Paderborn, Germany

    Achim Streit

Authors
  1. Achim Streit
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Mathematics and Computer Science, University of Paderborn, Fürstenallee 11, 33102, Paderborn, Germany

    Burkhard Monien

  2. Department of EE-Systems, Computer Engineering Division, University of Southern California, 3740 McClintock Avenue, EEB 200C, 90089-2562, Los Angeles, CA, USA

    Viktor K. Prasanna

  3. Independent Consultant, c/o Infosys Ltd., “Mangala”, Kuloor Ferry Road, Kottara, 575006, Mangalore, India

    Sriram Vajapeyam

Rights and permissions

Reprints and permissions

Copyright information

© 2001 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Streit, A. (2001). On Job Scheduling for HPC-Clusters and the dynP Scheduler. In: Monien, B., Prasanna, V.K., Vajapeyam, S. (eds) High Performance Computing — HiPC 2001. HiPC 2001. Lecture Notes in Computer Science, vol 2228. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45307-5_6

Download citation

  • DOI: https://doi.org/10.1007/3-540-45307-5_6

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43009-4

  • Online ISBN: 978-3-540-45307-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation