Abstract
The addition of time-slicing to space-shared gang scheduling improves the average response time of the jobs in a typical job stream. Recent research has shown that time-slicing is most effective when the jobs admitted for execution fit entirely into physical memory. The question is, how to select and map jobs to make the best use of the available physical memory. Specifically, the achievable degree of multi-programming is limited by the memory requirements, or physical memory pressure, of the admitted jobs. We investigate two techniques for improving the performance of gang scheduling in the presence of memory pressure: 1) a novel backfill approach which improves memory utilization, and 2) an adaptive multi-programming level which balances processor/memory utilization with job response time performance. Our simulations show that these techniques reduce the average wait time and slow-down performance metrics over naive first-come-first-serve methods on a distributed memory parallel system.
This work was supported by NASA NCC2-5268 and by Army High Performance Computing Research Center cooperative agreement number DAAH04-95-2-0003/contract number DAAH04-95-C-0008. Access to computing facilities was provided by AHPCRC, Minnesota Supercomputer Institute.
Chapter PDF
References
A. Batat and D.G. Feitelson. Gang scheduling with memory considerations. In Proceedings of the ISPDP’ 2000. IEEE Computer Society, May 2000.
D.C. Burger, R.S. Hyder, B.P. Miller, and D.A. Wood. Paging tradeoffs in distributed-shared-memory multiprocessors. In Supercomputing’ 94, 1994.
D.G. Feitelson. Packing schemes for gang scheduling. In D.G. Feitelson and L. Rudolph, editors, Job Scheduling Strategies for Parallel Processing, volume 1162, pages 65–88. Springer-Verlag, New York, 1996. LNCS.
D.G. Feitelson. Memory usage in the lanl cm-5 workload. In D.G. Feitelson and L. Rudolph, editors, Job Scheduling Strategies for Parallel Processing, volume 1291, pages 78–94. Springer-Verlag, New York, 1997. LNCS.
D.G. Feitelson and M.A. Jette. Improved utilization and responsiveness with gang scheduling. In D.G. Feitelson and L. Rudolph, editors, Job Scheduling Strategies for Parallel Processing, volume 1291. Springer-Verlag, New York, 1997. LNCS.
D.G. Feitelson and L. Rudolph. Evaluation of design choices for gang scheduling using distributed hierarchical control. J. of Parallel and Distr. Comp., 1996.
D.G. Feitelson and A.M. Weil. Utilization and predictability in scheduling the ibm sp2 with backfilling. In Proceedings of the IPPS/SPDP 1998, pages 542–546. IEEE Computer Society, 1998.
H. Franke, J. Jann, J.E. Moreira, and P. Pattnaik. An evaluation of parallel job scheduling for asci blue-pacific. In Supercomputing’ 99, November 1999.
J.K. Ousterhout. Scheduling techniques for concurrent systems. In 3rd Intl. Conf. Distributed Computing Systems, pages 22–30, Oct 1982.
V.G.J. Peris, M.S. Squillante, and V.K. Naik. Analysis of the impact of memory in distributed parallel processing systems. In Proc. of the ACM SIGMETRICS Conf. on Measurement and Modeling of Computer Systems, pages 5–18, 1994.
U. Schwiegelshohn and R. Yahyapour. Improving first-come-first-serve job scheduling by gang scheduling. In D.G. Feitelson and L. Rudolph, editors, Job Scheduling Strategies for Parallel Processing, volume 1459. Springer-Verlag, New York, March 1998. LNCS.
S. Setia, M.S. Squillante, and V.K. Naik. The impact of job memory requirements on gang-scheduling performance. Technical Report RC 21373, IBM T.J.Watson Research Center, March 1999.
Y. Zhang, H. Franke, J.E. Moreira, and A. Sivasubramaniam. Improving parallel job scheduling by combining gang scheduling and backfilling techniques. Technical Report RC 21579, IBM T.J.Watson Research Center, October 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Leinberger, W., Karypis, G., Kumar, V. (2000). Memory Management Techniques for Gang Scheduling. In: Bode, A., Ludwig, T., Karl, W., Wismüller, R. (eds) Euro-Par 2000 Parallel Processing. Euro-Par 2000. Lecture Notes in Computer Science, vol 1900. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44520-X_34
Download citation
DOI: https://doi.org/10.1007/3-540-44520-X_34
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67956-1
Online ISBN: 978-3-540-44520-3
eBook Packages: Springer Book Archive