Article

Value-maximizing deadline scheduling and its application to animation rendering

Authors:

Kamalika Chaudhuri,

Norman Salazar,

Cipriano Santos,

Ram Swaminathan,

Yunhong ZhouAuthors Info & Claims

SPAA '05: Proceedings of the seventeenth annual ACM symposium on Parallelism in algorithms and architectures

Pages 299 - 308

https://doi.org/10.1145/1073970.1074019

Published: 18 July 2005 Publication History

Abstract

We describe a new class of utility-maximization scheduling problem with precedence constraints, the disconnected staged scheduling problem (DSSP). DSSP is a nonpreemptive multiprocessor deadline scheduling problem that arises in several commercially-important applications, including animation rendering, protein analysis, and seismic signal processing. DSSP differs from most previously-studied deadline scheduling problems because the graph of precedence constraints among tasks within jobs is disconnected, with one component per job. Another difference is that in practice we often lack accurate estimates of task execution times, and so purely offline solutions are not possible. However we do know the set of jobs and their precedence constraints up front and therefore some offline planning is possible.Our solution decomposes DSSP into an offline job selection phase followed by an online task dispatching phase. We model the former as a knapsack problem and explore several solutions to it, describe a new dispatching algorithm for the latter, and compare both with existing methods. Our theoretical results show that while DSSP is NP-hard and inapproximable in general, our two-phase scheduling method guarantees a good performance bound for many special cases. Our empirical results include an evaluation of scheduling algorithms on a real animation-rendering workload; we present a characterization of this workload in a companion paper. The workload records eight weeks of activity on a 1,000-CPU cluster used to render portions of the full-length animated feature film Shrek 2 in 2004. We show that our improved scheduling algorithms can substantially increase the aggregate value of completed jobs compared to existing practices. Our new task dispatching algorithm LCPF performs well by several metrics, including job completion times as well as the aggregate value of completed jobs.

References

[1]

Thomas L. Adam, K. M. Chandy, and J. R. Dickson. A comparison of list schedules for parallel processing systems. Communications of the ACM, 17(12):685--690, 1974.]]

Digital Library

[2]

Guy E. Blelloch, Phillip B. Gibbons, and Yossi Matias. Provably efficient scheduling for languages with fine-grained parallelism. JACM, 46(2):281--321, 1999.]]

Digital Library

[3]

Robert D. Blumofe. Executing Multithreaded Programs Efficiently. PhD thesis, MIT, September 1995.]]

Digital Library

[4]

Robert D. Blumofe and Charles E. Leiserson. Space-efficient scheduling of multithreaded computations. SIAM J Comput, 27(1):202--229, 1998.]]

Digital Library

[5]

Robert D. Blumofe and Charles E. Leiserson. Scheduling multithreaded computations by work stealing. JACM, 46(5):720--748, September 1999.]]

Digital Library

[6]

Abhijit Bose. Personal communication, September 2004.]]

[7]

Brent. The parallel evaluation of general arithmetic expressions. JACM, 21(2):201--206, 1974.]]

Digital Library

[8]

Peter Brucker. Scheduling Algorithms. Springer, 3rd edition, 2001.]]

Digital Library

[9]

C. Chekuri and M. A. Bender. An efficient approximation algorithm for minimizing makespan on uniformly related machines. In Proc 6th Conf on Integer Programming & Combinatorial Optimization (IPCO'98), pages 383--393. Springer LNCS 1412, 1998.]]

[10]

Chandra Chekuri and Sanjeev Khanna. Approximation algorithms for minimizing average weighted completion time. In Joseph Leung, editor, Handbook of Scheduling: Algorithms, Models, and Performance Analysis. CRC Press, 2004.]]

[11]

F. A. Chudak and D. B. Shamoys. Approximation algorithms for precedence-constrained scheduling problems on parallel machines that run at different speeds. J Algorithms, 30:323--343, 1999.]]

Digital Library

[12]

E. Coffman and Ronald Graham. Optimal scheduling for two processor systems. Acta Informatica, pages 200--213, 1972.]]

Digital Library

[13]

Directed acyclic graph manager (DAGMan) for Condor scheduler. http://www.cs.wisc.edu/condor/dagman/.]]

[14]

Jeffrey Dean and Sanjay Ghemawat. MapReduce: Simplified data processing on large clusters. In OSDI, December 2004.]]

Digital Library

[15]

U. Feige and C. Scheideler. Improved bounds for acyclic job shop scheduling. In STOC, pages 624--633, 1998.]]

Digital Library

[16]

Dror G. Feitelson, Larry Rudolph, Uwe Schwiegelshohn, Kenneth C. Sevcik, and Parkson Wong. Theory and practice in parallel job scheduling. In Proceedings of JSSPP {26}, LNCS 1291, pages 1--34, 1997.]]

Digital Library

[17]

Michael R. Garey and David S. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman, 1979.]]

Digital Library

[18]

Ron Graham. Bounds for certain multiprocessor anomalies. Bell Sys Tech J, 45:1563--1581, 1966.]]

[19]

Ronald Graham. Bounds on multiprocessing time anomalies. SIAM J Appl Math, 17:263--269, 1969.]]

[20]

L. A. Hall. Approximability of flow shop scheduling. Mathematical Programming, 82:175--190, 1998.]]

[21]

Les Hatton. The T experiments: Errors in scientific software. IEEE Computational Sci & Eng, pages 27--38, April 1997.]]

Digital Library

[22]

Les Hatton and Andy Roberts. How accurate is scientific software? IEEE Trans Software Eng, 20(10):785--797, October 1994.]]

Digital Library

[23]

Elisa Heymann, Miquel A. Senar, Emilio Luque, and Miron Livny. Adaptive scheduling for master-worker applications on the computational grid. In Mark Baker Rajkumar Buyya, editor, Proceedings of the First IEEE/ACM International Workshop on Grid Computing (GRID 2000), LNCS 1971, pages 214--227. Springer, 2000.]]

Digital Library

[24]

ILOG Corporation. CPLEX and related software documentation. http://www.ilog.com.]]

[25]

David Johnson. The twelve open problems from {G&J}: Updates. J of Algorithms, 2(4):393--405, 1981.]]

[26]

Workshops on Job Scheduling Strategies for Parallel Processing (JSSPP). http://www.cs.huji.ac.il/~feit/parsched/index.html. Proceedings are published in Springer LNCS series: http://www.cs.huji.ac.il/~feit/parsched/lncs.html.]]

[27]

Richard M. Karp. Reducibility among combinatorial computations. In R. E. Miller and J. W. Thatcher, editors, Complexity of Computer Computations, pages 85--103. Plenum Press, 1972.]]

[28]

Hans Kellerer, Ulrich Pferschy, and David Pisinger. Knapsack Problems. Springer, 2004.]]

[29]

Maxim Sviridenko Klaus Jansen, Roberto Solis-Oba. Makespan minimization in job shops: A linear time approximation scheme. SIAM J on Discr Math, 16(2):288--300, 2003.]]

Digital Library

[30]

Jochen Krallmann, Uwe Schwiegelshohn, and Ramin Yahyapour. On the design and evaluation of job scheduling algorithms. In Proceedings of JSSPP {26}, LNCS 1659, pages 17--42, 1999.]]

Digital Library

[31]

Anurag Kumar and Rajeev Shorey. Performance analysis and scheduling of stochastic fork-join jobs in a multicomputer system. IEEE Trans Par Dist Sys, 4(10), October 1993.]]

Digital Library

[32]

Yu-Kwong Kwok and Ishfaq Ahmad. Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Computing Surveys, 31(4):406--471, December 1999.]]

Digital Library

[33]

J. K. Lenstra and A. H. G. Rinnooy Kan. Complexity of scheduling under precedence constraints. Operations Research, 26(1):22--35, January 1978.]]

Digital Library

[34]

J. K. Lenstra, D. B. Shmoys, and E. Tardos. Approximation algorithms for scheduling unrelated parallel machines. Mathematical Programming, 46:259--272, 1990.]]

Digital Library

[35]

David A. Lifka. The ANL/IBM SP scheduling system. In Proceedings of JSSPP {26}:JSSPP LNCS 949, pages 295--303, 1995.]]

Digital Library

[36]

T. Kasami M. Fuju and N. Ninomiya. Optimal sequence of two equivalent processors. SIAM J Appl Math, 17(3):784--789, 1971.]]

[37]

Lev Markov. Two stage optimization of job scheduling and assignment in heterogeneous compute farms. In Proc. IEEE Workshop on Future Trends in Distributed Computing Systems, pages 119--124, Suzhou, China, May 2004.]]

Digital Library

[38]

Michael Pinedo. Scheduling: Theory, Algorithms, and Systems. Prentice-Hall, 2nd edition, 2002.]]

[39]

Platform Computing. LSF Scheduler. http://www.platform.com/products/LSFfamily/.]]

[40]

Platform Computing. Administering Platform LSF, February 2003. Chapter 14.]]

[41]

S. V. Sevastianov and G. J. Woeginger. Makespan minimization in open shops: A polynomial time approximation scheme. Mathematical Programming, 82:191--198, 1998.]]

[42]

Jiri Sgall. On-line scheduling---a survey. In A. Fiat and G.J. Woeginger, editors, Online Algorithms: The State of the Art, number 1442 in LNCS, pages 196--231. Springer, 1998.]]

Digital Library

[43]

Ling Tan and Zahir Tari. Dynamic task assignment in server farms: Better performance by task grouping. In Proc. of the Int. Symposium on Computers and Communications (ISCC), pages 175--180, July 2002.]]

Digital Library

[44]

Adam Wierman and Mor Harchol-Balter. Classifying scheduling policies with respect to unfairness in an M/GI/1. In SIGMETRICS, pages 238--249, June 2003.]]

Digital Library

[45]

Francis Wray. The parallel implementation of closely coupled numerical algorithms. In A. Adey, editor, Parallel Processing in Engineering Applications. Springer, 1990.]]

[46]

Francis Wray. High performance numerically intensive applications on distributed memory parallel computers. In J. T. Devreese and P. E. Van Camp, editors, Scientific Computing on Supercomputers, volume III. Plenum, 1991.]]

[47]

Yunhong Zhou, Terence Kelly, Janet Wiener, and Eric Anderson. An extended evaluation of two-phase scheduling methods for animation rendering. In Proceedings of JSSPP {26}:JSSPP LNCS, Springer, 2005, to appear.]]

Digital Library

Cited By

Duan YWu J(2022)Reducing Average Job Completion Time for DAG-style Jobs by Adding Idle SlotsGLOBECOM 2022 - 2022 IEEE Global Communications Conference10.1109/GLOBECOM48099.2022.10001196(4504-4509)Online publication date: 4-Dec-2022
https://doi.org/10.1109/GLOBECOM48099.2022.10001196
Grandl RKandula SRao SAkella AKulkarni JKeeton KRoscoe T(2016)GrapheneProceedings of the 12th USENIX conference on Operating Systems Design and Implementation10.5555/3026877.3026885(81-97)Online publication date: 2-Nov-2016
https://dl.acm.org/doi/10.5555/3026877.3026885
Ferguson ABodik PKandula SBoutin EFonseca RFelber PBellosa FBos H(2012)JockeyProceedings of the 7th ACM european conference on Computer Systems10.1145/2168836.2168847(99-112)Online publication date: 10-Apr-2012
https://dl.acm.org/doi/10.1145/2168836.2168847
Show More Cited By

Index Terms

Value-maximizing deadline scheduling and its application to animation rendering
1. Theory of computation
  1. Design and analysis of algorithms
    1. Approximation algorithms analysis
      1. Scheduling algorithms
    2. Online algorithms
      1. Online learning algorithms
        Scheduling algorithms
  2. Theory and algorithms for application domains
    1. Machine learning theory
      1. Reinforcement learning
        Sequential decision making

Recommendations

Online deadline scheduling with preemption penalties

This paper presents a study of the problem of online deadline scheduling under the preemption penalty model of Zheng, Xu, and Zhang (2007). In that model, each preemption incurs a penalty of @r times the weight of the preempted job, where @r>=0 is the ...
Deadline scheduling for animation rendering
Performance evaluation review
Deadline scheduling for animation rendering
SIGMETRICS '05: Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SPAA '05: Proceedings of the seventeenth annual ACM symposium on Parallelism in algorithms and architectures

July 2005

346 pages

ISBN:1581139861

DOI:10.1145/1073970

General Chair:
Phil Gibbons
Intel Research
,
Program Chair:
Paul Spirakis
Research and Academic Computer Technology Institute, Greece

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SPAA05

Sponsor:

SPAA05: 17th ACM Symposium on Parallelism in Algorithms and Architectures 2005

July 18 - 20, 2005

Nevada, Las Vegas, USA

Acceptance Rates

Overall Acceptance Rate 447 of 1,461 submissions, 31%

Upcoming Conference

SPAA '25

Sponsor:
sigact
sigact

37th ACM Symposium on Parallelism in Algorithms and Architectures

July 28 - August 1, 2025

Portland , OR , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
513
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Duan YWu J(2022)Reducing Average Job Completion Time for DAG-style Jobs by Adding Idle SlotsGLOBECOM 2022 - 2022 IEEE Global Communications Conference10.1109/GLOBECOM48099.2022.10001196(4504-4509)Online publication date: 4-Dec-2022
https://doi.org/10.1109/GLOBECOM48099.2022.10001196
Grandl RKandula SRao SAkella AKulkarni JKeeton KRoscoe T(2016)GrapheneProceedings of the 12th USENIX conference on Operating Systems Design and Implementation10.5555/3026877.3026885(81-97)Online publication date: 2-Nov-2016
https://dl.acm.org/doi/10.5555/3026877.3026885
Ferguson ABodik PKandula SBoutin EFonseca RFelber PBellosa FBos H(2012)JockeyProceedings of the 7th ACM european conference on Computer Systems10.1145/2168836.2168847(99-112)Online publication date: 10-Apr-2012
https://dl.acm.org/doi/10.1145/2168836.2168847
AuYoung AVahdat ASnoeren APinfold W(2009)Evaluating the impact of inaccurate information in utility-based schedulingProceedings of the Conference on High Performance Computing Networking, Storage and Analysis10.1145/1654059.1654098(1-12)Online publication date: 14-Nov-2009
https://dl.acm.org/doi/10.1145/1654059.1654098
Keeton KKelly TMerchant ASantos CWiener JZhu XBeyer D(2007)Don't settle for less than the bestProceedings of the 11th USENIX workshop on Hot topics in operating systems10.5555/1361397.1361404(1-6)Online publication date: 7-May-2007
https://dl.acm.org/doi/10.5555/1361397.1361404
Keeton KBeyer DBrau EMerchant ASantos CZhang A(2006)On the road to recoveryACM SIGOPS Operating Systems Review10.1145/1218063.121795840:4(235-248)Online publication date: 18-Apr-2006
https://dl.acm.org/doi/10.1145/1218063.1217958
Keeton KBeyer DBrau EMerchant ASantos CZhang ABerbers YZwaenepoel W(2006)On the road to recoveryProceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 200610.1145/1217935.1217958(235-248)Online publication date: 18-Apr-2006
https://dl.acm.org/doi/10.1145/1217935.1217958
Zhou YKelly TWiener JAnderson E(2005)An extended evaluation of two-phase scheduling methods for animation renderingProceedings of the 11th international conference on Job Scheduling Strategies for Parallel Processing10.1007/11605300_6(123-145)Online publication date: 19-Jun-2005
https://dl.acm.org/doi/10.1007/11605300_6

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten