Skip to main content
SpringerLink
Log in

New Challenges of Parallel Job Scheduling

  • Conference paper
Job Scheduling Strategies for Parallel Processing (JSSPP 2007)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 4942))

Included in the following conference series:

Abstract

The workshop on job scheduling strategies for parallel processing (JSSPP) studies the myriad aspects of managing resources on parallel and distributed computers. These studies typically focus on large-scale computing environments, where allocation and management of computing resources present numerous challenges. Traditionally, such systems consisted of massively parallel supercomputers, or more recently, large clusters of commodity processor nodes. These systems are characterized by architectures that are largely homogeneous and workloads that are dominated by both computation and communication-intensive applications. Indeed, the large majority of the articles in the first ten JSSPP workshops dealt with such systems and addressed issues such as queuing systems and supercomputer workloads.

In this paper, we discuss some of the recent developments in parallel computing technologies that depart from this traditional domain of problems. In particular, we identify several recent and influential technologies that could have a significant impact on the future of research on parallel scheduling. We discuss some of the more specific research challenges that these technologies introduce to the JSSPP community, and propose to enhance the scope of future JSSPP workshops to include these topics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Antonopoulos, C.D., Nikolopoulos, D.S., Papatheodorou, T.S.: Scheduling algorithms with bus bandwidth considerations for SMPs. In: 32nd International Conference on Parallel Processing (ICPP), October 2003, Kaohsiung, Taiwan (2003), www.cs.wm.edu/~dsn/papers/icpp03.pdf

  2. Banachowski, S.A., Brandt, S.A.: The BEST Scheduler for Integrated Processing of Best-Effort and Soft Real-Time Processes. In: Multimedia Computing and Networking (MMCN), January 2002, San Jose, CA (2002), www.cse.ucsc.edu/~sbanacho/papers/banachowski-mmcn02.ps

  3. Becker, D., Wolf, F., Frings, W., Geimer, M., Wylie, B.J.N., Mohr, B.: Automatic trace-based performance analysis of metacomputing applications. In: 21st International Parallel and Distributed Processing Symposium (IPDPS), IEEE Computer Society, Los Alamitos (2007)

    Google Scholar 

  4. Berman, F., Casanova, H., Chien, A., Cooper, K., Dail, H., Dasgupta, A., Deng, W., Dongarra, J., Johnsson, L., Kennedy, K., Koelbel, C., Liu, B., Liu, X., Mandal, A., Marin, G., Mazina, M., Mellor-Crummey, J., Mendes, C., Olugbile, A., Patel, J.M., Reed, D., Shi, Z., Sievert, O., Xia, H., YarKhan, A.: New grid scheduling and rescheduling methods in the grads project. International Journal of Parallel Programming 33(2), 209–229 (2005)

    Article  Google Scholar 

  5. Bucur, A.I.D., Epema, D.: Scheduling policies for processor co-allocation in multicluster systems. IEEE Transactions on Parallel and Distributed Systems 18, 958–972 (2007)

    Article  Google Scholar 

  6. Bulpin, J.R., Pratt, I.A.: Multiprogramming performance of the Pentium 4 with hyper-threading. In: Second Annual Workshop on Duplicating, Deconstruction and Debunking (WDDD), pp. 53–62, Munchen, Germany (June 2004), www.ece.wisc.edu/~wddd/2004/06_bulpin.pdf

  7. Chase, J.S., Anderson, D.C., Thakar, P.N., Vahdat, A.M., Doyle, R.P.: Managing energy and server resources in hosting centers. SIGOPS Operating Systems Review 35(5), 103–116 (2001)

    Article  Google Scholar 

  8. Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. In: Symposium on Operating Systems Design and Implementation (OSDI), p. 10, Berkeley, CA, USA, USENIX Association (2004)

    Google Scholar 

  9. Ernemann, C., Hamscher, V., Schwiegelshohn, U., Streit, A., Yahyapour, R.: Enhanced algorithms for multi-site scheduling. In: Parashar, M. (ed.) GRID 2002. LNCS, vol. 2536, pp. 219–231. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  10. Ernemann, C., Song, B., Yahyapour, R.: Scaling of workload traces. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 166–182. Springer, Heidelberg (2003), www.cs.huji.ac.il/~feit/parsched/

    Chapter  Google Scholar 

  11. Ernemann, C., Yahyapour, R.: Applying economic scheduling methods to grid environments. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid Resource Management - State of the Art and Future Trends, pp. 491–506. Kluwer Academic Publishers, Dordrecht (2003)

    Google Scholar 

  12. Etsion, Y., Tsafrir, D., Feitelson, D.G.: Desktop scheduling: How can we know what the user wants. In: 14th ACM International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV), County Cork, Ireland, June 2004, pp. 110–115 (2004), www.cs.huji.ac.il/~feit/papers/HuCpri04NOSSDAV.pdf

  13. Feitelson, D.G.: Metrics for parallel job scheduling and their convergence. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 188–1205. Springer, Heidelberg (2001), www.cs.huji.ac.il/~feit/parsched/

    Chapter  Google Scholar 

  14. Feitelson, D.G., Rudolph, L.: Gang scheduling performance benefits for fine-grain synchronization. Journal of Parallel and Distributed Computing 16(4), 306–318 (1992), www.cs.huji.ac.il/~feit/papers/GangPerf92JPDC.ps.gz

    Article  MATH  Google Scholar 

  15. Feitelson, D.G., Rudolph, L., Schwigelshohn, U.: Parallel job scheduling – A status report. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 1–16. Springer, Heidelberg (2005), www.cs.huji.ac.il/~feit/parsched/

    Google Scholar 

  16. Flautner, K., Uhlig, R., Reinhardt, S., Mudge, T.: Thread-level parallelism and interactive performance of desktop applications. In: Ninth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), November 2000, pp. 129–138 (2000), www.eecs.umich.edu/~tnm/papers/asplos00.pdf

  17. Foster, I.T.: The anatomy of the grid: Enabling scalable virtual organizations. In: Sakellariou, R., Keane, J.A., Gurd, J.R., Freeman, L. (eds.) Euro-Par 2001. LNCS, vol. 2150, pp. 1–4. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  18. Foster, I.T., Kesselman, C. (eds.): The GRID: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  19. Fox, G.C., Gannon, D.: Concurrency and Computation: Practice and Experience. Special issue: Workflow in grid systems: Editorials 18(10), 1009–1019 (2006)

    Google Scholar 

  20. Frachtenberg, E.: Process Scheduling for the Parallel Desktop. In: Proceedings of the International Symposium on Parallel Architectures, Algorithms, and Networks (I-SPAN 2005), December 2005, Las Vegas, NV (2005)

    Google Scholar 

  21. Frachtenberg, E., Etsion, Y.: Hardware parallelism: Are operating systems ready (case studies in mis-scheduling). In: Second Workshop on the Interaction between Operating Systems and Computer Architecture (WIOSCA 2006), In conjunction with ISCA-33, June 2006, Boston, MA (June 2006)

    Google Scholar 

  22. Frachtenberg, E., Feitelson, D.G.: Pitfalls in parallel job scheduling evaluation. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 257–282. Springer, Heidelberg (2005), www.cs.huji.ac.il/~etcs/pubs/

    Chapter  Google Scholar 

  23. Frachtenberg, E., Feitelson, D.G, Petrini, F., Fernandez, J.: Flexible CoScheduling: Mitigating load imbalance and improving utilization of heterogeneous resources. In: 17th International Parallel and Distributed Processing Symposium (IPDPS), April 2003, Nice, France (2003), www.cs.huji.ac.il/~etcs/pubs/

  24. Frachtenberg, E., Feitelson, D.G., Petrini, F., Fernandez, J.: Adaptive parallel job scheduling with flexible coscheduling. IEEE Transactions on Parallel and Distributed Systems 16(11), 1066–1077 (2005), www.cs.huji.ac.il/~etcs/pubs/

    Article  Google Scholar 

  25. Franke, C., Lepping, J., Schwiegelshohn, U.: On advantages of scheduling using genetic fuzzy systems. In: Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2006. LNCS, vol. 4376, pp. 68–93. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  26. Goyal, B., Lawande, S.: Grid Revolution: An Introduction to Enterprise Grid Computing. The McGraw-Hill Companies, New York (2006)

    Google Scholar 

  27. Peter Hofstee, H.: Power efficient processor architecture and the Cell processor. In: 11th International Symposium on High-Performance Computer Architecture, February 2005, San Francisco, CA (2005), www.hpcaconf.org/hpca11/papers/25_hofstee-cellprocessor_final.pdf

  28. Hotovy, S.: Workload evolution on the cornell theory center IBM SP2. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1996 and JSSPP 1996. LNCS, vol. 1162, pp. 27–40. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  29. Huedo, E., Montero, R.S., Llorente, I.M.: Evaluating the reliability of computational grids from the end user’s point of view. Journal of Systems Architecture 52(12), 727–736 (2006)

    Article  Google Scholar 

  30. Jones, T., Tuel, W., Brenner, L., Fier, J., Caffrey, P., Dawson, S., Neely, R., Blackmore, R., Maskell, B., Tomlinson, P., Roberts, M.: Improving the scalability of parallel jobs by adding parallel awareness to the operating system. In: 15th IEEE/ACM Supercomputing, Phoenix, AZ, November 2003, ACM Press and IEEE Computer Society Press (2003), www.sc-conference.org/sc2003/paperpdfs/pap136.pdf

  31. Kanellos, M.: Designer puts 96 cores on single chip (October 2004), http://news.com.com/2100-1006_3-5399128.html

  32. Li, J., Yahyapour, R.: Learning-based negotiation strategies for grid scheduling. In: IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2006), Singapore, pp. 567–583. IEEE Press, Los Alamitos (2006)

    Google Scholar 

  33. Lu, Y.-H., Benini, L., De Micheli, G.: Low-power task scheduling for multiple devices. In: Proceedings of the Eighth International Workshop on Hardware/software codesign (CODES 2000), pp. 39–43. ACM, New York, NY, USA (2000)

    Chapter  Google Scholar 

  34. MacLaren, J., Sakellariou, R., Garibaldi, J., Ouelhadj, D., Krishnakumar, K.T.: Towards service level agreement based scheduling on the grid. In: Proceedings of the Workshop on Planning and Scheduling for Web and Grid Services, July 2004, pp. 100–102, Whistler, BC, Canada (2004)

    Google Scholar 

  35. Medernach, E.: Workload analysis of a cluster in a grid environment (JSSPP-page). In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 36–61. Springer, Heidelberg (2005), www.cs.huji.ac.il/~feit/parsched/

    Chapter  Google Scholar 

  36. Nieh, J., Hanko, J.G., Northcutt, J.D., Wall, G.A.: SVR4 UNIX scheduler unacceptable for multimedia applications. In: Fourth ACM International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV), November 1993 (1993), citeseer.ist.psu.edu/443381.html

  37. Ousterhout, J.K.: Scheduling techniques for concurrent systems. In: Third International Conference on Distributed Computing Systems, October 1982, Miami, FL, pp. 22–30 (1982)

    Google Scholar 

  38. Parallel workload archive, http://www.cs.huji.ac.il/labs/parallel/workload

  39. Pinedo, M.: Scheduling: Theory, Algorithms, and Systems, 2nd edn. Prentice-Hall, New Jersey (2002)

    Google Scholar 

  40. Piro, R.M., Guarise, A., Werbrouck, A.: An economy-based accounting infrastructure for the datagrid. In: Proceedings of the Fourth International Workshop on Grid Computing (GRID 2003), p. 202. IEEE Computer Society Press, Washington, DC, USA (2003)

    Google Scholar 

  41. Reinders, J.: Intel Threading Building Blocks. O’Reilly and Associates (July 2007)

    Google Scholar 

  42. Robertazzi, T.G., Yu, D.: Multi-Source Grid Scheduling for Divisible Loads. In: Proceedings of the 40th Annual Conference on Information Sciences and Systems, March 2006, pp. 188–191 (2006)

    Google Scholar 

  43. Sabin, G., Sadayappan, P.: Unfairness metrics for space-sharing parallel job schedulers. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 238–256. Springer, Heidelberg (2005), www.cs.huji.ac.il/~feit/parsched/

    Chapter  Google Scholar 

  44. Schwiegelshohn, U., Yahyapour, R.: Fairness in parallel job scheduling. Journal of Scheduling 3(5), 297–320 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  45. Schwiegelshohn, U., Yahyapour, R.: Attributes for communication between grid scheduling instances. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid Resource Management – State of the Art and Future Trends, pp. 41–52. Kluwer Academic Publishers, Dordrecht (2003)

    Google Scholar 

  46. Shankland, S.: Azul’s first-generation Java servers go on sale (April 2005), http://news.com.com/2100-1010_3-5673193.html?tag=nl

  47. Shin, Y., Choi, K.: Power conscious fixed priority scheduling for hard real-time systems. In: Proceedings of the 36th ACM/IEEE conference on Design automation (DAC 1999), pp. 134–139. ACM, New York, NY, USA (1999)

    Chapter  Google Scholar 

  48. Siddiqui, M., Villazón, A., Fahringer, T.: Grid capacity planning with negotiation-based advance reservation for optimized qos. In: Löwe, W., Südholt, M. (eds.) SC 2006. LNCS, vol. 4089, p. 103. Springer, Heidelberg (2006)

    Google Scholar 

  49. Snavely, A., Tullsen, D.M.: Symbiotic jobscheduling for a simultaneous multithreading processor. In: Ninth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), November 2000, Cambridge, MA, pp. 234–244 (2000), http://citeseer.ist.psu.edu/338334.html

  50. Sodan, A.C., Lan, L.: LOMARC—Lookahead matchmaking for multi-resource coscheduling. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 288–315. Springer, Heidelberg (2005), www.cs.huji.ac.il/~feit/parsched/

    Google Scholar 

  51. Tchernykh, A., Ramírez, J.M., Avetisyan, A., Kuzjurin, N., Grushin, D., Zhuk, S.: Two level job-scheduling strategies for a computational grid. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Waśniewski, J. (eds.) PPAM 2005. LNCS, vol. 3911, pp. 774–781. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  52. Terstyanszky, G., Kiss, T., Delaitre, T., Winter, S., Kacsuk, P.: Service-oriented production grids and user support. In: Gannon, D., Badia, R.M. (eds.) Proceedings of the Seventh IEEE/ACM international conference on Grid computing, Barcelona, pp. 323–324 (2006)

    Google Scholar 

  53. Venugopal, S., Buyya, R., Winton, L.: A grid service broker for scheduling distributed data-oriented applications on global grids. In: Proceedings of the Second Workshop on Middleware for grid computing (MGC 2004), pp. 75–80. ACM, New York, NY, USA (2004)

    Chapter  Google Scholar 

  54. Williams, S., Shalf, J., Oliker, L., Kamil, S., Husbands, P., Yelick, K.: The potential of the cell processor for scientific computing. In: Proceedings of the Third Conference on Computing frontiers (CF 2006), pp. 9–20. ACM, New York, NY, USA (2006)

    Chapter  Google Scholar 

  55. Wiseman, Y., Feitelson, D.G.: Paired gang scheduling. IEEE Transactions on Parallel and Distributed Systems 14(6), 581–592 (2003)

    Article  Google Scholar 

  56. Xie, T., Qin, X.: Enhancing security of real-time applications on grids through dynamic scheduling. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 219–237. Springer, Heidelberg (2005), www.cs.huji.ac.il/~feit/parsched/

    Chapter  Google Scholar 

  57. Ye, G., Rao, R., Li, M.: A multiobjective resources scheduling approach based on genetic algorithms in grid environment. In: Proceedings of the Fifth International Conference on Grid and Cooperative Computing Workshops (GCCW 2006), pp. 504–509. IEEE Computer Society, Los Alamitos (2006)

    Chapter  Google Scholar 

  58. Yoshimoto, K., Kovatch, P., Andrews, P.: Co-scheduling with user-settable reservations. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 146–156. Springer, Heidelberg (2005), www.cs.huji.ac.il/~feit/parsched/

    Chapter  Google Scholar 

  59. Zhou, D., Lo, V.: Wave scheduler: Scheduling for faster turnaround time in peer-based desktop grid systems. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 194–218. Springer, Heidelberg (2005), www.cs.huji.ac.il/~feit/parsched/

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Powerset, Inc.,  

    Eitan Frachtenberg

  2. Robotics Research Institute, University Dortmund, Otto-Hahn-Strasse 8, 44221, Dortmund, Germany

    Uwe Schwiegelshohn

Authors
  1. Eitan Frachtenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Uwe Schwiegelshohn
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Eitan Frachtenberg Uwe Schwiegelshohn

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Frachtenberg, E., Schwiegelshohn, U. (2008). New Challenges of Parallel Job Scheduling. In: Frachtenberg, E., Schwiegelshohn, U. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2007. Lecture Notes in Computer Science, vol 4942. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78699-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-78699-3_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-78698-6

  • Online ISBN: 978-3-540-78699-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation