Skip to main content
Springer Nature Link
Log in

Optimization of Execution Parameters of Moldable Ultrasound Workflows Under Incomplete Performance Data

  • Conference paper
  • First Online:
Job Scheduling Strategies for Parallel Processing (JSSPP 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13592))

Included in the following conference series:

  • 407 Accesses

Abstract

Complex ultrasound workflows calculating the outcome of ultrasound procedures such as neurostimulation, tumour ablation or photoacoustic imaging are composed of many computational tasks requiring high performance computing or cloud facilities to be computed in a sensible time. Most of these tasks are written as moldable parallel programs being able to run across various numbers of compute nodes. The number of compute nodes assigned to particular tasks strongly affects the overall execution and queuing times of the whole workflow (makespan) as well as the total computational cost.

This paper employs a genetic algorithm searching for a good resource distribution over the particular tasks, and a cluster simulator evaluating the makespan and cost of the candidate execution schedules. Since the exact execution time cannot be measured for every possible combination of the task, input data size, and assigned resources, several interpolation techniques are used to predict the task duration for a given amount of compute resources. The best execution schedules are eventually submitted to a real cluster with a PBS scheduler to validate the whole technique.

The experimental results confirm the proposed cluster simulator corresponds to a real PBS job scheduler with a sufficient fidelity. The investigation of the interpolation techniques showed that incomplete performance data can successfully be completed by linear and quadratic interpolations keeping the maximum mean error below 10%. Finally, the paper introduces a user defined parameter instructing the genetic algorithm to prefer either the makespan or cost, or find a suitable trade-off.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    IT4Innovations, Czech republic, https://docs.it4i.cz/barbora/introduction/.

  2. 2.

    https://docs.it4i.cz/general/job-priority/.

References

  1. Amdahl, G.M.: Validity of the single processor approach to achieving large scale computing capabilities. In: Proceedings of the April 1820 1967 Spring Joint Computer Conference, vol. 23(4), pp. 483–485 (1967)

    Google Scholar 

  2. Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M.-H., Vahi, K.: Characterization of scientific workflows. In: Third Workshop on Workflows in Support of Large-Scale Science, pp. 1–10. IEEE (2008)

    Google Scholar 

  3. Bleuse, R., Hunold, S., Kedad-Sidhoum, S., Monna, F., Mounie, G., Trystram, D.: Scheduling independent moldable tasks on multi-cores with GPUs. IEEE Trans. Parallel Distrib. Syst. 28(9), 2689–2702 (2017)

    Article  Google Scholar 

  4. Chan, T.F., Mathew, T.P.: Domain decomposition algorithms. Acta Numer 3, 61–143 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chirkin, A.M., et al.: Execution time estimation for workflow scheduling. Future Generat. Comput. Syst. 75 (2017)

    Google Scholar 

  6. Deelman, E., Vahi, K., Juve, G., et al.: Pegasus: a workflow management system for science automation. Future Generat. Comput. Syst. (2014)

    Google Scholar 

  7. Dutot, P.-F., Netto, M.A.S., Goldman, A., Kon, F.: Scheduling moldable BSP tasks. In: Feitelson, D., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 157–172. Springer, Heidelberg (2005). https://doi.org/10.1007/11605300_8

    Chapter  Google Scholar 

  8. Feitelson, D.G., Rudolph, L.: Toward convergence in job schedulers for parallel supercomputers. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1996. LNCS, vol. 1162, pp. 1–26. Springer, Heidelberg (1996). https://doi.org/10.1007/BFb0022284

    Chapter  MATH  Google Scholar 

  9. Gad, A.F.: Geneticalgorithmpython: Building genetic algorithm in python (2021)

    Google Scholar 

  10. Goldberg, D.E.: Genetic algorithms in search, optimization, and machine learning. Addison-Wesley Longman, Boston (1989)

    MATH  Google Scholar 

  11. Henderson, R.L.: Job scheduling under the portable batch system. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1995. LNCS, vol. 949, pp. 279–294. Springer, Heidelberg (1995). https://doi.org/10.1007/3-540-60153-8_34

    Chapter  Google Scholar 

  12. Hovestadt, M., Kao, O., Keller, A., Streit, A.: Scheduling in HPC resource management systems: queuing vs. planning. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 1–20. Springer, Heidelberg (2003). https://doi.org/10.1007/10968987_1

    Chapter  Google Scholar 

  13. Izadkhah, H.: Learning based genetic algorithm for task graph scheduling. Applied Computational Intelligence and Soft Computing (2019)

    Google Scholar 

  14. Jansen, K., Land, F.: Scheduling Monotone Moldable Jobs in Linear Time. In: 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 172–181. IEEE, May 2018

    Google Scholar 

  15. Jaros, J., Rendell, A.P., Treeby, B.E.: Full-wave nonlinear ultrasound simulation on distributed clusters with applications in high-intensity focused ultrasound. Int. J. High Perfor. Comput. Appli. 30(2), 137–155 (2016)

    Article  Google Scholar 

  16. Jaros, M., Jaros, J.: Performance-cost optimization of moldable scientific workflows. In: Klusáček, D., Cirne, W., Rodrigo, G.P. (eds.) JSSPP 2021. LNCS, vol. 12985, pp. 149–167. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88224-2_8

    Chapter  Google Scholar 

  17. Jaros, M., Sasak, T., Treeby, B.E., Jaros, J.: Estimation of execution parameters for k-Wave simulations. In: Kozubek, T., Arbenz, P., Jaroš, J., Říha, L., Šístek, J., Tichý, P. (eds.) HPCSE 2019. LNCS, vol. 12456, pp. 116–134. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67077-1_7

    Chapter  Google Scholar 

  18. Jaros, M., Treeby, E.B., Jaros, J., Georgiou, P.: k-dispatch: A workflow management system for the automated execution of biomedical ultrasound simulations on remote computing resources. In: Platform for Advanced Scientific Computing Conference, pp. 1–10. ACM (2020)

    Google Scholar 

  19. Omara, F.A., Arafa, M.M.: Genetic algorithms for task scheduling problem. J. Paral. Distrib. Comput. 70(1), 13–22 (2010)

    Article  MATH  Google Scholar 

  20. Poudel, J., Lou, Y., Anastasio, M.A.: A survey of computational frameworks for solving the acoustic inverse problem in three-dimensional photoacoustic computed tomography. Phys. Med. Biol. (2019)

    Google Scholar 

  21. Rajaei, H., Dadfar, M.: Comparison Of backfilling algorithms for job scheduling in distributed memory parallel system. In: 2006 Annual Conference and Exposition Proceedings, ASEE Conferences, pp. 11.339.1-11.339.12 (2007)

    Google Scholar 

  22. Robert, Y.: Task Graph Scheduling. In: Encyclopedia of Parallel Computing, pp. 2013–2025 (2011)

    Google Scholar 

  23. Szabo, T.L.: Diagnostic Ultrasound Imaging: Inside Out (2014)

    Google Scholar 

  24. Treeby, B., Cox, B.: k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields. J. Biomed. Opt. 15, 021314 (2010)

    Google Scholar 

  25. Virtanen, P., Gommers, R., Oliphant, T. E. A. O.: Fundamental algorithms for scientific computing in python. SciPy 1.0. Nat. Methods 17, 261–272 (2020)

    Google Scholar 

  26. Wolstencroft, K., Haines, R., Fellows, D., et al.: The taverna workflow suite: designing and executing workflows of web services on the desktop, web or in the cloud. Nucleic Acids Res. 41(W1), W557–W561 (2013)

    Article  Google Scholar 

  27. Ye, D., Chen, D.Z., Zhang, G.: Online scheduling of moldable parallel tasks. J. Sched. 21(6), 647–654 (2018). https://doi.org/10.1007/s10951-018-0556-2

    Article  MathSciNet  MATH  Google Scholar 

  28. Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003). https://doi.org/10.1007/10968987_3

    Chapter  Google Scholar 

Download references

Acknowledgments

This work was supported by the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90140). This work was supported by Brno University of Technology under project numbers IGA FIT/FSI-J-22-7980 Acceleration of Selected Evolutionary Communication Techniques for Solving Combinatoric Tasks and FIT-S-20-6309 Design, Optimization and Evaluation of Application Specific Computer Systems.

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Centre of Excellence IT4Innovations, Brno University of Technology, Brno, Czech Republic

    Marta Jaros & Jiri Jaros

Authors
  1. Marta Jaros
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiri Jaros
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marta Jaros .

Editor information

Editors and Affiliations

  1. CESNET, Prague, Czech Republic

    Dalibor Klusáček

  2. Polytechnic University of Catalonia, Barcelona, Spain

    Corbalán Julita

  3. Apple, Cupertino, CA, USA

    Gonzalo P. Rodrigo

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jaros, M., Jaros, J. (2023). Optimization of Execution Parameters of Moldable Ultrasound Workflows Under Incomplete Performance Data. In: Klusáček, D., Julita, C., Rodrigo, G.P. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2022. Lecture Notes in Computer Science, vol 13592. Springer, Cham. https://doi.org/10.1007/978-3-031-22698-4_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-22698-4_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-22697-7

  • Online ISBN: 978-3-031-22698-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics