Skip to main content
SpringerLink
Log in

An Approach to Forecast Queue Time in Adaptive Scheduling: How to Mediate System Efficiency and Users Satisfaction

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

The minimisation of the total cost of ownership is hard to be faced by the owners of large scale computing systems, without affecting negatively the quality of service for the users. Modern datacenters, often included in distributed environments, appear to be “elastic”, i.e., they are able to shrink or enlarge the number of local physical or virtual resources, also by recruiting them from private/public clouds. This increases the degree of dynamicity, making the infrastructure management more and more complex. Here, we report some advances in the realisation of an adaptive scheduling controller (ASC) which, by interacting with the datacenter resource manager, allows an effective and an efficient usage of resources. In particular, we focus on the mathematical formalisation of the ASC’s kernel that allows to dynamically configure, in a suitable way, the datacenter resources manager. The described formalisation is based on a probabilistic approach that, starting from both a hystorical resources usage and on the actual users request of the datacenter resources, identifies a suitable probability distribution for queue time with the aim to perform a short term forecasting. The case study is the SCoPE datacenter at the University of Naples Federico II.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Aguilar, J., Gelenbe, E.: Task assignment and transaction clustering heuristics for distributed systems. Inf. Sci. 97, 1–2 (1997)

    Article  Google Scholar 

  2. Barone, G., Boccia, V., Bottalico, D., Campagna, R., Carracciuolo, L., Laccetti, G.: An approach to model resources rationalisation in hybrid clouds through users activity characterisation. In: Proceedings of Future Computing 2015—The Seventh International Conference on Future Computational Technologies and Applications, Proceedings of FUTURE COMPUTING, International Conference on Future Computational Technologies and Applications, pp. 48–53 (2015). http://www.thinkmind.org/download.php?articleid=future_computing_2015_3_10_30022

  3. Barone, G., Boccia, V., Bottalico, D., Carracciuolo, L., Doria, A., Laccetti, G.: Modelling the behaviour of an adaptive scheduling controller. In: 2012 Sixth International Conference on Complex, Intelligent and Software Intensive Systems (CISIS), pp. 438–442 (2012). doi:10.1109/CISIS.2012.26

  4. Caruso, P., Laccetti, G., Lapegna, M.: A performance contract system in a grid enabling, component based programming environment. In: Advances in Grid Computing—EGC 2005, European Grid Conference, Amsterdam, The Netherlands, February 14–16, 2005, Revised Selected Papers, pp. 982–992 (2005). doi:10.1007/11508380_100

  5. Casavant, T., Kuhl, J.: A taxonomy of scheduling in general-purpose distributed computing systems. IEEE Trans. Softw. Eng. 14(2), 141–154 (1988). doi:10.1109/32.4634

    Article  Google Scholar 

  6. Federated cloud: Building a grid of clouds. https://www.egi.eu/. Accessed 31 July 2016

  7. Foster, I.: What’s faster a supercomputer or EC2? (2009). http://ianfoster.typepad.com/blog/2009/08/whats-fastera-supercomputer-or-ec2.html. Accessed 31 July 2016

  8. Gkoutioudi, K., Karatza, H.D.: Task cluster scheduling in a grid system. Simul. Model. Pract. Theory 18(9), 1242–1252 (2010)

    Article  Google Scholar 

  9. Italian Grid Infrastructure: http://www.italiangrid.it/. Accessed 31 July 2016

  10. James Allen Fill, M.M.: Stochastic monotonicity and realizable monotonicity. Ann. Probab. 29(2), 938–978 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  11. Laccetti, G., Lapegna, M., Mele, V., Romano, D., Murli, A.: A double adaptive algorithm for multidimensional integration on multicore based HPC systems. Int. J. Parallel Program. 40(4), 397–409 (2012). doi:10.1007/s10766-011-0191-4

    Article  Google Scholar 

  12. Merola, L.: on behalf of S.Co.P.E. Project: The S.Co.P.E. Project. In: Proceedings of the Final Workshop of the Grid Projects of the Italian National Operational Programme 2000–2006 Call 1575, Edited by Consorzio COMETA, pp. 18–35 (2008)

  13. Murli, A., Boccia, V., Carracciuolo, L., D’Amore, L., Laccetti, G., Lapegna, M.: Monitoring and migration of a PETSc-based parallel application for medical imaging in a grid computing PSE. IFIP Int. Fed. Inf. Process. 239, 421–432 (2007). doi:10.1007/978-0-387-73659-4_25

    Google Scholar 

  14. Nadarajah, S.: The waiting time distribution. Comput. Ind. Eng. 53(4), 693–699 (2007). doi:10.1016/j.cie.2007.06.004

    Article  Google Scholar 

  15. Nazir, A., Srensen, S.A.: Cost-benefit analysis of high performance computing infrastructures. In: SOCA, pp. 1–8. IEEE (2010)

  16. Papazachos, Z.C., Karatza, H.D.: The impact of task service time variability on gang scheduling performance in a two-cluster system. Simul. Model. Pract. Theory 17, 1276–1289 (2009)

    Article  Google Scholar 

  17. Potts, C.N., Strusevich, V.A.: Fifty years of scheduling: a survey of milestones. JORS (2009). doi:10.1057/jors.2009.2

    MATH  Google Scholar 

  18. Powers, S.: A study of the impact of scheduling parameters in heterogeneous computing environments. In: Proceedings of the 2014 Winter Simulation Conference. WSC ’14, pp. 933–942. IEEE Press, Piscataway (2014)

  19. Serazzi, G., Calzarossa, M.: Adaptive optimization of a system’s load. IEEE Trans. Softw. Eng. 10(6), 837–845 (1984)

    Article  Google Scholar 

  20. STAT 415 Intro Mathematical Statistics online course. https://onlinecourses.science.psu.edu/stat414/node/3. Accessed 31 July 2016

  21. Sun, H., Cao, Y., Hsu, W.J.: Fair and efficient online adaptive scheduling for multiple sets of parallel applications. In: ICPADS, pp. 64–71. IEEE (2011)

  22. Sztrik, J.: Queueing theory and its applications: a personal view. In: Proceedings of the Third Symposium on Information and Communication Technology, SoICT ’12, pp. 1–1. ACM, New York (2012). doi:10.1145/2350716.2350717

  23. Wilk, M.B., Gnanadesikan, R.: Probability plotting methods for the analysis of data. Biometrika 55(1), 1–17 (1968). doi:10.1093/biomet/55.1.1

    Google Scholar 

Download references

Acknowledgments

This work is part of the activities of a multidisciplinary group (GTT), responsible for the SCoPE infrastructure management. It has been realised thanks to the use of the SCoPE computing infrastructure at the University of Naples, also in the framework of PON ”Rete di Calcolo per SuperB e le altre applicazioni” (ReCaS) project.

Author information

Authors and Affiliations

  1. University of Naples Federico II, Naples, Italy

    G. B. Barone, D. Bottalico, R. Campagna, G. Laccetti & M. Lapegna

  2. Italian National Institute of Nuclear Physics, Naples, Italy

    V. Boccia

  3. Italian National Research Council, Naples, Italy

    L. Carracciuolo

Authors
  1. G. B. Barone
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. Boccia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. Bottalico
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Campagna
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. L. Carracciuolo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. G. Laccetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. M. Lapegna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to L. Carracciuolo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barone, G.B., Boccia, V., Bottalico, D. et al. An Approach to Forecast Queue Time in Adaptive Scheduling: How to Mediate System Efficiency and Users Satisfaction. Int J Parallel Prog 45, 1164–1193 (2017). https://doi.org/10.1007/s10766-016-0457-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-016-0457-y

Keywords

Search

Navigation