Skip to main content
SpringerLink
Log in

Improving reliability and reducing cost of task execution on preemptible VM instances using machine learning approach

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Cloud users can acquire resources in the form of virtual machines (VMs) instances for computing. These instances can be on-demand, reserved and spot instances. Spot-priced virtual machines are offered at the reduced cost compared to on-demand and reserved but are unreliable to use as their availability depends on user’s bid. To use spot instances (preemptible VMs), users have to bid for resources and trade-off between monetary cost and reliability as reliability increases with the increase in cost of execution. The cost of execution can be reduced significantly with the use of preemptible VM instances. These instances are only available until users bid higher in comparison with spot price that is fixed by the cloud providers. Hence, it becomes a critical challenge to minimize the associated cost and increases the reliability for a given deadline. In this article, an algorithm has been designed for predicting the spot price to facilitate the users in bidding. Further, a checkpointing algorithm has been proposed for saving the task’s progress at optimal time intervals by the use of the proposed spot price prediction algorithm. The proposed algorithms in the article emphasize the use of preprocessed data for prediction of prices in short intervals. The prediction algorithm is based on machine learning techniques. It predicts the price and provides a comprehensive comparison for prediction of the prices for long term (12 h) as well as short term (10 min). For predicting the long-term and short-term prices, different machine learning techniques have been used on the basis of least error in prediction. The best suitable machine learning algorithm with least error is selected for prediction as well as checkpointing. Using these algorithms, one can improve reliability and reduce cost of computing on preemptible VM instances significantly. To the best of our knowledge, this is the first attempt of its kind in this field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. AWS Command Line Interface (2018). https://aws.amazon.com/cli/, https://aws.amazon.com/cli/. Accessed 2 Mar 2018

  2. Agarwal S, Mishra AK, Yadav DK (2017) Forecasting price of amazon spot instances using neural networks. Int J Appl Eng Res 12(20):10276–10283

    Google Scholar 

  3. Agmon Ben-Yehuda O, Ben-Yehuda M, Schuster A, Tsafrir D (2013) Deconstructing amazon ec2 spot instance pricing. ACM Trans Econ Comput 1(3):16

    Article  Google Scholar 

  4. Alkharif S, Lee K, Kim H (2018) Time-series analysis for price prediction of opportunistic cloud computing resources. In: Proceedings of the 7th International Conference on Emerging Databases. Springer, pp 221–229

  5. Calheiros RN, Ranjan R, Beloglazov A, Rose CAFD, Buyya R (2011) Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw, Pract Exper 41(1):23–50. https://doi.org/10.1002/spe.995

    Article  Google Scholar 

  6. Chichin S, Vo QB, Kowalczyk R (2017) Towards efficient and truthful market mechanisms for double-sided cloud markets. IEEE Trans Serv Comput 10(1):37–51

    Article  Google Scholar 

  7. Domanal SG, Reddy GRM (2018) An efficient cost optimized scheduling for spot instances in heterogeneous cloud environment. Future Gener Comput Syst 84:11–21

    Article  Google Scholar 

  8. Doulai P, Cahill W (2001) Short-term price forecasting in electric energy market. In: Proceedings of the International Power Engineering Conference, pp 17–19

  9. Guresen E, Kayakutlu G, Daim TU (2011) Using artificial neural network models in stock market index prediction. Exp Syst Appl 38(8):10389–10397

    Article  Google Scholar 

  10. Hasan M, Goraya MS (2018) Fault tolerance in cloud computing environment: a systematic survey. Comput Ind 99:156–172

    Article  Google Scholar 

  11. Jung D, Chin S, Chung K, Yu H, Gil J (2011) An efficient checkpointing scheme using price history of spot instances in cloud computing environment. In: IFIP International Conference on Network and Parallel Computing. Springer, pp 185–200

  12. Karunakaran S, Sundarraj R (2015) Bidding strategies for spot instances in cloud computing markets. IEEE Internet Comput 1:1–1

    Google Scholar 

  13. Khatua S, Mukherjee N (2013) A novel checkpointing scheme for amazon ec2 spot instances. In: 2013 13th IEEE/ACM international symposium on cluster, cloud and grid computing (CCGrid). IEEE, pp 180–181

  14. Latiff MSA, Madni SHH, Abdullahi M et al (2018) Fault tolerance aware scheduling technique for cloud computing environment using dynamic clustering algorithm. Neural Comput Appl 29(1):279–293

    Article  Google Scholar 

  15. Latiff MSA et al (2017) A checkpointed league championship algorithm-based cloud scheduling scheme with secure fault tolerance responsiveness. Appl Soft Comput 61:670–680

    Article  Google Scholar 

  16. Meroufel B, Belalem G (2018) Optimization of checkpointing/recovery strategy in cloud computing with adaptive storage management. Concurr comput Pract Exper 30(24):e4906

    Article  Google Scholar 

  17. Mishra AK, Umrao BK, Yadav DK (2018) A survey on optimal utilization of preemptible vm instances in cloud computing. J Supercomput 74(11):5980–6032

    Article  Google Scholar 

  18. Morgan J (2014) Classification and regression tree analysis. Report no 1, Boston University School of Public Health

  19. Sahay KB, Tripathi M (2014) An analysis of short-term price forecasting of power market by using ann. In: 2014 6th IEEE Power India International Conference (PIICON). IEEE, pp 1–6

  20. Salehan A, Deldari H, Abrishami S (2017) An online valuation-based sealed winner-bid auction game for resource allocation and pricing in clouds. J Supercomput 73(11):4868–4905

    Article  Google Scholar 

  21. Singh VK, Dutta K (2015) Dynamic price prediction for amazon spot instances. In: 2015 48th Hawaii International Conference on System Sciences (HICSS). IEEE, pp 1513–1520

  22. Tang S, Yuan J, Wang C, Li XY (2014) A framework for amazon ec2 bidding strategy under sla constraints. IEEE Trans Parallel Distrib Syst 25(1):211

    Google Scholar 

  23. Turchenko V, Shults V, Turchenko I, Wallace RM, Sheikhalishahi M, Vazquez-Poletti JL, Grandinetti L (2014) Spot price prediction for cloud computingusing neural networks. Int J Comput 12(4):348359

    Google Scholar 

  24. Wallace RM, Turchenko V, Sheikhalishahi M, Turchenko I, Shults V, Vazquez-Poletti JL, Grandinetti L (2013) Applications of neural-based spot market prediction for cloud computing. In: 2013 IEEE 7th International Conference on Intelligent Data Acquisition and Advanced Computing Systems (IDAACS), vol 2. IEEE, pp 710–716

  25. Yi S, Kondo D, Andrzejak A (2010) Reducing costs of spot instances via checkpointing in the amazon elastic compute cloud. In: 2010 IEEE 3rd International Conference on Cloud Computing (CLOUD). IEEE, pp 236–243

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, Motilal Nehru National Institute of Technology Allahabad, Allahabad, 211004, India

    Ashish Kumar Mishra, Dharmendra K. Yadav, Yogesh Kumar & Naman Jain

Authors
  1. Ashish Kumar Mishra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dharmendra K. Yadav
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yogesh Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Naman Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ashish Kumar Mishra.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mishra, A.K., Yadav, D.K., Kumar, Y. et al. Improving reliability and reducing cost of task execution on preemptible VM instances using machine learning approach. J Supercomput 75, 2149–2180 (2019). https://doi.org/10.1007/s11227-018-2717-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-018-2717-7

Keywords

Search

Navigation