Abstract
With the increasing growth of information through smart devices, enhancing the quality of human life necessitates the adoption of various computational paradigms including, cloud, fog, and edge in the Internet of Things (IoT) network. Among these paradigms, cloud computing, as an emerging technology, extends cloud layer services to the network edge. This enables resource allocation operations to occur closer to the end user, thereby reducing resource processing time and network traffic overhead. Consequently, the resource allocation problem for cloud service providers, in terms of presenting a suitable platform using computational paradigms, is considered a challenge. Also, Energy Efficiency, Heterogeneity, and Scalability are problems of the cloud computing environment. To solve these problems, resource allocation approaches are divided into two methods: auction-based methods (aimed at increasing profits for service providers while ensuring user satisfaction and usability) and optimization-based methods (focused on energy, cost, network exploitation, runtime, and time delay reduction). Hence, this paper presents a comprehensive literature study (CLS) on artificial intelligence methods such as machine and deep learning for resource allocation optimization in computing environments, such as cloud computing, fog computing, and edge computing. Since deep learning methods are widely used in resource allocation problems, this paper also explores resource allocation approaches based on deep learning techniques, such as deep reinforcement learning, Q-learning, reinforcement learning, and online learning, as well as classical learning methods like Bayesian learning. As a main important achievement, the use of deep reinforcement learning-based methods has increased in the fog paradigm in the past few years.







Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
References
Ning Y, Chen X, Wang Z, Li X (2017) An uncertain multi-objective programming model for machine scheduling problem. Int J Mach Learn Cybern 8(5):1493–1500
Gai K, Qiu M, Sun X (2018) A survey on FinTech. J Netw Comput Appl 103:262–273
Liu R, Vellaithurai C, Biswas SS, Gamage TT, Srivastava AK (2015) Analyzing the cyber-physical impact of cyber events on the power grid. IEEE Trans Smart Grid 6(5):2444–2453
Gai K, Qiu M (2018) Optimal resource allocation using reinforcement learning for IoT content-centric services. Appl Soft Comput 70:12–21
Hahanov V, Gharibi W, Man KL, Iemelianov I, Liubarskyi M, Abdullayev V, Litvinova E, Chumachenko S (2018) Cyber-physical technologies: Hype cycle 2017. Cyber Physical Computing for IoT-driven Services, pp 259–272
Shamshirband S, Joloudari JH, Shirkharkolaie SK, Mojrian S, Rahmani F, Mostafavi S, Mansor Z (2021) Game theory and evolutionary optimization approaches applied to resource allocation problems in computing environments: A survey. Math Biosci Eng 18(6):9190–9232
AlQerm I, Shihada B (2016) A cooperative online learning scheme for resource allocation in 5G systems. In 2016 IEEE International conference on communications (ICC) (pp. 1–7). IEEE
Rohmer T, Nakib A, Nafaa A (2015) A learning-based resource allocation approach for P2P streaming systems. IEEE Network 29(1):4–11
Banik D, Ekbal A, Bhattacharyya P (2018) Machine learning based optimized pruning approach for decoding in statistical machine translation. IEEE Access 7:1736–1751
Banerjee A, Kumar V, Banik D (2022) Better Qualitative searching for effecting the performance of machine translation. In International conference on applied machine learning and data analytics (pp. 119–130). Cham: Springer Nature Switzerland
Deo S, Banik D (2022) Text Summarization using textrank and lexrank through latent semantic analysis. In 2022 OITS international conference on information technology (OCIT) (pp. 113–118). IEEE
Banik D, Ekbal A, Bhattacharyya P (2020) Statistical machine translation based on weighted syntax–semantics. Sādhanā 45:1–12
Banik D (2021) Phrase table re-adjustment for statistical machine translation. Int J Speech Technol 24:903–911
Liang L, Ye H, Yu G, Li GY (2019) Deep-learning-based wireless resource allocation with application to vehicular networks. Proc IEEE 108(2):341–356
Nasir YS, Guo D (2019) Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks. IEEE J Sel Areas Commun 37(10):2239–2250
Tong L, Li Y, Gao W (2016) A hierarchical edge cloud architecture for mobile computing. In IEEE INFOCOM 2016-The 35th Annual IEEE international conference on computer communications (pp. 1–9). IEEE
Bitam S, Mellouk A (2012) Its-cloud: cloud computing for intelligent transportation system. In 2012 IEEE global communications conference (GLOBECOM) (pp. 2054–2059). IEEE
Bitam S, Mellouk A, Zeadally S (2015) VANET-cloud: a generic cloud computing model for vehicular Ad Hoc networks. IEEE Wirel Commun 22(1):96–102
Karthiban K, Raj JS (2020) An efficient green computing fair resource allocation in cloud computing using modified deep reinforcement learning algorithm. Soft Comput 24(19):14933–14942
Bandyopadhyay A et al (2023) A Game-Theoretic approach for rendering immersive experiences in the metaverse. Mathematics 11(6):1286
Liu N, Li Z, Xu J, Xu Z, Lin S, Qiu Q, Tang J, Wang Y (2017). A hierarchical framework of cloud resource allocation and power management using deep reinforcement learning. In 2017 IEEE 37th international conference on distributed computing systems (ICDCS) (pp. 372–382). IEEE
Cheng X, Dale C, Liu J (2008) Statistics and social network of youtube videos. In: 2008 16th interntional workshop on quality of service. IEEE, pp 229–238
Stojmenovic I, Wen S (2014) The fog computing paradigm: scenarios and security issues. In 2014 federated conference on computer science and information systems (pp. 1–8). IEEE
Tuli S (2023) Ai and co-simulation driven resource management in fog computing environments. ACM SIGMETRICS Perform Eval Rev 50(3):16–19
Sadek RA (2018) Hybrid energy aware clustered protocol for IoT heterogeneous network. Future Comput Inform J 3(2):166–177
Wang J, Zhao L, Liu J, Kato N (2019) Smart resource allocation for mobile edge computing: A deep reinforcement learning approach. IEEE Trans Emerg Topics Comput 9(3):1529–1541
Wang Z, Zheng S, Ge Q, Li K (2020) Online offloading scheduling and resource allocation algorithms for vehicular edge computing system. IEEE Access 8:52428–52442
Xu X, Fu S, Cai Q, Tian W, Liu W, Dou W, Sun X, Liu AX (2018) Dynamic resource allocation for load balancing in fog environment. Wirel Commun Mob Comp 2018
Salahuddin MA, Al-Fuqaha A, Guizani M (2016) Reinforcement learning for resource provisioning in the vehicular cloud. IEEE Wirel Commun 23(4):128–135
Yousafzai A, Gani A, Noor RM, Sookhak M, Talebian H, Shiraz M, Khan MK (2017) Cloud resource allocation schemes: review, taxonomy, and opportunities. Knowl Inf Syst 50(2):347–381
Othman A, Nayan NA (2019) Efficient admission control and resource allocation mechanisms for public safety communications over 5G network slice. Telecommun Syst 72(4):595–607
Han B, Lianghai J, Schotten HD (2018) Slice as an evolutionary service: Genetic optimization for inter-slice resource management in 5G networks. IEEE Access 6:33137–33147
Bega D, Gramaglia M, Banchs A, Sciancalepore V, Samdanis K, Costa-Perez X (2017) Optimising 5G infrastructure markets: the business of network slicing. In IEEE INFOCOM 2017-IEEE conference on computer communications (pp. 1–9). IEEE
Ghobaei-Arani M, Souri A, Rahmanian AA (2020) Resource management approaches in fog computing: a comprehensive review. J Grid Comput 18(1):1–42
Hameed A et al (2016) A survey and taxonomy on energy efficient resource allocation techniques for cloud computing systems. Computing 98(7):751–774
Beloglazov A, Buyya R, Lee YC, Zomaya A (2011) A taxonomy and survey of energy-efficient data centers and cloud computing systems. Adv Comput 82:47–111
Shuja J, Bilal K, Madani SA, Othman M, Ranjan R, Balaji P, Khan SU (2014) Survey of techniques and architectures for designing energy-efficient data centers. IEEE Syst J 10(2):507–519
Aceto G, Botta A, De Donato W, Pescapè A (2013) Cloud monitoring: a survey. Comput Netw 57(9):2093–2115
Jennings B, Stadler R (2015) Resource management in clouds: Survey and research challenges. J Netw Syst Manage 23(3):567–619
Goyal A, Dadizadeh S (2009) A survey on cloud computing. Univ B C Tech Rep CS 508:55–58
Hussain H et al (2013) A survey on resource allocation in high performance distributed computing systems. Parallel Comput 39(11):709–736
Huang L, Chen H-S, Hu T-T (2013) Survey on resource allocation policy and job scheduling algorithms of cloud computing1. JSW 8(2):480–487
Ahmad RW, Gani A, Hamid SHA, Shiraz M, Xia F, Madani SA (2015) Virtual machine migration in cloud data centers: a review, taxonomy, and open research issues. J Supercomput 71(7):2473–2515
Ahmad RW, Gani A, Hamid SHA, Shiraz M, Yousafzai A, Xia F (2015) A survey on virtual machine migration and server consolidation frameworks for cloud data centers. J Netw Comput Appl 52:11–25
Vinothina V, Sridaran R, Ganapathi P (2012) A survey on resource allocation strategies in cloud computing. Int J Adv Comput Sci Appl 3(6):97–104
Anuradha VP, Sumathi D (2014) A survey on resource allocation strategies in cloud computing. In International conference on information communication and embedded systems (ICICES2014) (pp. 1–7). IEEE
Mohamaddiah MH, Abdullah A, Subramaniam S, Hussin M (2014) A survey on resource allocation and monitoring in cloud computing. Int J Mach Learn Comput 4(1):31–38
Mohan NR, Raj EB (2012) Resource allocation techniques in cloud computing–research challenges for applications. In 2012 fourth international conference on computational intelligence and communication networks (pp. 556–560). IEEE
Castaneda E, Silva A, Gameiro A, Kountouris M (2016) An overview on resource allocation techniques for multi-user MIMO systems. IEEE Commun Surv Tutor 19(1):239–284
Manvi SS, Shyam GK (2014) Resource management for Infrastructure as a Service (IaaS) in cloud computing: a survey. J Netw Comput Appl 41:424–440
Su R et al (2019) Resource allocation for network slicing in 5G telecommunication networks: a survey of principles and models. IEEE Network 33(6):172–179
Zhang X, Debroy S (2023) Resource management in mobile edge computing: a comprehensive survey. ACM Comput Surv 55(13):1–37
Aqib M, Kumar D, Tripathi S (2023) Machine learning for fog computing: review, opportunities and a fog application classifier and scheduler. Wireless Pers Commun 129(2):853–880
Majumder D, Kumar SM (2022) A review on resource allocation methodologies in fog/edge computing. In 2022 8th International conference on smart structures and systems (ICSSS) (pp. 1–04). IEEE
Shi Q, Razaviyayn M, Luo Z-Q, He C (2011) An iteratively weighted MMSE approach to distributed sum-utility maximization for a MIMO interfering broadcast channel. IEEE Trans Signal Process 59(9):4331–4340
Djigal H, Xu J, Liu L, Zhang Y (2022) Machine and deep learning for resource allocation in multi-access edge computing: a survey. IEEE Commun Surv Tutor
Gupta S, Singh N (2023) Toward intelligent resource management in dynamic fog computing-based internet of things environment with deep reinforcement learning: a survey. Int J Commun Syst 36(4):e5411
Vergara J, Botero J, Fletscher L (2023) A comprehensive survey on resource allocation strategies in fog/cloud environments. Sensors 23(9):4413
Shi R, Zhang J, Chu W, Bao Q, Jin X, Gong C, Zhu Q, Yu C, Rosenberg S (2015) MDP and machine learning-based cost-optimization of dynamic resource allocation for network function virtualization. In 2015 IEEE International conference on services computing (pp. 65–73). IEEE
Chowdhury NMK, Boutaba R (2010) A survey of network virtualization. Comput Netw 54(5):862–876
Arkian HR, Atani RE, Diyanat A, Pourkhalili A (2015) A cluster-based vehicular cloud architecture with learning-based resource management. J Supercomput 71(4):1401–1426
Arkian HR, Atani RE, Kamali S (2014) FcVcA: a fuzzy clustering-based vehicular cloud architecture. In 2014 7th International workshop on communication technologies for vehicles (Nets4Cars-Fall) (pp. 24–28). IEEE
Tal I, Muntean GM (2013) User-oriented fuzzy logic-based clustering scheme for vehicular ad-hoc networks. In 2013 IEEE 77th vehicular technology conference (VTC Spring) (pp. 1–5). IEEE
Banik D, Ekbal A, Satapathy SC (2023) Fuzzy influenced process to generate comparable to parallel corpora. ACM transactions on asian and low-resource language information processing, pp 1–23
Gerla M, Tsai JT-C (1995) Multicluster, mobile, multimedia radio network. Wireless Netw 1(3):255–265
Mershad K, Artail H (2013) Finding a STAR in a vehicular cloud. IEEE Intell Transp Syst Mag 5(2):55–68
Salahuddin MA, Al-Fuqaha A, Guizani M (2014) Software-defined networking for rsu clouds in support of the internet of vehicles. IEEE Internet Things J 2(2):133–144
Sahu H, Biswas P, Kumar S, Agrawal D, Pandey S (2023) Resource allocation with edge computing in IoT networks via machine learning and deep learning. In AIP Conference Proceedings (Vol. 2705, No. 1). AIP Publishing
Tripathi KN, Kaur G, Arora N, Agrawal R (2023) An efficient mobile edge computing based resource allocation using optimal double weighted support vector transfer regression. J Grid Comput 21(3):1–20
Reiss C, Wilkes J, Hellerstein JL (2011) Google cluster-usage traces: format+ schema. Google Inc., White Paper, 1:1–14
Chen X, Leng S, Zhang K, Xiong K (2019) A machine-learning based time constrained resource allocation scheme for vehicular fog computing. China Commun 16(11):29–41
Zhao J, Kong M, Li Q, Sun X (2019) Contract-based computing resource management via deep reinforcement learning in vehicular fog computing. IEEE Access 8:3319–3329
Lakhan A, Mohammed MA, Obaid OI, Chakraborty C, Abdulkareem KH, Kadry S (2022) Efficient deep-reinforcement learning aware resource allocation in SDN-enabled fog paradigm. Autom Softw Eng 29:1–25
Talaat FM (2022) Effective prediction and resource allocation method (EPRAM) in fog computing environment for smart healthcare system. Multimed Tools Appl 81(6):8235–8258
Baek J, Kaddoum G (2021) Online partial offloading and task scheduling in SDN-fog networks with deep recurrent reinforcement learning. IEEE Internet Things J 9(13):11578–11589
Aghapour Z, Sharifian S, Taheri H (2023) Task offloading and resource allocation algorithm based on deep reinforcement learning for distributed AI execution tasks in IoT edge computing environments. Comput Netw 223:109577
Jamil B, Ijaz H, Shojafar M, Munir K (2023) IRATS: A DRL-based intelligent priority and deadline-aware online resource allocation and task scheduling algorithm in a vehicular fog network. Ad Hoc Netw 141:103090
Caria M, Das T, Jukan A, Hoffmann M (2015) Divide and conquer: partitioning OSPF networks with SDN. In 2015 IFIP/IEEE International symposium on integrated network management (IM) (pp. 467–474). IEEE
Hazarika B, Singh K, Biswas S, Li C-P (2022) DRL-based resource allocation for computation offloading in IoV networks. IEEE Trans Industr Inf 18(11):8027–8038
Banik D, Rawat S, Thakur A, Parwekar P, Satapathy SC (2023) Automatic approach for mask detection: effective for COVID-19. Soft Comput 27(11):7513–7523
Banik D, Dixit U, Mishra RN, Maharana M, Mishra R, Satapathy SC (2021) Fabula: Hybridized weightage based book recommendation system. In knowledge graphs and semantic web: third Iberoamerican conference and second Indo-American conference, KGSWC 2021, Kingsville, Texas, USA, November 22–24, 2021, Proceedings 3 (pp. 182–196). Springer International Publishing
Banerjee A, Banik D (2023) Pooled hybrid-spectral for hyperspectral image classification. Multimed Tools Appl 82(7):10887–10899
Banerjee A, Banik D (2022) Resnet-2D-ConvLSTM: A means to extract features from hyperspectral image. In International conference on neural information processing (pp. 365–376). Singapore: Springer Nature Singapore
Maity A, Roy SG, Banik D (2022) Evolutionary approaches toward traditional to deep learning-based chatbot. In International conference on machine learning and big data analytics (pp. 159–170). Cham: Springer International Publishing
Banik D (2022) The important influencing factors in machine translation. In International conference on machine learning and big data analytics (pp. 119–126). Cham: Springer International Publishing
Banik D, Pal S, Naskar MN, Bandyopadhyay A (2022) Transformer based technique for high resolution image restoration. In 2022 OITS International conference on information technology (OCIT) (pp. 558–563). IEEE
Kumari S, Gourisaria MK, Das H, Banik D (2023) Deep learning based approach for milk quality prediction. In 2023 11th International conference on emerging trends in engineering & technology-signal and information processing (ICETET-SIP) (pp. 1–6). IEEE
Pati N, Gourisaria MK, Das H, Banik D (2023) Wind speed prediction using machine learning techniques. In 2023 11th International conference on emerging trends in engineering & technology-signal and information processing (ICETET-SIP) (pp. 1–6). IEEE
Omar TR, Kamal AE, Chang JM (2014) Downlink spectrum allocation in 5g hetnets. In: 2014 International wireless communications and mobile computing conference (IWCMC). IEEE, pp 12–17
Rostami S, Arshad K, Rapajic P (2015) A joint resource allocation and link adaptation algorithm with carrier aggregation for 5G LTE-Advanced network. In 2015 22nd International conference on telecommunications (ICT) (pp. 102–106). IEEE.
Kazmi SA, Tran NH, Ho TM, Oo TZ, LeAnh T, Moon S, Hong CS (2015) Resource management in dense heterogeneous networks. In 2015 17th Asia-Pacific network operations and management symposium (APNOMS) (pp. 440–443). IEEE
Hussain F, Anpalagan A, Khwaja AS, Naeem M (2017) Resource allocation and congestion control in clustered M2M communication using Q-learning. Trans Emerg Telecommun Technol 28(4):e3039
Banik D, Sen S, Ekbal A, Bhattacharyya P (2016) Can smt and rbmt improve each other’s performance?-an experiment with english-hindi translation. In Proceedings of the 13th international conference on natural language processing (pp. 10–19)
Banik D, Bhattacharyya P, Ekbal A (2016) Rule based hardware approach for machine transliteration: A first thought. In 2016 Sixth international symposium on embedded computing and system design (ISED) (pp. 192–195). IEEE
Ye H, Li GY, Juang B-HF (2019) Deep reinforcement learning based resource allocation for V2V communications. IEEE Trans Veh Technol 68(4):3163–3173
Ashraf MI, Bennis M, Perfecto C, Saad W (2016) Dynamic proximity-aware resource allocation in vehicle-to-vehicle (V2V) communications. In 2016 IEEE Globecom Workshops (GC Wkshps) (pp. 1–6). IEEE
Banik D, Ekbal A, Bhattacharyya P, Bhattacharyya S (2019) Assembling translations from multi-engine machine translation outputs. Appl Soft Comput 78:230–239
Banik D, Ekbal A, Bhattacharyya P, Bhattacharyya S, Platos J (2019) Statistical-based system combination approach to gain advantages over different machine translation systems. Heliyon 5(9):e02504
Lee W, Kim M, Cho D-H (2018) Deep power control: Transmit power control scheme based on convolutional neural network. IEEE Commun Lett 22(6):1276–1279
Zhao J, Zhang Y, Nie Y, Liu J (2020) Intelligent resource allocation for train-to-train communication: a multi-agent deep reinforcement learning approach. IEEE Access 8:8032–8040
Zhou Q, Hu X, Lin J, Wu Z (2018) Train-to-train communication resource allocation scheme for train control system. In 2018 10th international conference on communication software and networks (ICCSN) (pp. 210–214). IEEE
Ranjan V, Sharma L (2023) Real-time task scheduling and resource scheduling in fog computing using deep learning techniques. In 2023 International conference on distributed computing and electrical circuits and electronics (ICDCECE) (pp. 1–6). IEEE
Chen T, Zhang X, You M, Zheng G, Lambotharan S (2021) A GNN-based supervised learning framework for resource allocation in wireless IoT networks. IEEE Internet Things J 9(3):1712–1724
Pham T-P, Durillo JJ, Fahringer T (2017) Predicting workflow task execution time in the cloud using a two-stage machine learning approach. IEEE Trans Cloud Comput 8(1):256–268
Pietri I, Juve G, Deelman E, Sakellariou R (2014) A performance model to estimate execution time of scientific workflows on the cloud. In 2014 9th workshop on workflows in support of large-scale science (pp. 11–19). IEEE
Shirzad E, Saadatfar H (2022) Job failure prediction in Hadoop based on log file analysis. Int J Comput Appl 44(3):260–269
Saadatfar H, Fadishei H, Deldari H (2012) Predicting job failures in AuverGrid based on workload log analysis. N Gener Comput 30(1):73–94
Gill SS et al (2020) ThermoSim: deep learning based framework for modeling and simulation of thermal-aware resource management for cloud computing environments. J Syst Softw 166:110596
Shapi MKM, Ramli NA, Awalin LJ (2021) Energy consumption prediction by using machine learning for smart building: Case study in Malaysia. Devel Built Environ 5:100037
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Table 4 describes the abbreviations and acronyms related to intelligent computing environments in this study.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hassannataj Joloudari, J., Mojrian, S., Saadatfar, H. et al. Resource allocation problem and artificial intelligence: the state-of-the-art review (2009–2023) and open research challenges. Multimed Tools Appl 83, 67953–67996 (2024). https://doi.org/10.1007/s11042-024-18123-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-024-18123-0