Skip to main content
SpringerLink
Log in

Dueling Double Deep Q Network Strategy in MEC for Smart Internet of Vehicles Edge Computing Networks

  • Research
  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Advancing in communication systems requires nearby devices to act as networks when devices are not in use. Such technology is mobile edge computing, which provides enormous communication services in the network. In this research, we explore a multiuser smart Internet of Vehicles (IoV) network with mobile edge computing (MEC) assistance, where the first edge server can assist in completing the intense computing jobs from the vehicular users. Many currently available works for MEC networks primarily concentrate on minimising system latency to ensure the quality of service (QoS) for users by designing some offloading strategies. Still, they need to account for the retail prices from the server and, as a result, the budgetary constraints of the users. To solve this problem, we present a Dueling Double Deep Q Network (D3QN) with an Optimal Stopping Theory (OST) strategy that helps to solve the multi-task joint edge problems and minimises the offloading problems in MEC-based IoV networks. The multi-task-offloading model aims to increase the likelihood of offloading to the ideal servers by utilising the OST characteristics. Lastly, simulators show how the proposed methods perform better than the traditional ones. The findings demonstrate that the suggested offloading techniques may be successfully applied in mobile nodes and significantly cut the anticipated time required to process the workloads.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. Mahmood, Omar Abdulkareem, et al. "Distributed Edge Computing for Resource Allocation in Smart Cities Based on the IoT." Information 13.7: 328,2022.

  2. J., X., X., Z., S., H. P., & K., G, The Alleviation of Perceptual Blindness During Driving in Urban Areas Guided by Saccades Recommendation. IEEE Transactions on Intelligent Transportation Systems, 1-11,2022.

  3. J., X., S., H. P., X., Z., & J., H, The Improvement of Road Driving Safety Guided by Visual Inattentional Blindness. IEEE Transactions on Intelligent Transportation Systems, 23(6), 4972-4981,2022.

  4. Chen, Y.: Research on collaborative innovation of key common technologies in new energy vehicle industry based on digital twin technology. Energy Reports 8, 15399–15407 (2022)

    Article  Google Scholar 

  5. Xu, J., Guo, K., & Sun, P. Z. H, Driving Performance Under Violations of Traffic Rules: Novice Vs. Experienced Drivers. IEEE Transactions on Intelligent Vehicles,2022.

  6. Li, H., Huang, Q., Huang, J., & Susilo, W, Public-Key Authenticated Encryption With Keyword Search Supporting Constant Trapdoor Generation and Fast Search. IEEE Transactions on Information Forensics and Security, 18, 396-410,2023.

  7. Sun, G., Sheng, L., Luo, L., & Yu, H, Game Theoretic Approach for Multipriority Data Transmission in 5G Vehicular Networks. IEEE Transactions on Intelligent Transportation Systems, 23(12), 24672-24685,2022.

  8. Sun, G., Song, L., Yu, H., Chang, V., Du, X.,... Guizani, M, V2V Routing in a VANET Based on the Autoregressive Integrated Moving Average Model. IEEE Transactions on Vehicular Technology, 68(1), 908-922,2019.

  9. Be¸stepe, F.; Yildirim, S.Ö. Acceptance of IoT-based and sustainability-oriented smart city services: A mixed methods study. Sustain. Cities Soc, 80, 103794,2022.

  10. Kuru, K.: Planning the Future of Smart Cities with Swarms of Fully Autonomous Unmanned Aerial Vehicles Using a Novel Framework. IEEE Access 9, 6571–6595 (2021)

    Article  Google Scholar 

  11. Sun, G., Zhang, Y., Yu, H., Du, X., & Guizani, M, Intersection Fog-Based Distributed Routing for V2V Communication in Urban Vehicular Ad Hoc Networks. IEEE Transactions on Intelligent Transportation Systems, 21(6), 2409-2426,2020.

  12. Liu, X., Zhou, G., Kong, M., Yin, Z., Li, X., Yin, L.,... Zheng, W, Developing Multi-Labelled Corpus of Twitter Short Texts: A Semi-Automatic Method. Systems, 11(8),2023.

  13. Yu, S., Zhao, C., Song, L., Li, Y., & Du, Y, Understanding traffic bottlenecks of long freeway tunnels based on a novel location-dependent lighting-related car-following model. Tunnelling and Underground Space Technology, 136,2023.

  14. Zhang, X., Pan, W., Scattolini, R., Yu, S., & Xu, X, Robust tube-based model predictive control with Koopman operators. Automatica, 137,2022.

  15. Abdellah, A.R., Mahmood, O.A., Kirichek, R., Paramonov, A., Koucheryavy, A.: Machine Learning Algorithm for Delay Prediction in IoT and Tactile Internet. Future Internet 13, 304 (2021)

    Article  Google Scholar 

  16. Zhang, X., Wang, Y., Yuan, X., Shen, Y., & Lu, Z, Adaptive Dynamic Surface Control With Disturbance Observers for Battery/Supercapacitor-Based Hybrid Energy Sources in Electric Vehicles. IEEE Transactions on Transportation Electrification, 9(4), 5165-5181,2023.

  17. Zhang, X., Wang, Z., & Lu, Z, Multi-objective load dispatch for microgrid with electric vehicles using modified gravitational search and particle swarm optimization algorithm. Applied Energy, 306,2022.

  18. Liu, X., Wang, S., Lu, S., Yin, Z., Li, X., Yin, L.,... Zheng, W,Adapting Feature Selection Algorithms for the Classification of Chinese Texts. Systems, 11(9),2023.

  19. Zhang, Y., Li, S., Wang, S., Wang, X., & Duan, H, Distributed bearing-based formation maneuver control of fixed-wing UAVs by finite-time orientation estimation. Aerospace Science and Technology, 136,2023.

  20. Dai, W., Zhou, X., Li, D., Zhu, S., & Wang, X, Hybrid Parallel Stochastic Configuration Networks for Industrial Data Analytics. IEEE Transactions on Industrial Informatics, 18(4), 2331-2341,2022.

  21. Ren, J., Wang, H., Hou, T., Zheng, S., Tang, C.: Collaborative Edge Computing and Caching with Deep Reinforcement Learning Decision Agents. IEEE Access 8, 120604–120612 (2021)

    Article  Google Scholar 

  22. Tang, M., Wong, V.W.S.: Deep Reinforcement Learning for Task Offloading in Mobile Edge Computing Systems. IEEE Trans. Mob. Comput 21, 1985–1997 (2022)

    Article  Google Scholar 

  23. Xiao, Y., & Konak, A, The heterogeneous green vehicle routing and scheduling problem with time-varying traffic congestion. Transportation Research Part E: Logistics and Transportation Review, 88, 146-166,2016.

  24. Zhu, B., Sun, Y., Zhao, J., Han, J., Zhang, P.,... Fan, T, A Critical Scenario Search Method for Intelligent Vehicle Testing Based on the Social Cognitive Optimization Algorithm. IEEE Transactions on Intelligent Transportation Systems, 24(8), 7974-7986,2023.

  25. Guo, R., Liu, H., & Liu, D, When Deep Learning-Based Soft Sensors Encounter Reliability Challenges: A Practical Knowledge-Guided Adversarial Attack and Its Defense. IEEE Transactions on Industrial Informatics,2023.

  26. Li, Y., Qi, F., Wang, Z., Yu, X., Shao, S.: Distributed Edge Computing Offloading Algorithm Based on Deep Reinforcement Learning. IEEE Access 8, 85204–85215 (2020)

    Article  Google Scholar 

  27. Ning, Z., Zhang, K., Wang, X., Guo, L., Hu, X., Huang, J., Hu, B., Kwok, R.Y.K.: Intelligent Edge Computing in Internet of Vehicles: A Joint Computation Offloading and Caching Solution. IEEE Trans. Intell. Transp. Syst 22, 2212–2225 (2021)

    Article  Google Scholar 

  28. Li, S., Chen, J., Peng, W., Shi, X., & Bu, W, A vehicle detection method based on disparity segmentation. Multimedia Tools and Applications, 82(13), 19643-19655,2023.

  29. Jiang, B., Zhao, Y., Dong, J., & Hu, J, Analysis of the influence of trust in opposing opinions: An inclusiveness-degree based Signed Deffuant–Weisbush model. Information Fusion, 104,2024.

  30. Yin, Y., Guo, Y., Su, Q., & Wang, Z, Task Allocation of Multiple Unmanned Aerial Vehicles Based on Deep Transfer Reinforcement Learning. Drones, 6(8),2022.

  31. Zhao, K., Jia, Z., Jia, F., & Shao, H, Multi-scale integrated deep self-attention network for predicting remaining useful life of aero-engine. Engineering Applications of Artificial Intelligence, 120,2023.

  32. Xiao, Z., Shu, J., Jiang, H., Min, G., Chen, H.,... Han, Z, Perception Task Offloading With Collaborative Computation for Autonomous Driving. IEEE Journal on Selected Areas in Communications, 41(2), 457-473,2023.

  33. Xiao, Z., Shu, J., Jiang, H., Min, G., Liang, J.,... Iyengar, A, Toward Collaborative Occlusion-free Perception in Connected Autonomous Vehicles. IEEE Transactions on Mobile Computing,2023.

  34. Ma, J., & Hu, J, Safe consensus control of cooperative-competitive multi-agent systems via differential privacy. Kybernetika, 58(3), 426-439,2022.

  35. Wang, Q., Hu, J., Wu, Y., & Zhao, Y, Output synchronization of wide-area heterogeneous multi-agent systems over intermittent clustered networks. Information Sciences, 619, 263-275,2023.

  36. Fu, Y., Li, C., Yu, F. R., Luan, T. H., & Zhao, P, An Incentive Mechanism of Incorporating Supervision Game for Federated Learning in Autonomous Driving. IEEE Transactions on Intelligent Transportation Systems, 24(12), 14800-14812,2023.

  37. Hedayati, S., Maleki, N., Olsson, T., Ahlgren, F., Seyednezhad, M., Berahmand, K.: MapReduce scheduling algorithms in Hadoop: a systematic study. Journal of Cloud Computing 12(1), 143 (2023)

    Article  Google Scholar 

  38. Doumari, S.A., Berahmand, K. and Ebadi, M.J., 2023. Early and High-Accuracy Diagnosis of Parkinson’s Disease: Outcomes of a New Model. Computational and Mathematical Methods in Medicine, 2023.

  39. Zamani, M.G., Nikoo, M.R., Jahanshahi, S., Barzegar, R. and Meydani, A., 2023. Forecasting water quality variable using deep learning and weighted averaging ensemble models. Environmental Science and Pollution Research, pp.1-25.

  40. Choupanzadeh, R. and Zadehgol, A., 2023. A Deep Neural Network Modeling Methodology for Efficient EMC Assessment of Shielding Enclosures Using MECA-Generated RCS Training Data. IEEE Transactions on Electromagnetic Compatibility.

  41. Yue, W., Li, C., Wang, S., Xue, N., & Wu, J, Cooperative Incident Management in Mixed Traffic of CAVs and Human-Driven Vehicles. IEEE Transactions on Intelligent Transportation Systems, 24(11), 12462-12476,2023.

  42. Ding, C., Li, C., Xiong, Z., Li, Z., & Liang, Q.,Intelligent Identification of Moving Trajectory of Autonomous Vehicle Based on Friction Nano-Generator. IEEE Transactions on Intelligent Transportation Systems,2023.

  43. Min, H., Li, Y., Wu, X., Wang, W., Chen, L.,... Zhao, X, A Measurement Scheduling Method for Multi-vehicle Cooperative Localization Considering State Correlation. Vehicular Communications,2023.

  44. Zhao, X., Fang, Y., Min, H., Wu, X., Wang, W.,... Teixeira, R, Potential sources of sensor data anomalies for autonomous vehicles: An overview from road vehicle safety perspective. Expert Systems with Applications,2024.

  45. Mou, J., Gao, K., Duan, P., Li, J., Garg, A.,... Sharma, R, A Machine Learning Approach for Energy-Efficient Intelligent Transportation Scheduling Problem in a Real-World Dynamic Circumstances. IEEE Transactions on Intelligent Transportation Systems, 24(12), 15527-15539,2023.

  46. Sheng, H., Wang, S., Chen, H., Yang, D., Huang, Y., Shen, J.,... Ke, W, Discriminative Feature Learning with Co-occurrence Attention Network for Vehicle ReID. IEEE Transactions on Circuits and Systems for Video Technology,2023.

  47. Cao, B., Li, Z., Liu, X., Lv, Z., & He, H, Mobility-Aware Multiobjective Task Offloading for Vehicular Edge Computing in Digital Twin Environment. IEEE Journal on Selected Areas in Communications, 41(10), 3046-3055,2023.

  48. Lu, J., & Osorio, C, On the Analytical Probabilistic Modeling of Flow Transmission Across Nodes in Transportation Networks. Transportation Research Record, 2676(12), 209-225,2022.

  49. Xuemin, Z., Ying, R., Zenggang, X., Haitao, D., Fang, X.,... Yuan, L,Resource-Constrained and Socially Selfish-Based Incentive Algorithm for Socially Aware Networks. Journal of Signal Processing Systems,2023.

  50. Wu, Q., Fang, J., Zeng, J., Wen, J., & Luo, F,,Monte Carlo Simulation-Based Robust Workflow Scheduling for Spot Instances in Cloud Environments. Tsinghua Science and Technology, 29(1), 112-126,2024.

  51. Xu, H., Han, S., Li, X., & Han, Z, Anomaly Traffic Detection Based on Communication-Efficient Federated Learning in Space-Air-Ground Integration Network. IEEE Transactions on Wireless Communications,2023.

  52. Ma, B., Liu, Z., Dang, Q., Zhao, W., Wang, J., Cheng, Y.,... Yuan, Z, Deep Reinforcement Learning of UAV Tracking Control Under Wind Disturbances Environments. IEEE Transactions on Instrumentation and Measurement, 72,2023.

Download references

Funding

No funding was obtained for this study

Author information

Authors and Affiliations

  1. College of Electronic and Information Engineering, Tongji University, Shanghai, 201804, China

    Haotian Pang

  2. School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou, 450001, China

    Zhanwei Wang

Authors
  1. Haotian Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhanwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

H.P: Conceptualization, Methodology, Formal analysis, Supervision, Writing - original draft, Writing - review & editing.

Z.W: Writing - original draft, Writing - review & editing.

Corresponding author

Correspondence to Zhanwei Wang.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for Publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pang, H., Wang, Z. Dueling Double Deep Q Network Strategy in MEC for Smart Internet of Vehicles Edge Computing Networks. J Grid Computing 22, 37 (2024). https://doi.org/10.1007/s10723-024-09752-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-024-09752-8

Keywords

Search

Navigation