research-article

Deep Reinforcement Learning (DRL)-based Resource Management in Software-Defined and Virtualized Vehicular Ad Hoc Networks

Authors:

Azzedine BoukercheAuthors Info & Claims

DIVANet '17: Proceedings of the 6th ACM Symposium on Development and Analysis of Intelligent Vehicular Networks and Applications

Pages 47 - 54

https://doi.org/10.1145/3132340.3132355

Published: 21 November 2017 Publication History

Abstract

Vehicular ad hoc networks (VANETs) have attracted great interests from both industry and academia. The developments of VANETs are heavily influenced by information and communications technologies, which have fueled a plethora of innovations in various areas, including networking, caching and computing. Nevertheless, these important enabling technologies have traditionally been studied separately in the existing works on vehicular networks. In this paper, we propose an integrated framework that can enable dynamic orchestration of networking, caching and computing resources to improve the performance of next generation vehicular networks. We formulate the resource allocation strategy in this framework as a joint optimization problem, where the gains of not only networking but also caching and computing are taken into consideration in the proposed framework. The complexity of the system is very high when we jointly consider these three technologies. Therefore, we propose a novel deep reinforcement learning approach in this paper. Simulation results with different system parameters are presented to show the effectiveness of the proposed scheme.

References

[1]

M. Abadi, A. Agarwal, et al. Nov. 2015. TensorFlow: Large-scale machine learning on heterogeneous systems. arXiv:1603.04467 (Nov. 2015).

[2]

M.Amadeo, C. Campolo, andA.Molinaro. 2016. Information-centric networking for connected vehicles: a survey and future perspectives. IEEE Comm. Magazine 54, 2 (Feb. 2016), 98--104.

Digital Library

[3]

Michael Armbrust, Armando Fox, Rean Griffith, and Anthony D. Joseph. Apr. 2010. A View of Cloud Computing. Commun. ACM 53, 4 (Apr. 2010), 50--58.

Digital Library

[4]

Y. Cai, F. R. Yu, C. Liang, B. Sun, and Q. Yan. Sept. 2016. Software Defined Deviceto- Device (D2D) Communications in VirtualWireless Networks with Imperfect Network State Information (NSI). IEEE Trans. Veh. Tech. 9 (Sept. 2016), 7349-- 7360.

[5]

Laizhong Cui, F. R. Yu, and Qiao Yan. 2016. When Big Data Meets Softwarede ned Networking (SDN): SDN for Big Data and Big Data for SDN. IEEE Network 30, 1 (Jan. 2016), 58--65.

[6]

R. dos Reis Fontes, C. Campolo, C. E. Rothenberg, and A. Molinaro. 2017. From Theory to Experimental Evaluation: ResourceManagement in Software-Defined Vehicular Networks. IEEE Access PP, 99 (2017), 1--1.

[7]

ETSI. 2014. Mobile-Edge Computing - Introductory TechnicalWhite Paper. ETSI White Paper (Sep. 2014).

[8]

A. Silva Fabrício, Celes Clayson, and Azzedine Boukerche. 2015. Filling the Gaps of Vehicular Mobility Traces. In Proc. 18th ACM Int'l Conference on Modeling, Analysis and Simulation ofWireless and Mobile Systems (MSWiM'15). ACM, New York, NY, USA, 47--54. https://doi.org/10.1145/2512921.2516962

[9]

C. Fang, F. R. Yu, T. Huang, J. Liu, and Y. Liu. 2015. A Survey of Green Information-Centric Networking: Research Issues and Challenges. IEEE Comm. Surveys Tutorials 17, 3 (Thirdquarter 2015), 1455--1472.

[10]

Agata Grzybek, Grégoire Danoy, Marcin Seredynski, and Pascal Bouvry. 2013. Evaluation of Dynamic Communities in Large-scale Vehicular Networks. In Proc. Third ACM Int'l Symp. Design and Analysis of Intelligent Vehicular Networks and Applications (DIVANet'13). 93--100. https://doi.org/10.1145/2512921.2516962

Digital Library

[11]

Quansheng Guan, F. R. Yu, Shengming Jiang, and Gang Wei. 2010. Prediction- Based Topology Control and Routing in Cognitive Radio Mobile Ad Hoc Networks. IEEE Trans. Veh. Tech. 59, 9 (Nov. 2010), 4443 --4452.

[12]

J. Guo, B. Song, Y. He, F. Richard Yu, and M. Sookhak. 2017. A Survey on Compressed Sensing in Vehicular Infotainment Systems. IEEE Comm. Surveys and Tutorials (May 2017).

[13]

Y. He, C. Liang, F. Richard Yu, N. Zhao, and H. Yin. 2017. Optimization of Cacheenabled Opportunistic Interference Alignment Wireless Networks: A Big Data Deep Reinforcement Learning Approach. In Proc. IEEE ICC'17. Paris, France.

[14]

Y. He, F. R. Yu, N. Zhao, V. C. M. Leung, and H. Yin. 2017. Software-defined Networks with Mobile Edge Computing and Caching for Smart Cities: A Big Data Deep Reinforcement Learning Approach. IEEE Commun. Mag. 55, 12 (Dec. 2017).

Digital Library

[15]

Y. He, F. R. Yu, N. Zhao, H. Yin, H. Yao, and R. C. Qiu. 2016. Big Data Analytics in Mobile Cellular Networks. IEEE Access 4 (Mar. 2016), 1985--1996.

[16]

X. Hou, Y. Li, M. Chen, D.Wu, D. Jin, and S. Chen. 2016. Vehicular Fog Computing: A Viewpoint of Vehicles as the Infrastructures. IEEE Trans. Veh. Tech. 65, 6 (June 2016), 3860--3873.

[17]

Abboud Khadige and W. Zhuang. 2014. Impact of node mobility on single-hop cluster overlap in vehicular ad hoc networks. In Proc. 17th ACM Int'l Conference on Modeling, Analysis and Simulation ofWireless and Mobile Systems (MSWiM'14). 65--72. https://doi.org/10.1145/2512921.2516962

[18]

D. Kreutz, M.V. Ramos, P.E. Verissimo, C.E. Rothenberg, S. Azodolmolky, and S. Uhlig. 2015. Software-Defined Networking: A Comprehensive Survey. Proc. IEEE 103, 1 (Jan. 2015), 14--76.

[19]

N. Kumar, S. Zeadally, and J. J. P. C. Rodrigues. 2016. Vehicular delay-tolerant networks for smart grid data management using mobile edge computing. IEEE Comm. Magazine 54, 10 (Oct. 2016), 60--66.

Digital Library

[20]

L. Zhang, A. Afanasyev, F. Burke, V. Jacobson, K.C. Claffy, P. Crowley, C. Papadopoulos, L. Wang, and B. Zhang. Jul. 2014. Named Data Networking. ACM SIGCOMM Comput. Commun. Rev. 44, 3 (Jul. 2014), 66--73.

Digital Library

[21]

Z. Li, F. R. Yu, andM. Huang. 2010. A Distributed Consensus-Based Cooperative Spectrum Sensing in Cognitive Radios. IEEE Trans. Veh. Tech. 59, 1 (Jan. 2010), 383--393.

[22]

C. Liang and F. R. Yu. 2015. Wireless Network Virtualization: A Survey, Some Research Issues and Challenges. IEEE Commun. Surveys Tutorials 17, 1 (Firstquarter 2015), 358--380.

[23]

C. Liang and F. R. Yu. 2015. Wireless Virtualization for Next Generation Mobile Cellular Networks. IEEE Wireless Comm. 22, 1 (Feb. 2015), 61--69.

[24]

C. Liang, F. R. Yu, and X. Zhang. 2015. Information-Centric Network Function Virtualization Over 5G Mobile Wireless Networks. IEEE Network 29, 3 (May 2015), 68--74.

[25]

K. Liu, J. K. Y. Ng, V. C. S. Lee, S. H. Son, and I. Stojmenovic. 2016. Cooperative Data Scheduling in Hybrid Vehicular Ad Hoc Networks: VANET as a Software Defined Network. IEEE/ACM Trans. Networking 24, 3 (June 2016), 1759--1773.

Digital Library

[26]

K. Liu, J. K. Y. Ng, J. Wang, V. C. S. Lee, W. Wu, and S. H. Son. 2016. Network-Coding-Assisted Data Dissemination via Cooperative Vehicle-to- Vehicle/-Infrastructure Communications. IEEE Trans. Intell. Transp. Sys. 17, 6 (June 2016), 1509--1520.

[27]

C. Luo, F. R. Yu, H. Ji, and V. C. M. Leung. 2010. Cross-layer Design for TCP Performance Improvement in Cognitive Radio Networks. IEEE Trans. Veh. Tech. 59, 5 (2010), 2485--2495.

[28]

L. Ma, F. Yu, V. C. M. Leung, and T. Randhawa. 2004. A new method to support UMTS/WLAN vertical handover using SCTP. IEEEWireless Commun. 11, 4 (Aug. 2004), 44--51.

Digital Library

[29]

M. N. Mejri and Jalel Ben-Othman. 2014. Entropy as a New Metric for Denial of Service Attack Detection in Vehicular Ad-hoc Networks. In Proc. 17th ACM Int'l Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems (MSWiM'14). 73--79. https://doi.org/10.1145/2512921.2516962

Digital Library

[30]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).

[31]

VolodymyrMnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529--533.

[32]

Hao Yi Ong, Kevin Chavez, and Augustus Hong. 2015. Distributed Deep QLearning. arXiv preprint arXiv:1508.04186 (2015).

[33]

S. Sarkar, S. Chatterjee, and S. Misra. Oct. 2015. Assessment of the Suitability of Fog Computing in the Context of Internet of Things. IEEE Trans. Cloud Computing pp, 99 (Oct. 2015), 1.

[34]

David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484--489.

[35]

M. Sookhak, F. Richard Yu, Y. He, H. Talebian, N. Zhao, M. K. Khan, and N. Kumar. 2017. Fog Vehicular Computing: Augmentation of Fog Computing Using Vehicular Cloud Computing. IEEE Veh. Tech. Mag. (Sept. 2017). on-line.

[36]

Yifei Wei, F. R. Yu, and Mei Song. 2010. Distributed Optimal Relay Selection in Wireless Cooperative NetworksWith Finite-StateMarkov Channels. IEEE Trans. Veh. Tech. 59, 5 (June 2010), 2149 --2158.

[37]

G. Xylomenos, C. N. Ververidis, V. A. Siris, N. Fotiou, C. Tsilopoulos, X. Vasilakos, K. V. Katsaros, and G. C. Polyzos. 2014. A survey of information-centric networking research. IEEE Commun. Surveys Tuts. 16, 2 (Second Quarter 2014), 1024--1049.

[38]

Qiao Yan, F. R. Yu, Qingxiang Gong, and Jianqiang Li. 2016. Software-Defined Networking (SDN) and Distributed Denial of Service (DDoS) Attacks in Cloud Computing Environments: A Survey, Some Research Issues, and Challenges. IEEE Commun. Survey and Tutorials 18, 1 (2016), 602--622.

[39]

F. Yu and V. Krishnamurthy. 2007. Optimal Joint Session Admission Control in Integrated WLAN and CDMA Cellular Networks with Vertical Handoff. IEEE Trans. Mobile Computing 6, 1 (Jan. 2007), 126--139.

[40]

Fei Yu and V. C. M. Leung. Apr. 2001. Mobility-based predictive call admission control and bandwidth reservation in wireless cellular networks. In Proc. IEEE INFOCOM'01. Anchorage, AK.

[41]

F. R. Yu,Minyi Huang, and H. Tang. 2010. Biologically inspired consensus-based spectrum sensing in mobile Ad Hoc networks with cognitive radios. IEEE Network 24, 3 (May 2010), 26 --30.

Digital Library

[42]

H. Zhang,Q. Zhang, and X.Du. 2015. Toward Vehicle-Assisted Cloud Computing for Smartphones. IEEE Trans. Veh. Tech. 64, 12 (Dec. 2015), 5610--5618.

[43]

Q. Zheng, K. Zheng, H. Zhang, and V. C. M. Leung. 2016. Delay-Optimal Virtualized Radio Resource Scheduling in Software-Defined Vehicular Networks via Stochastic Learning. IEEE Trans. Veh. Tech. 65, 10 (Oct. 2016), 7857--7867.

Cited By

Idris MAhmedy ISoon TYahuza MTambuwal AAli U(2024)Cognitive radio and machine learning modalities for enhancing the smart transportation system: A systematic literature reviewICT Express10.1016/j.icte.2024.05.001Online publication date: May-2024
https://doi.org/10.1016/j.icte.2024.05.001
Zhao PLi CYu FFu YCoutinho RBani Younes M(2023)Enhanced 3D Sensor Deployment Method for Cooperative Sensing in Connected and Autonomous VehiclesProceedings of the Int'l ACM Symposium on Design and Analysis of Intelligent Vehicular Networks and Applications10.1145/3616392.3624703(23-30)Online publication date: 30-Oct-2023
https://dl.acm.org/doi/10.1145/3616392.3624703
Afachao KAbu-Mahfouz A(2023)An Examination of Virtualization Technologies for Enabling Intelligent Edge Computing2023 International Conference on Sustainable Computing and Smart Systems (ICSCSS)10.1109/ICSCSS57650.2023.10169357(1612-1617)Online publication date: 14-Jun-2023
https://doi.org/10.1109/ICSCSS57650.2023.10169357
Show More Cited By

Index Terms

Deep Reinforcement Learning (DRL)-based Resource Management in Software-Defined and Virtualized Vehicular Ad Hoc Networks
1. Networks
  1. Network architectures
  2. Network performance evaluation

Recommendations

A Deep Reinforcement Learning-based Trust Management Scheme for Software-defined Vehicular Networks
DIVANet'18: Proceedings of the 8th ACM Symposium on Design and Analysis of Intelligent Vehicular Networks and Applications

Vehicular ad hoc networks (VANETs) have become a promising technology in intelligent transportation systems (ITS) with rising interest of expedient, safe, and high-efficient transportation. VANETs are vulnerable to malicious nodes and result in ...
Software-defined Vehicular Ad Hoc Networks with Trust Management
DIVANet '16: Proceedings of the 6th ACM Symposium on Development and Analysis of Intelligent Vehicular Networks and Applications

With the rising interest of expedient, safe, and high-efficient transportation, vehicular ad hoc networks (VANETs) have turned into a critical technology in smart transportation systems. Because of the high mobility of nodes, VANETs are vulnerable to ...
Node Connectivity in Vehicular Ad Hoc Networks with Structured Mobility
LCN '07: Proceedings of the 32nd IEEE Conference on Local Computer Networks

Vehicular Ad hoc NETworks (VANETs) is a subclass of Mobile Ad hoc NETworks (MANETs). However, automotive ad hoc networks will behave in fundamentally different ways than the predominated models in MANET research. Driver behaviour, mobility constraints ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DIVANet '17: Proceedings of the 6th ACM Symposium on Development and Analysis of Intelligent Vehicular Networks and Applications

November 2017

160 pages

ISBN:9781450351645

DOI:10.1145/3132340

General Chair:
Mirela S.M.A. Notare
Faculdade de Tecnologia em Transporte Aéreo, Brazil
,
Program Chair:
Rodolfo Coutinho
UFMG, Brazil

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSIM: ACM Special Interest Group on Simulation and Modeling

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MSWiM '17

Sponsor:

SIGSIM

MSWiM '17: 20th ACM Int'l Conference on Modelling, Analysis and Simulation of Wireless and Mobile Systems

November 21 - 25, 2017

Florida, Miami, USA

Acceptance Rates

Overall Acceptance Rate 70 of 308 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
843
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)3

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Idris MAhmedy ISoon TYahuza MTambuwal AAli U(2024)Cognitive radio and machine learning modalities for enhancing the smart transportation system: A systematic literature reviewICT Express10.1016/j.icte.2024.05.001Online publication date: May-2024
https://doi.org/10.1016/j.icte.2024.05.001
Zhao PLi CYu FFu YCoutinho RBani Younes M(2023)Enhanced 3D Sensor Deployment Method for Cooperative Sensing in Connected and Autonomous VehiclesProceedings of the Int'l ACM Symposium on Design and Analysis of Intelligent Vehicular Networks and Applications10.1145/3616392.3624703(23-30)Online publication date: 30-Oct-2023
https://dl.acm.org/doi/10.1145/3616392.3624703
Afachao KAbu-Mahfouz A(2023)An Examination of Virtualization Technologies for Enabling Intelligent Edge Computing2023 International Conference on Sustainable Computing and Smart Systems (ICSCSS)10.1109/ICSCSS57650.2023.10169357(1612-1617)Online publication date: 14-Jun-2023
https://doi.org/10.1109/ICSCSS57650.2023.10169357
Zhang DShi WYang R(2023)A Blockchain-Based Distributed Machine Learning (BDML) Approach for Resource Allocation in Vehicular Ad-Hoc NetworksGreen, Pervasive, and Cloud Computing10.1007/978-3-031-26118-3_8(110-121)Online publication date: 1-Feb-2023
https://doi.org/10.1007/978-3-031-26118-3_8
Khani MJamali SSohrabi MSadr MGhaffari A(2023) Resource allocation in 5G cloud‐RAN using deep reinforcement learning algorithms: A review Transactions on Emerging Telecommunications Technologies10.1002/ett.492935:1Online publication date: 26-Dec-2023
https://doi.org/10.1002/ett.4929
Jiang XYu FSong TLeung V(2022)Resource Allocation of Video Streaming Over Vehicular Networks: A Survey, Some Research Issues and ChallengesIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2021.306520923:7(5955-5975)Online publication date: Jul-2022
https://doi.org/10.1109/TITS.2021.3065209
Nayak BHota LKumar ATuruk AChong P(2022)Autonomous Vehicles: Resource Allocation, Security, and Data PrivacyIEEE Transactions on Green Communications and Networking10.1109/TGCN.2021.31108226:1(117-131)Online publication date: Mar-2022
https://doi.org/10.1109/TGCN.2021.3110822
Deng YJiang HCai PWu TZhou PLi BLu HWu JChen XWang K(2022)Resource Provisioning for Mitigating Edge DDoS Attacks in MEC-Enabled SDVNIEEE Internet of Things Journal10.1109/JIOT.2022.31899759:23(24264-24280)Online publication date: 1-Dec-2022
https://doi.org/10.1109/JIOT.2022.3189975
Yuan TDa Rocha Neto WRothenberg CObraczka KBarakat CTurletti T(2022)Machine learning for next‐generation intelligent transportation systemsTransactions on Emerging Telecommunications Technologies10.1002/ett.442733:4Online publication date: 17-Apr-2022
https://dl.acm.org/doi/10.1002/ett.4427
Raw RKumar MSingh N(2021)Software-Defined Vehicular Adhoc NetworkCloud-Based Big Data Analytics in Vehicular Ad-Hoc Networks10.4018/978-1-7998-2764-1.ch007(141-164)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-2764-1.ch007
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten