Deep reinforcement learning approach for autonomous vehicle systems for maintaining security and safety using LSTM-GAN
Introduction
5G communication provides an inherent support for vehicle-to-everything (V2X) communication. It will bring new capabilities for autonomous vehicles as well as for modern Intelligent Transportation System (ITS) [1], [2]. The feature like very high throughput (i.e. up to multi-Gpbs) with additional uniformity incorporating the wider bandwidths equipped with advance antenna systems would provide an inherent support for various applications especially for autonomous driving. 5G networks can provide end-to-end latency over a quicker much malleable frame structure; with added functionalities like new up-link RSMA non-orthogonal access would mean that it would meet the lower latency criteria required for efficient autonomous driving more easily and efficiently than any of the previous available technologies [3], [4], [5].
For a very realistic autonomous driving scenario any autonomous vehicle should be able to control and process a sizable volume of information of ITS obtained through huge numbers of sensors and 5G-links. Consistency and reliability of such information plays an important role for maintaining safety and security of autonomous vehicle especially this information may be critical to decrease the chances of road accidents and improve the smooth flow of traffic on ITS roads [6], [7]. But this over dependence for autonomous vehicles makes them extremely vulnerable to various security and safety concerns especially in terms of cyber-physical attacks. An adversary might interrupt the autonomous vehicle's data and impact its consistency and reliability by infusing the incorrect information; which can lead to accidents or can have an adverse impact on the flow of traffic [8], [9]. These types of traffic flow interruptions may also influence the interdependent structure connected to ITSs for example 5G-Communication systems or power grids etc. [10], [11].
In recent past various security mechanisms have been suggested for tackling the security concerns present in autonomous vehicles for maintaining the optimum safe intra-vehicle distance [12], [13], [14], [15], [16]. In [12] the key susceptibilities of a vehicle's controller have been presented and several intrusion detection algorithms have been suggested to overcome these susceptibilities. [13] presented the concept of long-range wireless attacks on autonomous vehicle in which existing security mechanism can be interrupted by controller area network which have impact on optimum safe intra-vehicle distance. [14] discussed the security challenges related to the plug-in electric vehicles which have impact on the power system. A detailed study on the security threats and safety systems in the vehicular networks has been introduced in [15]. [16] presents a deep reinforcement learning based solution for maintaining optimum safe intra-vehicle distance.
Moreover, the vehicular network security issues and their related solutions have been also presented in [17], [18], [19], [20], [21]. [17] lists the security and safety susceptibilities present in the cooperative vehicular networks and try to analyze and optimize the computational cost for beacon signal encryption scheme through using the concept of short-term authentication process. In [18] the authors focused on data infusion attacks and presented the solution to overcome these security issues through the concept of multi-source filters. [19] suggested the idea of merging angle of arrival and Doppler effect to improve security aspect for beacon signals. Furthermore, in [20] solution to spoofing and denial of service attacks through use of collaborative control mechanism have been proposed. A detailed review on vehicular networks and their interaction with modern ITS have been exhibited in [21].
The key problem with these above-mentioned security solutions is that they do not consider the interdependence present between the cyber domain and physical domain of autonomous vehicle while formulating the security and safety resolution. Additionally, there is no proper work in modeling the adversary actions and objectives. In-fact by taking into account the cyber-physical inter-dependencies of adversary's acts and objectives will enable us to provide the better security and safety mechanism. The only work that focus on these inter-dependencies is [16] but it lacks in terms of performance and effectiveness. Thus, an effective safety and security mechanism for autonomous vehicles on modern ITS should be flexible and robust to these attacks on intra-vehicle sensors and for vehicle to vehicle communication. Besides this, all the current works on ITS security usually believe a stable state for the adversary's actions, but in-fact in reality this might not be the case as the adversary may alter its approach adaptively to cause a severe effect of its acts.
The major contribution of this paper is to present a new Deep reinforcement learning algorithm (NDRL) structure that will provide secured autonomous vehicle (AVh) control dynamic system. It is based upon car following model [22], [23]. In this approach focus is on the control system of autonomous vehicle that uses leading autonomous vehicle to adjust its speed and maintain safe-optimal distance. This type of infrastructure requires considering two very important key features:
- a.
Autonomous vehicle's Sensors Data.
- b.
Autonomous vehicle's Beaconing Signal.
In this work we will consider four important intra-vehicle resources that can point towards an effective safe-optimal distance i.e.
- 1.
Intra-vehicle Camera Sensor:
Optical Information from the camera can be used for the measurement of the speed of moving objects around the autonomous vehicle. Mainly there are two approaches being used for using camera as speed estimator i.e. Inter-frame flow measurement (Optical-method) and Intra-frame measurement (motion-blur-method). Moreover, Camera sensor is also being used for other type of applications as well like number plate recognition system. In this work we have considered the Camera senor using the properties of [24].
- 2.
Intra-vehicle Radar Sensor:
Conventionally speed measurement systems in many vehicles uses RADAR technology. Although RADAR method of estimation of speed is more expensive than Camera sensor but it is more traditional and older method with higher accuracy. RADAR uses radio waves which are repeatedly sent and their returns are measured to determine the distance and velocity of next autonomous vehicle. This concept is known as Doppler effect.
- 3.
Smart Roadside Units (sRSUs):
Smart Roadside units are now being considered for estimating speed of autonomous vehicle. As sRSUs in 5G-V2X have real time connectivity with the autonomous vehicles on the road. And these autonomous vehicles will keep updating their speed and other road condition reports to sRSUs. Thus, sRSUs can be very helpful in maintaining a safe optimal distance between the two autonomous vehicles as it will update the following vehicle about the speed of its leading vehicle.
- 4.
Inter-vehicle Beaconing:
Another important technology that can be used for relying key information (speed, distance etc.) between the two autonomous vehicles is beacon signal between the vehicles i.e. V2V communication for this purpose the Dedicated Short Range Communication (DSRC) (IEEE 802.11p)-based wireless communication technology is used that enables highly secure, high-speed direct communication between the autonomous vehicles. Thus, key information like speed and distance between the two autonomous vehicles can be exchanged.
Therefore, the NDRL algorithm for the adversary aims to mislead the autonomous vehicle and introduce deviation between the autonomous vehicles i.e. impacting optimal safe distance spacing while autonomous vehicle tries to minimize this deviation.
Section snippets
System model
First of all we introduce the concept of smart Intelligent Transportation System (ITS) comprising of smart roads which includes autonomous vehicles (), Smart Roadside Units (sRSUs) using 5G communication links etc. Every is considered to be equipped with a camera that can be used to take images from its surroundings, smart radar scanner for calculating the distance between the objects present around and transceiver for transmitting important acceleration speed position (AVP)
Performance evaluation
In this section we will evaluate the proposed work and compare its performance with respect to the previous suggested works. The proposed algorithm is simulated in MATLAB i.e. All the simulation on the vehicle dynamics, communication graph, and the sensor attack on vehicles are implemented in MATLAB.
Conclusions
In this paper, we have presented a new adversarial Deep Reinforcement Learning method (NDRL) that uses Long-Short-Term-Memory (LSTM)-Generative Adversarial Network (GAN) model to provide safety and security mechanism necessary for autonomous vehicles on smart ITS roads equipped with 5G communication links. This method is able to provide effective measure for data infusion attacks that are aim at disrupting the flow of traffic or cause autonomous vehicles to collide leading to accident. To
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (38)
- et al.
A review on safety failures, security attacks, and available countermeasures for autonomous vehicles
Ad Hoc Netw.
(2019) - et al.
A collaborative approach for improving the security of vehicular scenarios: the case of platooning
Comput. Commun.
(2018) - et al.
Car-following: a historical review
Transp. Res., Part F Traffic Psychol. Behav.
(1999) - et al.
The Fréchet distance between multivariate normal distributions
J. Multivar. Anal.
(1982) - et al.
Artificial intelligence defined 5g radio access networks
IEEE Commun. Mag.
(2019) Basics of 5g
- et al.
Millimeter-wave propagation modeling and measurements for 5g mobile networks
IEEE Wirel. Commun.
(2019) - et al.
Distributed edge computing to assist ultra-low-latency vanet applications
Future Internet
(2019) - et al.
Intelligent vehicle network routing with adaptive 3d beam alignment for mmwave 5g-based v2x communications
IEEE Trans. Intell. Transp. Syst.
(2020) - et al.
Security vulnerabilities of connected vehicle streams and their impact on cooperative driving
IEEE Commun. Mag.
(2015)
Key Security Challenges for Cloud-Assisted Connected and Autonomous Vehicles
Secure vehicular communication systems: implementation, performance, and research challenges
IEEE Commun. Mag.
A systems and control perspective of cps security
Annu. Rev. Control
A colonel blotto game for interdependence-aware cyber-physical systems security in smart cities
Security aspects of the in-vehicle network in the connected car
A practical wireless attack on the connected car and security protocol for in-vehicle can
IEEE Trans. Intell. Transp. Syst.
Security concerns of a plug-in vehicle
Survey on security threats and protection mechanisms in embedded automotive networks
Robust deep reinforcement learning for security and safety in autonomous vehicle systems
Cited by (48)
Enhancing road safety through advanced predictive analytics in V2X communication networks
2024, Computers and Electrical EngineeringHesitant convolutional neural networks and intelligent drive algorithm fused subjective guidance
2023, Applied Soft ComputingArtificial Intelligence techniques to mitigate cyber-attacks within vehicular networks: Survey
2022, Computers and Electrical EngineeringCitation Excerpt :Alongside, they used a tilt license plate correction algorithm to adapt the practical scene to improve the efficiency of the designed model. For network attack detection and prevention, authors in [69] proposed a DRL approach based on LSTM and generative adversarial network (GAN) to increase the security and the safety of AVs equipped with 5G communication. The model tries to detect attackers attempting to inject erroneous data into the sensor readings of the autonomous vehicle to disrupt the safe and optimal distance between autonomous vehicles.
Cyber-security and reinforcement learning — A brief survey
2022, Engineering Applications of Artificial IntelligenceCitation Excerpt :Fig. 9 summarizes the common topics for existing literature under the IDS, IPS, IoT, and IAM domains. Many topics like computation and data offloading (Huang et al., 2020), anti-jamming (Aref et al., 2017), interference avoidance (Suwannalai and Polprasert, 2020), navigation (Rasheed et al., 2020), and authentication (Xiao et al., 2021) also fall under the category of networks. An important issue that needs to be addressed in these applications is the high generalization ability for the high-variance and dynamic network traffic conditions commonly encountered in all networks.
A journey towards fully autonomous driving - fueled by a smart communication system
2022, Vehicular Communications