A robust, real-time and calibration-free lane departure warning system
Introduction
Population growth and the rapid increase in the number of vehicles have led to more traffic accidents every year, which become a serious problem. According to the Association for Safe International Road Travel (ASIRT) organization, about 1.3 million people die and about 20–50 million are injured or disabled annually due to traffic accidents [1]. It has been estimated that deaths will reach 2.4 million people annually by 2030 unless immediate action is taken.
There are number of reasons causing the traffic accidents, ranging from the driver's behavior, mechanical failures, environmental conditions to road design. Unintended lane departures, which are caused by drivers, occupy the fourth rank among these reasons. According to the National Highway Traffic Safety Administration (NHTSA), 37% of deaths in traffic accidents in USA are caused by lane departures [2]. That prompts the development of many Driver Assistance Systems (DASs) which provide the driver with essential information about the surroundings and prevent driver from making unintended mistakes [3].
Lane Departure Warning System (LDWS) is a mechanism designed to warn the driver when the vehicle begins to move out of its lane unless a turn signal is on in that direction. This system can be implemented using two different technologies: machine vision (MV) or GPS technology. GPS uses high-resolution map databases with its highly accurate positioning ability. On the other hand, MV uses single or multiple cameras with image processing algorithms to detect lanes on the road. Unlike GPS, MV uses the existing infrastructure and it can be adapted easily with road design changes. Therefore, most of the proposed techniques in the literature uses MV technology to implement the LDWS [3]. These implementations are mainly based on the inverse perspective mapping (IPM) to ease lane detection by getting the bird's eye view of the road in front of the car. IPM has a high computational time, which affects the real-time performance of the system. Also, it is parameter-based and requires calibration of the camera for every different type of car which makes the system unportable [4]. A new algorithm based on MV to implement a non-IPM-based LDWS with high efficiency and real-time performance is proposed in this paper, as shown in Fig. 1. It is a real-time and calibration-free LDWS (RTCFLDWS) algorithm.
The remaining of this paper is organized as follows. In Section 2, a brief overview of the related previous work is presented. In Section 3, the proposed algorithm is introduced and described in details. In Section 4, experimental results for the proposed algorithm are provided. Some conclusions are portrayed in Section 5.
Section snippets
Related work
In this section, the related work in LDWS is reviewed. In the data acquisition stage of the LDWS which is based on MV technology, there are two main approaches of using the camera: the single-camera approach [5], [6], and the multi-camera approach [7]. In the single camera approach, one camera is fixed behind the windshield mirror. This approach is the most widely used in the industry because of its low cost. In the multi-camera approach [7], two or more cameras are used in front and in rear of
The proposed algorithm
The proposed algorithm extracts the ROI to reduce the outlier lines in the image (trees boundaries, roadsides, etc.). Then, image smoothing stage is carried out using Gaussian pyramid as it smoothes the image without harming the needed edges in it. Edge Drawing Lines (EDLines) algorithm is then used, which is real-time and very powerful edge and line segment detection technique with high performance and high false detection control. In the following stage, basic machine learning (ML) concepts
Experimental results
According to ISO 17361:2017 standard, the environment conditions to test LDWS are flat and dry asphalt, lane markings being directly visible by the driver and horizontal visibility range being greater than 1 km. The stated conditions in the standard describe an ideal environment. Real life is not ideal. To guarantee that the driver safety is accomplished by RTCFLDWS, the system is tested by a various challenging weather and illumination conditions: clear, cloudy, rainy, day, sunset and night.
Conclusions
In this paper, a new reliable and robust algorithm to implement LDWS is introduced. The RTCFLDWS algorithm is real-time and scalable. It reduces the input image using region of interest extraction. Edge detection and line segmentation method EDLines is applied. It is fast and accurate with false detection control. The filtering and clustering block uses basic machine learning to select only the lines related to lane boundaries from the detected lines. The lane boundaries are tracked while they
Declaration of Competing Interest
None.
Islam Gamal received his B.Sc. in Electronics and Communications Engineering, with honor, from Cairo University in 2018. Currently, he is an Embedded Software Engineer at Mentor Graphics Company, a Siemens business, where he manages projects to both internal and external customers, including development/integration of AUTOSAR basic software components and tests for automotive. Islam is an instructor at Mentor Embedded Academy of Excellence. He has been a Certified LabVIEW Associate Developer
References (25)
- et al.
Ego-Lane Analysis System (ELAS): dataset and algorithms
Image Vis. Comput.
(2017) - et al.
Multiscale contextual spatial modelling with the Gaussian scale space
Geoderma
(2018) - ASIRT Organization, [Online]....
Road departure crash warning system field operational test: methodology and results
Technical report
(2006)- et al.
Ultra-low complexity block-based lane detection and departure warning system
IEEE Trans. Circuits Syst. Video Technol.
(2019) - et al.
Lane departure warning tracking system based on score mechanism
- et al.
Real-Time lane departure warning system on a lower resource platform
- et al.
A novel illumination-invariant lane detection system
- et al.
A new multi-camera approach for lane departure warning
- et al.
Lane departure detection using image processing techniques
Lane line detection in real time based on morphological operations for driver assistance system
A novel linelet-based representation for line segment detection
IEEE Trans. Pattern Anal. Mach. Intell.
Cited by (8)
Lane detection under artificial colored light in tunnels and on highways: an IoT-based framework for smart city infrastructure
2023, Complex and Intelligent SystemsAC-UNet: lane line detection based on U-Net network fusion attention mechanism and cross convolution
2023, Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile EngineeringObject Detection of Autonomous Vehicles under Adverse Weather Conditions
2022, 2022 International Conference on Data Science, Agents and Artificial Intelligence, ICDSAAI 2022Satisfaction of Modeling Requirements for Intelligent Navigation Systems: Risk Management Context
2022, Lecture Notes in Electrical EngineeringA survey of safety warnings under connected vehicle environments
2021, IEEE Transactions on Intelligent Transportation Systems
Islam Gamal received his B.Sc. in Electronics and Communications Engineering, with honor, from Cairo University in 2018. Currently, he is an Embedded Software Engineer at Mentor Graphics Company, a Siemens business, where he manages projects to both internal and external customers, including development/integration of AUTOSAR basic software components and tests for automotive. Islam is an instructor at Mentor Embedded Academy of Excellence. He has been a Certified LabVIEW Associate Developer since 2015, a Certified LabVIEW Developer since 2017 and a LabVIEW Student Ambassador (LSA) who trains students for LabVIEW development for four years. His current research interests are in AUTOSAR-based ECUs and Calibration and Measurement Protocol and Tools.
Awab M. W. Al-Habal as born in 1995. He received the B.S. degree with honor from the Electronics and Electrical Communications Engineering Department at Cairo University in 2018. He is currently a Research Assistant at the Electronics and Electrical Communications Engineering Department at Cairo University, Egypt. His research interests include embedded systems, Internet of Things, Machine learning and Data science.
Keroles karam Khalil was born in Cairo, Egypt,in 1991. He received the B.S. degree (Cumulative grade : very good) in, electronics and communications engineering Ain Shams University, Cairo, in 2013, he was an Embedded System Engineer with the Mentor Embedded System Division, MentorGraphics Corporation, Cairo. Linkedin Profile https://www.linkedin.com/in/keroles-karam-2a86057b.
Magdy A. El-Moursy was born in Cairo, Egypt in 1974. He received the B.S. degree in electronics and communications engineering (with honors) and the Master's degree in computer networks from Cairo University, Cairo, Egypt, in 1996 and 2000, respectively, and the Master's and the Ph.D. degrees in electrical engineering in the area of high-performance VLSI/IC design from University of Rochester, Rochester, NY, USA, in 2002 and 2004, respectively.
In summer of 2003, he was with STMicroelectronics, Advanced System Technology, San-Diego, CA, USA. Between September 2004 and September 2006 he was a Senior Design Engineer at Portland Technology Development, Intel Corporation, Hillsboro, OR, USA. During September 2006 and February 2008 he was assistant professor in the Information Engineering and Technology Department of the German University in Cairo (GUC), Cairo, Egypt. Between February 2008 and October 2010 he was Technical Lead in the Mentor Hardware Emulation Division, Mentor Graphics Corporation, Cairo, Egypt.
Dr. El-Moursy is currently Senior Engineering Manager in Integrated Circuits Verification and Solutions Division, Mentor, A Siemens Business and Associate Professor in the Microelectronics Department, Electronics Research Institute, Cairo, Egypt. He is Associate Editor in the Editorial Board of Elsevier Microelectronics Journal, International Journal of Circuits and Architecture Design and Journal of Circuits, Systems, and Computers and Technical Program Committee of many IEEE Conferences such as ISCAS, ICM, ICAINA, PacRim CCCSP, ISESD, SIECPC, and IDT.
His research interest is in Networks-on-Chip/System-on-Chip, interconnect design and related circuit level issues in high performance VLSI circuits, clock distribution network design, digital ASIC circuit design, VLSI/SoC/NoC design and validation/verification, circuit verification and testing, Embedded Systems and low power design. He is the author of over 80 papers, five book chapters, and four books in the fields of high speed and low power CMOS design techniques, NoC/SoC and Embedded Systems.
Ahmed Khattab (S'05, M'12, SM'17) is an Associate Professor in the Electronics and Electrical Communications Engineering Department at Cairo University. He is also an adjunct Associate Professor in the American University in Cairo (AUC). He received his Ph.D. in Computer Engineering from the Center for Advanced Computer Studies (CACS) at the University of Louisiana at Lafayette, USA, in 2011. He received a Master of Electrical Engineering Degree from Rice University, USA, in 2009. He also received M.Sc. and B.Sc. (Honors) degrees in Electrical Engineering from Cairo University, Cairo, Egypt, in 2004 and 2002, respectively. He is an IEEE senior member. Dr. Khattab has authored/co-authored 3 books, over 70 journal and conference publications, and a US patent. He serves as a reviewer in many IEEE transactions, journals and conferences, and is a member of the technical committee of several prestigious conferences such as IEEE Globecom, IEEE ICC, IEEE ICCCN, and IEEE WF-IoT. Dr. Khattab was awarded Egypt State Award in 2017. His current research interests are in the broad areas of wireless networking including, but not limited to, the Internet of Things (IoT), wireless sensor networks, vehicular networks, cognitive radio networks, security and machine learning.