research-article

Traffic Lights Detection and Recognition Method using Deep Learning with Improved YOLOv5 for Autonomous Vehicle in ROS2

Authors:

Khang Hoang Nguyen,

Hoang Ngoc TranAuthors Info & Claims

ICIIT '23: Proceedings of the 2023 8th International Conference on Intelligent Information Technology

Pages 117 - 122

https://doi.org/10.1145/3591569.3591589

Published: 13 July 2023 Publication History

Abstract

One of the most significant uses of autonomous cars in recent years is the detection of traffic light signals. Deep learning technology, which has a number of benefits including high detection accuracy and quick response to changes, is supporting the development of traffic light recognition under various environmental situations. In this paper, we use two methods to improve the traffic light detection and recognition method. First, we speed up training time by using the K-means clustering algorithm to compress image data. Second, a real time traffic light signal (red, yellow, green) identity based on the You Only Look Once (Yolov5) model is introduced. We utilised a variety of datasets including a freely available Roboflow dataset, a set of data obtained from Gazebo simulator, and a traffic light of CanTho city dataset to train and evaluate the proposed system. Furthermore, our algorithm was validated on a vehicle model in a simulated environment Gazebo of Robot Operating System 2 (ROS2).

References

[1]

W.-C. S. Cheng-Chin, Ming-Che Ho, “Detecting and recognizing traffic lights by genetic approximate ellipse detection and spatial texture layouts,” ICIC 2011, 2011.

[2]

T. H.-P. Tran, C. C. Pham, T. P. Nguyen, T. T. Duong, and J. W. Jeon, “Real-time traffic light detection using color density,” in 2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia). IEEE, pp. 1–4, 2016.

[3]

J. Levinson, J. Askeland, J. Dolson, and S. Thrun, “Traffic light mapping, localization, and state detection for autonomous vehicles,” Int. Conf. on Robotics and Automation (ICRA), pp. 5784–5791, 2011.

[4]

Fitri Utaminingrum, Renaldi P. Prasetya, and Rizdania, "Combining Multiple Feature for Robust Traffic Sign Detection," Journal of Image and Graphics, Vol. 8, No. 2, pp. 53-58, June 2020.

[5]

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

[6]

Ryo Hasegawa, Yutaro Iwamoto, and Yen-Wei Chen, "Robust Japanese Road Sign Detection and Recognition in Complex Scenes Using Convolutional Neural Networks," Journal of Image and Graphics, Vol. 8, No. 3, pp. 59-66, September 2020.

[7]

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” in European conference on computer vision. Springer, 2016.

[8]

Z. Gan, L. Wenju, C. Wanghui and S. Pan, "Traffic sign recognition based on improved YOLOv4," 2021 6th International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), 2021.

[9]

F.-A. Redmon, Joseph, “Yolov3: An incremental improvement,” Tech. Rep., 2018.

[10]

Cai Z.-X., Gu M.-Q. Accurate Recognition and State Estimation of Traffic Lights in Urban Environment. Control. Decis. 2014.

[11]

Philipsen M.P., Jensen M.B., Møgelmose A., Moeslund T.B., Trivedi M.M. Traffic Light Detection: A Learning Algorithm and Evaluations on Challenging Dataset; Proceedings of the 2015 IEEE 18th International Conference on Intelligent Transportation Systems; Gran Canaria, Spain. 15–18 September 2015; Piscataway, NJ, USA: IEEE; 2015.

[12]

Du X.-P., Li Y., Guo Y., Xiong H. Vision-Based Traffic Light Detection for Intelligent Vehicles; Proceedings of the 2017 4th International Conference on Information Science and Control Engineering; Changsha, China. 21–23 July 2017; Piscataway, NJ, USA: IEEE; 2017.

[13]

Behrendt K., Novak L., Botros R. A Deep Learning Approach to Traffic Lights: Detection, Tracking, and Classification; Proceedings of the 2017 IEEE International Conference on Robotics and Automation; Singapore. 29 May–3 June 2017; Piscataway, NJ, USA: IEEE; 2017.

[14]

Pan W.-G., Chen Y.-H., Liu B., Shi H.-L. Detection and Recognition of Traffic Lights Based on Faster R-CNN. Transducer Microsyst. Technol. 2019;38:147–149.

[15]

Pan W.-G., Chen Y.-H., Liu B., Shi H.-L. Detection and Recognition of Traffic Lights Based on Faster R-CNN. Transducer Microsyst. Technol. 2019.

[16]

Morten B.J., Kamal N., Thomas B. Evaluating State-of-the-art Object Detector on Challenging Traffic Light Data; Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition; Honolulu, HI, USA. 21–26 July 2017.

[17]

Muller J., Dietmayer K. Detecting Traffic Lights by Single Shot Detection; Proceedings of the 21st IEEE International Conference on Intelligent Transportation Systems; Maui, HI, USA. 4–7 November 2018; Piscataway, NJ, USA: IEEE; 2018.

Digital Library

[18]

Possatti L.C., Guidolini R., Cardoso V.B., Berriel R.F., Paixão T.M., Badue C., de Souza A.F., Oliveira-Santos T. Traffic Light Recognition Using Deep Learning and Prior Maps for Autonomous Cars; Proceedings of the 2019 International Joint Conference on Neural Networks; Budapest, Hungary. 14–19 July 2019.

[19]

Du L., Chen W., Fu S., Kong H., Li C., Pei Z. Real-time Detection of Vehicle and Traffic Light for Intelligent and Connected Vehicles Based on YOLOv3 Network; Proceedings of the 2019 5th International Conference on Transportation Information and Safety; Liverpool, UK. 14–17 July 2019.

[20]

Tran, T. H.-P., & Jeon, J. W. (2020). Accurate Real-Time Traffic Light Detection Using YOLOv4. 2020 IEEE International Conference on Consumer Electronics - Asia (ICCE-Asia).

[21]

S. Wang, "K-Means Clustering With Incomplete Data," in IEEE Access, vol. 7, pp. 69162-69171, 2019.

[22]

Hoang Tran Ngoc and Luyl-Da Quach, “Adaptive Lane Keeping Assist for an Autonomous Vehicle based on Steering Fuzzy-PID Control in ROS” International Journal of Advanced Computer Science and Applications(IJACSA), 13(10), 2022.

Cited By

Tran HNguyen NLe NNguyen NLe TNguyen V(2025)Enhancing semantic scene segmentation for indoor autonomous systems using advanced attention-supported improved UNetSignal, Image and Video Processing10.1007/s11760-024-03779-w19:2Online publication date: 6-Jan-2025
https://doi.org/10.1007/s11760-024-03779-w
Hong PKhanh HVinh NTrung NQuoc ANgoc H(2024)Deep Learning-Based Lane-Keeping Assist System for Self-Driving Cars Using Transfer Learning and Fine TuningJournal of Advances in Information Technology10.12720/jait.15.3.322-32915:3(322-329)Online publication date: 2024
https://doi.org/10.12720/jait.15.3.322-329
Ngoc HHong PQuoc AQuach L(2024)Steering Angle Prediction for Autonomous Vehicles Using Deep Transfer LearningJournal of Advances in Information Technology10.12720/jait.15.1.138-14615:1(138-146)Online publication date: 2024
https://doi.org/10.12720/jait.15.1.138-146
Show More Cited By

Index Terms

Traffic Lights Detection and Recognition Method using Deep Learning with Improved YOLOv5 for Autonomous Vehicle in ROS2
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Robotics
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Deriving Pattern in Driver's Observability in Road Turns & Traffic Lights: Eye-Tracking based Analysis
C3VP '18: Proceedings of the 1st International Workshop on Communication and Computing in Connected Vehicles and Platooning

As one move towards driverless cars, there will always be a big worry of how autonomous cars would behave in the presence of vehicles driven by humans. In the co-existence model, it is essential for autonomous systems to 'understand' the behavior and ...
Appearance-based Brake-Lights recognition using deep learning and vehicle detection
2016 IEEE Intelligent Vehicles Symposium (IV)
Vehicle following is one of the fundamental functions of an autonomous driving system. Detection and recognition of tail light signal is important to prevent an autonomous vehicle from rear-end collisions or accidents. Although sensors like acoustic sonar ...
Architecture of autonomous vehicle simulation and control framework
AIR '15: Proceedings of the 2015 Conference on Advances In Robotics

With the current advances in Autonomous Navigation Systems for Self Driving Vehicles, there is a need for specialized and focused software architecture for ease in simulating the vehicle behavior and associated modules such as perception, drive by wire ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICIIT '23: Proceedings of the 2023 8th International Conference on Intelligent Information Technology

February 2023

310 pages

ISBN:9781450399616

DOI:10.1145/3591569

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICIIT 2023

ICIIT 2023: 2023 8th International Conference on Intelligent Information Technology

February 24 - 26, 2023

Da Nang, Vietnam

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
120
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tran HNguyen NLe NNguyen NLe TNguyen V(2025)Enhancing semantic scene segmentation for indoor autonomous systems using advanced attention-supported improved UNetSignal, Image and Video Processing10.1007/s11760-024-03779-w19:2Online publication date: 6-Jan-2025
https://doi.org/10.1007/s11760-024-03779-w
Hong PKhanh HVinh NTrung NQuoc ANgoc H(2024)Deep Learning-Based Lane-Keeping Assist System for Self-Driving Cars Using Transfer Learning and Fine TuningJournal of Advances in Information Technology10.12720/jait.15.3.322-32915:3(322-329)Online publication date: 2024
https://doi.org/10.12720/jait.15.3.322-329
Ngoc HHong PQuoc AQuach L(2024)Steering Angle Prediction for Autonomous Vehicles Using Deep Transfer LearningJournal of Advances in Information Technology10.12720/jait.15.1.138-14615:1(138-146)Online publication date: 2024
https://doi.org/10.12720/jait.15.1.138-146
Amin RAmran Hossain MSchmid LWiese VObermaisser R(2024)Power Efficient Real-Time Traffic Signal Classification for Autonomous Driving Using FPGAs2024 6th International Conference on Communications, Signal Processing, and their Applications (ICCSPA)10.1109/ICCSPA61559.2024.10794303(1-5)Online publication date: 8-Jul-2024
https://doi.org/10.1109/ICCSPA61559.2024.10794303
Amin RHasan MWiese VObermaisser R(2024)FPGA-Based Real-Time Object Detection and Classification System Using YOLO for Edge ComputingIEEE Access10.1109/ACCESS.2024.340462312(73268-73278)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3404623
Tran HLe TNguyen NNguyen NNguyen A(2024)Fine-tuned depth-augmented U-Net for enhanced semantic segmentation in indoor autonomous vision systemsJournal of Real-Time Image Processing10.1007/s11554-024-01578-722:1Online publication date: 6-Dec-2024
https://doi.org/10.1007/s11554-024-01578-7
Le TNguyen NNguyen NLe NNguyen NTran H(2024)Semantic scene segmentation for indoor autonomous vision systems: leveraging an enhanced and efficient U-NET architectureMultimedia Tools and Applications10.1007/s11042-024-19302-9Online publication date: 9-May-2024
https://doi.org/10.1007/s11042-024-19302-9
Suasnavas DPachacama DOñate W(2024)YOLO-V7 and YOLO-V8 Benchmark for Firearm Detection and Deep Learning Model RetrainingProceedings of the International Conference on Computer Science, Electronics and Industrial Engineering (CSEI 2023)10.1007/978-3-031-69228-4_11(167-181)Online publication date: 23-Dec-2024
https://doi.org/10.1007/978-3-031-69228-4_11
Nguyen DLuong NLe TNguyen DNgoc H(2023)Robust Traffic Sign Detection and Classification Through the Integration of YOLO and Deep Learning NetworksIntelligence of Things: Technologies and Applications10.1007/978-3-031-46573-4_29(310-321)Online publication date: 20-Oct-2023
https://doi.org/10.1007/978-3-031-46573-4_29

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten