Skip to main content
Springer Nature Link
Log in

An improved lane departure warning algorithm based on fusion of F-Kalman filter and F-TLC

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In order to reduce the lane departure accident caused by driver’s negligence, LDWS (lane departure warning system) has become increasingly popular and important. However, most researches proposed mainly focus on how to detect lane markings. In this paper, we propose an improved lane departure warning algorithm based on fusion of F-Kalman filter (kalman filter based on fuzzy logic) and F-TLC (time to lane crossing based on fuzzy logic). First of all, least square method is used to calculate the distance between vehicle and lane markings. Then the estimation of vehicle states in the future is generated by means of the traditional kalman filter. To better work for lateral offset estimation, a fuzzy model is adopted to change the size of covariance matrices, which is used to adjust the traditional kalman filter in time. Finally, we further put forward to utilize F-TLC to generate multi-grade alarm. Extensive experiments are conducted on different conditions. Experimental results indicate that our warning method works efficiently. The average time consumed for system in each frame is 20.0216 ms. The proposed method possesses good robustness, and can be widely us\ed in LDWS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Albousefi AA, Ying H, Filev D, Syed F, Prakah-Asante KO, Tseng F, Yang H-H (2017) A two-stage-training support vector machine approach to predicting unintentional vehicle lane departure. J Intell Transp Syst 21(1):41–51. https://doi.org/10.1080/15472450.2016.1196141

    Article  Google Scholar 

  2. Ali AA, Hussein HA (2016) Distance estimation and vehicle position detection based on monocular camera. In: 2016 Al-Sadeq International Conference on Multidisciplinary in IT and Communication Science and Applications (AIC-MITCSA). IEEE, pp 1–4. https://doi.org/10.1109/AIC-MITCSA.2016.7759904

  3. Ambarak JM, Ying H, Syed F, Filev D (2017) A neural network for predicting unintentional lane departures. In: 2017 IEEE International Conference on Industrial Technology (ICIT). IEEE, pp 492–497. https://doi.org/10.1109/ICIT.2017.7913281

  4. Bubeníková E, Franeková M, Holečko P (2016) Conceptual design of driving lane-crossing alarm threshold in C-ITS applications and its implementation. In: 2016 Cybernetics & Informatics (K&I). IEEE, pp 1–6. https://doi.org/10.1109/CYBERI.2016.7438598

  5. Cai Z, Gu Z, Yu ZL, Liu H, Zhang K (2016) A real-time visual object tracking system based on Kalman filter and MB-LBP feature matching. Multimed Tools Appl 75(4):2393–2409. https://doi.org/10.1007/s11042-014-2411-6

    Article  Google Scholar 

  6. China NBoSo (2017) China Statistical Yearbook. China Statistics Press, Beijing

    Google Scholar 

  7. Cicchino JB, Zuby DS (2017) Prevalence of driver physical factors leading to unintentional lane departure crashes. Traffic Inj Prev 18(5):481–487. https://doi.org/10.1080/15389588.2016.1247446

    Article  Google Scholar 

  8. Dahmani H, Chadli M, Rabhi A, El Hajjaji A (2015) Vehicle dynamics and road curvature estimation for lane departure warning system using robust fuzzy observers: experimental validation. Veh Syst Dyn 53(8):1135–1149. https://doi.org/10.1080/00423114.2015.1026609

    Article  Google Scholar 

  9. Ding Y, Xu Z, Zhang YB, Sun K (2017) Fast lane detection based on bird's eye view and improved random sample consensus algorithm. Multimed Tools Appl 76(21):22979–22998. https://doi.org/10.1007/s11042-016-4184-6

    Article  Google Scholar 

  10. Dorj B, Lee DJ (2016) A Precise Lane Detection Algorithm Based on Top View Image Transformation and Least-Square Approaches. Journal of Sensors 2016:1–13. https://doi.org/10.1155/2016/4058093

    Article  Google Scholar 

  11. Gaikwad V, Lokhande S (2015) Lane departure identification for advanced driver assistance. IEEE Trans Intell Transp Syst 16(2):910–918. https://doi.org/10.1109/TITS.2014.2347400

    Article  Google Scholar 

  12. Glaser S, Mammar S, Sentouh C (2010) Integrated driver–vehicle–infrastructure road departure warning unit. IEEE Trans Veh Technol 59(6):2757–2771. https://doi.org/10.1109/TVT.2010.2049670

    Article  Google Scholar 

  13. Godthelp H, Milgram P, Blaauw GJ (1984) The development of a time-related measure to describe driving strategy. Hum Factors 26(3):257–268. https://doi.org/10.1177/001872088402600302

    Article  Google Scholar 

  14. Hsiao P-Y, Yeh C-W, Huang S-S, Fu L-C (2009) A portable vision-based real-time lane departure warning system: day and night. IEEE Trans Veh Technol 58(4):2089–2094. https://doi.org/10.1109/TVT.2008.2006618

    Article  Google Scholar 

  15. Ibarra-Bonilla MN, Escamilla-Ambrosio PJ, Ramirez-Cortes JM (2015) Attitude estimation using a Neuro-Fuzzy tuning based adaptive Kalman filter. J Intell Fuzzy Syst 29(2):479–488. https://doi.org/10.3233/ifs-141183

    Article  Google Scholar 

  16. ISO/TC204 (2017) Intelligent transport systems - Lane departure warning systems - Performance requirements and test procedures. ISO 17361:2017

  17. Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng 82(1):35–45. https://doi.org/10.1115/1.3662552

    Article  Google Scholar 

  18. Kobayashi K, Cheok KC, Watanabe K (1995) Estimation of absolute vehicle speed using fuzzy logic rule-based Kalman filter. In: Proceedings of the 1995 American Control Conference. IEEE, pp 3086–3090. https://doi.org/10.1109/ACC.1995.532084

  19. Kortli Y, Marzougui M, Atri M (2016) Efficient implementation of a real-time lane departure warning system. In: 2016 International Image Processing, Applications and Systems (IPAS). IEEE, pp 1–6. https://doi.org/10.1109/IPAS.2016.7880072

  20. Lan M, Rofouei M, Soatto S, Sarrafzadeh M (2009) SmartLDWS: A robust and scalable lane departure warning system for the smartphones. In: 2009 12th International IEEE Conference on Intelligent Transportation Systems (ITSC). IEEE, pp 1–6. https://doi.org/10.1109/ITSC.2009.5309685

  21. Lee JW (2002) A machine vision system for lane-departure detection. Comput Vis Image Underst 86(1):52–78. https://doi.org/10.1006/cviu.2002.0958

    Article  MATH  Google Scholar 

  22. Madrid N, Hurtik P (2016) Lane departure warning for mobile devices based on a fuzzy representation of images. Fuzzy Sets Syst 291(C):144–159. https://doi.org/10.1016/j.fss.2015.09.009

    Article  MathSciNet  Google Scholar 

  23. Mammar S, Glaser S, Netto M (2006) Time to line crossing for lane departure avoidance: A theoretical study and an experimental setting. IEEE Trans Intell Transp Syst 7(2):226–241. https://doi.org/10.1109/TITS.2006.874707

    Article  Google Scholar 

  24. McCall JC, Trivedi MM (2006) Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation. IEEE Trans Intell Transp Syst 7(1):20–37. https://doi.org/10.1109/TITS.2006.869595

    Article  Google Scholar 

  25. Midya A, Chakraborty J, Ranjan R (2017) Video error concealment through 3-D face model. Multimed Tools Appl 76(22):23931–23955. https://doi.org/10.1007/s1104

    Article  Google Scholar 

  26. Motwani A, Liu W, Sharma S, Sutton R, Bucknall R (2015) An interval Kalman filter–based fuzzy multi-sensor fusion approach for fault-tolerant heading estimation of an autonomous surface vehicle. Proceedings of the Institution of Mechanical Engineers, Part M: Journal of Engineering for the Maritime Environment 230(3):491–507. https://doi.org/10.1177/1475090215596180

    Article  Google Scholar 

  27. Narote SP, Bhujbal PN, Narote AS, Dhane DM (2018) A review of recent advances in lane detection and departure warning system. Pattern Recogn 73:216–234. https://doi.org/10.1016/j.patcog.2017.08.014

    Article  Google Scholar 

  28. Navarro J, Deniel J, Yousfi E, Jallais C, Bueno M, Fort A (2017) Influence of lane departure warnings onset and reliability on car drivers' behaviors. Appl Ergon 59:123–131. https://doi.org/10.1016/j.apergo.2016.08.010

    Article  Google Scholar 

  29. Pomerleau D (1995) RALPH: Rapidly adapting lateral position handler. In: Proceedings of the Intelligent Vehicles '95 Symposium. IEEE, pp 506–511. https://doi.org/10.1109/IVS.1995.528333

  30. Salari E, Ouyang D (2013) Camera-based Forward Collision and lane departure warning systems using SVM. In: 2013 IEEE 56th International Midwest Symposium on Circuits and Systems (MWSCAS). IEEE, pp 1278–1281. https://doi.org/10.1109/MWSCAS.2013.6674888

  31. Sam-Yong K, Se-Young O (2003) A driver adaptive lane departure warning system based on image processing and a fuzzy evolutionary technique. In: Proceedings of IEEE Intelligent Vehicles Symposium. IEEE, pp 361–365. https://doi.org/10.1109/IVS.2003.1212937

  32. Sandström M, Lampsijärvi E, Holmström A, Maconi G, Ahmadzai S, Meriläinen A, Hæggström E, Forsman P (2017) Detecting lane departures from steering wheel signal. Accid Anal Prev 99:272–278. https://doi.org/10.1016/j.aap.2016.12.003

    Article  Google Scholar 

  33. Singh Y, Mehra R (2015) Relative Study of Measurement Noise Covariance R and Process Noise Covariance Q of the Kalman Filter in Estimation. IOSR Journal of Electrical and Electronics Engineering 10(6):112–116

    Google Scholar 

  34. Son J, Yoo H, Kim S, Sohn K (2015) Real-time illumination invariant lane detection for lane departure warning system. Expert Syst Appl 42(4):1816–1824. https://doi.org/10.1016/j.eswa.2014.10.024

    Article  Google Scholar 

  35. Sternlund S, Strandroth J, Rizzi M, Lie A, Tingvall C (2017) The effectiveness of lane departure warning systems-A reduction in real-world passenger car injury crashes. Traffic Inj Prev 18(2):225–229. https://doi.org/10.1080/15389588.2016.1230672

    Article  Google Scholar 

  36. Tan D, Chen W, Wang H (2017) On the Use of Monte-Carlo Simulation and Deep Fourier Neural Network in Lane Departure Warning. IEEE Intell Transp Syst Mag 9(4):76–90. https://doi.org/10.1109/MITS.2017.2743204

    Article  Google Scholar 

  37. Tapia-Espinoza R, Torres-Torriti M (2013) Robust lane sensing and departure warning under shadows and occlusions. Sensors (Basel) 13(3):3270–3298. https://doi.org/10.3390/s130303270

    Article  Google Scholar 

  38. Tu C, van Wyk B, Hamam Y, Djouani K, Du S (2013) Vehicle position monitoring using Hough transform. IERI Procedia 4:316–322. https://doi.org/10.1016/j.ieri.2013.11.045

    Article  Google Scholar 

  39. Viswanath P, Swami P (2016) A robust and real-time image based lane departure warning system. In: 2016 IEEE International Conference on Consumer Electronics (ICCE). IEEE, pp 73–76. https://doi.org/10.1109/ICCE.2016.7430527

  40. Wang J-G, Lin C-J, Chen S-M (2010) Applying fuzzy method to vision-based lane detection and departure warning system. Expert Syst Appl 37(1):113–126. https://doi.org/10.1016/j.eswa.2009.05.026

    Article  Google Scholar 

  41. Yoo JH, Lee SW, Park SK, Kim DH (2017) A Robust Lane Detection Method Based on Vanishing Point Estimation Using the Relevance of Line Segments. IEEE Trans Intell Transp Syst 18(12):3254–3266. https://doi.org/10.1109/TITS.2017.2679222

    Article  Google Scholar 

  42. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334. https://doi.org/10.1109/34.888718

    Article  Google Scholar 

  43. Zhang JD, Jia XY, Li JF (2015) Integration of scanning and image processing algorithms for lane detection based on fuzzy method. J Intell Fuzzy Syst 29(6):2779–2786. https://doi.org/10.3233/IFS-151982

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by National Key Research and Development Program of China (2017YFB0102500), Natural Science Foundation of Jilin province (20170101133JC), Korea Foundation for Advanced Studies’ International Scholar Exchange Fellowship for the academic year of 2017-2018, and Jilin University (5157050847, 2017XYB252).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Xuelong Yin, Jindong Zhang & Ju Huang

  2. Key Laboratory of Symbol Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Jindong Zhang

  3. State Key Laboratory of Automobile Simulation and Control, Jilin University, Changchun, 130025, China

    Jindong Zhang

  4. College of Software, Jilin University, Changchun, 130012, China

    Xinggang Wu, Yanping Xu & Linyao Zhu

Authors
  1. Xuelong Yin
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jindong Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xinggang Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ju Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yanping Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Linyao Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jindong Zhang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yin, X., Zhang, J., Wu, X. et al. An improved lane departure warning algorithm based on fusion of F-Kalman filter and F-TLC. Multimed Tools Appl 78, 12203–12222 (2019). https://doi.org/10.1007/s11042-018-6762-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6762-2

Keywords