Skip to main content
SpringerLink
Log in

Real-Time Lane Detection by Using Biologically Inspired Attention Mechanism to Learn Contextual Information

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Background State-of-the-art lane detection methods have achieved prominent performance in complex scenarios, but many limits have also existed. For example, only a fixed number of lanes can be detected, and the cost of detection time is unaffordable in many cases. Methods Inspired by human vision, attention mechanism makes network learning more concerned features. In this paper, we propose a real-time lane detection method by using attention mechanism. The network proposed consists of three modules: an encoder module that extracts the feature of lanes; the instance feature maps of lanes are predicted by two decoder modules, namely binary decoder and embeddable decoder. In the encoder, we use the biologically inspired attention to extract features, which contain many details of the target area. The correlation between the features obtained from the convolutions and that extracted by the attention is established to learn the contextual information. In the decoder, the contextual information is fused with the features from up-sampling, to compensate for the lost detailed information. Binary decoder classifies all the pixels into lane or background. Embeddable decoder obtains the distinguishable lanes. And then, the outputs of the binary decoder serve as one of the inputs to the embeddable decoder to guiding the generation of exact pixel points on the lanes. Results Comparative experiments on two benchmarks (TuSimple and Caltech lanes datasets) show that the proposed method is independent of lane number and lane pattern. It can handle an indefinite number of lanes and run at 10ms in the TuSimple dataset. Conclusions Experiments verify that our method outperforms a lot of state-of-the-art methods while maintaining a real-time performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Ding D, Ding Z, Wei G, Han F. An improved reinforcement learning algorithm based on knowledge transfer and applications in autonomous vehicles. Neurocomputing. 2019;361:243–55.

    Article  Google Scholar 

  2. Liu X, Deng Z. Segmentation of drivable road using deep fully convolutional residual network with pyramid pooling. Cogn Comput. 2018;10(2):272–81.

    Article  MathSciNet  Google Scholar 

  3. Zou Q, Jiang H, Dai Q, Yue Y, Chen L, Wang Q. Robust lane detection from continuous driving scenes using deep neural networks. IEEE Trans Veh Technol. 2019;69(1):41–54.

    Article  Google Scholar 

  4. El Hajjouji I, Mars S, Asrih Z, El Mourabit A. A novel fpga implementation of hough transform for straight lane detection. Int J Eng Sci Technol. 2020;23(2):274–80.

    Google Scholar 

  5. Hssayeni MD, Saxena S, Ptucha R, Savakis A. Distracted driver detection: Deep learning vs handcrafted features. J Electron Imaging. 2017;2017(10):20–6.

    Article  Google Scholar 

  6. Zare MR, Alebiosu DO, Lee SL. Comparison of handcrafted features and deep learning in classification of medical x-ray images. In 2018 Fourth International Conference on Information Retrieval and Knowledge Management (CAMP). IEEE 2018;1–5. 

  7. Fukui H, Hirakawa T, Yamashita T, Fujiyoshi H. Attention branch network: Learning of attention mechanism for visual explanation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019;10705–14.

  8. Tian Y, Gelernter J, Wang X, Chen W, Gao J, Zhang Y, Li X. Lane marking detection via deep convolutional neural network. Neurocomputing. 2018;280:46–55.

    Article  Google Scholar 

  9. Wang S, Wang X, Wang S, Wang D. Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting. Int J Electr Power Energy Syst. 2019;109:470–9.

    Article  Google Scholar 

  10. Fan R, Wang X, Hou Q, Liu H, Mu T-J. Spinnet: Spinning convolutional network for lane boundary detection. Comput Vis Media. 2019;5(4):417–28.

    Article  Google Scholar 

  11. Kim J, Park C. End-to-end ego lane estimation based on sequential transfer learning for self-driving cars. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017;30–8.

  12. Hou Y, Ma Z, Liu C, Loy CC. Learning lightweight lane detection cnns by self attention distillation. In Proceedings of the IEEE International Conference on Computer Vision. 2019;1013–21.

  13. Li J, Mei X, Prokhorov D, Tao D. Deep neural network for structural prediction and lane detection in traffic scene. IEEE Trans Neural Netw Learn Syst. 2016;28(3):690–703.

    Article  Google Scholar 

  14. Yoo S, Seok Lee H, Myeong H, Yun S, Park H, Cho J, Hoon Kim D. End-to-end lane marker detection via row-wise classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020;1006–7.

  15. Stasse O, Kuniyoshi Y, Cheng G. Development of a biologically inspired real-time visual attention system. In International Workshop on Biologically Motivated Computer Vision. Springer 2000;150–9.

  16. Parajuli A, Celenk M, Riley HB, et al. Robust lane detection in shadows and low illumination conditions using local gradient features. Open J Appl Sci. 2013;3(01):68.

    Article  Google Scholar 

  17. Sivaraman S, Trivedi MM. Integrated lane and vehicle detection, localization, and tracking: A synergistic approach. IEEE Trans Intell Transp Syst. 2013;14(2):906–17.

    Article  Google Scholar 

  18. Suddamalla U, Kundu S, Farkade S, Das A. A novel algorithm of lane detection addressing varied scenarios of curved and dashed lanemarks. In 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA). IEEE 2015;87–92.

  19. Kim J-H, Kim S-K, Lee S-H, Lee T-M, Lim J. Lane recognition algorithm using lane shape and color features for vehicle black box. In 2018 International Conference on Electronics, Information, and Communication (ICEIC). IEEE 2018;1–2.

  20. Kim J. Efficient vanishing point detection for driving assistance based on visual saliency map and image segmentation from a vehicle black-box camera. Symmetry. 2019;11(12):1492–510.

    Article  Google Scholar 

  21. Song W, Yang Y, Fu M, Li Y, Wang M. Lane detection and classification for forward collision warning system based on stereo vision. IEEE Sensors J. 2018;18(12):5151–63.

    Article  Google Scholar 

  22. Zhou S, Jiang Y, Xi J, Gong J, Xiong G, Chen H. A novel lane detection based on geometrical model and gabor filter. In 2010 IEEE Intelligent Vehicles Symposium. IEEE 2010;59–64.

  23. Wu D, Zhao R, Wei Z. A multi-segment lane-switch algorithm for efficient real-time lane detection. In 2014 IEEE International Conference on Information and Automation (ICIA). 2014;273–8.

  24. Zhang Z, Ma X, et al. Lane recognition algorithm using the hough transform based on complicated conditions. J Comput Commun. 2019;7(11):65–75.

    Article  Google Scholar 

  25. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint https://arxiv.org/abs/1409.1556, 2014.

  26. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In Proc IEEE Conf Comput Vis Pattern Recognit. 2017;4700–8.

  27. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proc IEEE Conf Comput Vis Pattern Recognit. 2018;4510–20.

  28. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.

    Article  Google Scholar 

  29. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In Proc IEEE Conf Comput Vis Pattern Recognit. 2015;3431–40.

  30. He B, Ai R, Yan Y, Lang X. Accurate and robust lane detection based on dual-view convolutional neutral network. In 2016 IEEE Intelligent Vehicles Symposium (IV). IEEE 2016;1041–6.

  31. Huang Y, Chen S, Chen Y, Jian Z, Zheng N. Spatial-temproal based lane detection using deep learning. In IFIP International conference on artificial Intelligence applications and innovations. Springer 2018;143–54.

  32. Ye YY, Hao XL, Chen HJ. Lane detection method based on lane structural analysis and cnns. IET Intell Transp Syst. 2018;12(6):513–20.

    Article  Google Scholar 

  33. Pan X, Shi J, Luo P, Wang X, Tang X. Spatial as deep: Spatial cnn for traffic scene understanding. In AAAI Conference on Artificial Intelligence (AAAI). 2018.

  34. Neven D, De Brabandere B, Georgoulis S, Proesmans M, Van Gool L. Towards end-to-end lane detection: an instance segmentation approach. In 2018 IEEE intelligent vehicles symposium (IV). IEEE 2018;286–91.

  35. Qiao D, Wu X, Wang T. A lane recognition based on line-cnn network. In 2020 Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC). IEEE 2020;96–100.

  36. Sun J, Jiang J, Liu Y. An introductory survey on attention mechanisms in computer vision problems. In 2020 6th International Conference on Big Data and Information Analytics (BigDIA). IEEE 2020;295–300.

  37. Chen Y, Nakano T, Lin L, Guo W, Mahfuz MU. Biologically inspired computing and networking. Mob Netw Appl 2021;1–3.

  38. Shi Y, Tian Y, Wang Y, Zeng W, Huang T. Learning long-term dependencies for action recognition with a biologically-inspired deep network. In Proc IEEE Int Conf Comput Vis. 2017;716–25.

  39. Jaderberg M, Simonyan K, Zisserman A et al. Spatial transformer networks. In Adv Neural Inf Process Syst. 2015;2017–25.

  40. Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In Proc IEEE Conf Comput Vis Pattern Recognit. 2018;7132–41.

  41. Woo S, Park J, Lee J-Y, So Kweon I. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV). 2018;3–19.

  42. Neven D, De Brabandere B, Georgoulis S, Proesmans M, Van Gool L. Fast scene understanding for autonomous driving. arXiv preprint https://arxiv.org/abs/1708.02550, 2017.

  43. TuSimple. Tusimple lane detection benchmark, https://github.com/TuSimple/tusimple-benchmark, 2017.

  44. Shirke S, Udayakumar R. Lane datasets for lane detection. In 2019 International Conference on Communication and Signal Processing (ICCSP). IEEE 2019;4–6.

  45. Huval B, Wang T, Tandon S, Kiske J, Song W, Pazhayampallil J, Andriluka M, Rajpurkar P, Migimatsu T, Cheng-Yue R, et al. An empirical evaluation of deep learning on highway driving. arXiv preprint https://arxiv.org/abs/1504.01716, 2015.

  46. Paszke A, Chaurasia A, Kim S, Culurciello E. Enet: A deep neural network architecture for real-time semantic segmentation. arXiv preprint https://arxiv.org/abs/1606.02147, 2016.

  47. Pizzati F, Allodi M, Barrera A, García F. Lane detection and classification using cascaded cnns. In International Conference on Computer Aided Systems Theory. Springer 2019;95–103.

  48. Yang W-J, Cheng Y-T, Chung P-C. Improved lane detection with multilevel features in branch convolutional neural networks. IEEE Access. 2019;7:173148–56.

    Article  Google Scholar 

  49. Kim J, Kim J, Jang G-J, Lee M. Fast learning method for convolutional neural networks using extreme learning machine and its application to lane detection. Neural Netw. 2017;87:109–21.

    Article  Google Scholar 

  50. Ruyi J, Reinhard K, Tobi V, Shigang W. Lane detection and tracking using a new lane model and distance transform. Mach Vis Appl. 2011;22(4):721–37.

    Article  Google Scholar 

  51. Benligiray B, Topal C, Akinlar C. Video-based lane detection using a fast vanishing point estimation method. In 2012 IEEE International Symposium on Multimedia. IEEE 2012;348–351.

  52. Irshad A, Khan AA, Yunus I, Shafait F. Real-time lane departure warning system on a lower resource platform. In 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA). IEEE 2017;1–8.

Download references

Acknowledgements

This work was supported by the National Nature Science Foundation of China Grand (No: 91320301), the Innovation Engineering Project for New Energy and Intelligent Networked Automobile of Anhui Province, the Youth Spark Project of the Dean Fund of Hefei Institutes of Physical Science, CAS (No: YZJJ2020QN20), the Natural Science Foundation of Education Bureau of Anhui Province (No: KJ2020A0111), and the Anhui Provincial Key Laboratory of Multimodal Cognitive Computation (No: MMC202007). This work also was supported by the Nature Science Foundation of Anhui Province, China (No: 2108085MF195), and the Talent Research Foundation of Hefei University (No: 20RC16). The authors would like to thank the anonymous reviews for their helpful and constructive comments and suggestions regarding this manuscript.

Author information

Authors and Affiliations

  1. Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, 230031, China

    Lu Zhang, Fengling Jiang, Bin Kong, Jing Yang & Can Wang

  2. University of Science and Technology of China, Hefei, 230026, China

    Lu Zhang, Fengling Jiang, Bin Kong, Jing Yang & Can Wang

  3. The Key Laboratory of Biomimetic Sensing and Advanced Robot Technology, Hefei, 230031, China

    Lu Zhang, Fengling Jiang, Bin Kong, Jing Yang & Can Wang

  4. School of Computer Science and Technology, Hefei Normal University, Hefei, 230061, China

    Fengling Jiang

  5. Hefei University, Hefei, 230601, China

    Jing Yang

Authors
  1. Lu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fengling Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Can Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Kong.

Ethics declarations

Informed Consent

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5). Additional informed consent was obtained from all patients for which identifying information is included in this article.

Human and Animal Rights

This article does not contain any studies with human or animal subjects performed by the any of the authors.

Conflict of Interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, L., Jiang, F., Kong, B. et al. Real-Time Lane Detection by Using Biologically Inspired Attention Mechanism to Learn Contextual Information. Cogn Comput 13, 1333–1344 (2021). https://doi.org/10.1007/s12559-021-09935-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-021-09935-5

Keywords

Search

Navigation