Abstract
Lane detection models based deep semantic segmentation often fail in challenging scenarios for lacking enough contextual information. In this paper, we first point out that a soft label extracted from a trained semantic segmentation model is capable of encoding rich contextual information. Then, we find that introducing the soft label extracted from the top layer of a well-trained model to low-level features gains substantial improvement. Based on this observation, we propose a novel framework, i.e., Soft Label Attention(SLA), which allows a model to learn contextual attention from the higher layer through a loss function. Significantly, our SLA does not increase the run time in the inference stage because it is only applied in the training stage. We apply the proposed SLA to modified ERFNet and propose an attention deep neural network (DNN) for lane detection (ERFNet-SLA). Tests on the TuSimple dataset show that the ERFNet-SLA outperforms ResNet-18 and ResNet-34 by 3.4% and 3.3% respectively with at least 2.3× faster in computation speed, and is 12.5× faster than SCNN with 0.6% performance loss. Tests on the CULane dataset show that the ERFNet-SLA outperforms ResNet-50 by 5.6% and is 16× faster than ResNet-101 with 0.5% performance loss.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bottou L (2010) Large-scale machine learning with stochastic gradient descent. COMPSTAT 177–186
Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848
Chiu K-Y, Lin S-F (2005) Lane detection using color-based segmentation. In: IEEE Proceedings. Intelligent vehicles symposium, 2005. pp 706–711
Fan G-F, Yu M, Dong S-Q, Yeh Y-H, Hong W-C (2021) Forecasting short-term electricity load using hybrid support vector regression with grey catastrophe and random forest modeling. Util Policy 73:101294
Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 3146–3154
Ghafoorian M, Nugteren C, Baka N, Booij O, Hofmann M (2018) El-gan: embedding loss driven generative adversarial networks for lane detection. In: Proceedings of the european conference on computer vision (ECCV) Workshops, pp 256–272
Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580
Hinton GE, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv:1503.02531
Hou Y, Ma Z, Liu C, Loy CC (2019) Learning lightweight lane detection cnns by self attention distillation. In: 2019 IEEE/CVF International conference on computer vision (ICCV), pp 1013–1021
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 7132–7141
Huang G, Chen D, Li T, Wu F, van der Maaten L, Weinberger KQ (2018) Multi-scale dense networks for resource efficient image classification. In: International conference on learning representations
Hur J, Kang S-N, Seo S-W (2013) Multi-lane detection in urban driving environments using conditional random fields. In: 2013 IEEE Intelligent vehicles symposium (IV), pp 1297–1302
Jung H, Min J, Kim J (2013) An efficient lane detection algorithm for lane departure detection. In: 2013 IEEE Intelligent vehicles symposium (IV), pp 976–981
Li Z-Q, Ma H-M, Liu Z-Y (2016) Road lane detection with gabor filters. In: 2016 International conference on information system and artificial intelligence (ISAI), pp 436–440
Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 510–519
Ma N, Zhang X, Zheng H-T, Sun J (2018) Shufflenet v2: Practical guidelines for efficient cnn architecture design. In: Proceedings of the european conference on computer vision (ECCV), pp 122– 138
Neven D, Brabandere BD, Georgoulis S, Proesmans M, Gool LV (2018) Towards end-to-end lane detection: an instance segmentation approach. In: 2018 IEEE Intelligent vehicles symposium (IV), pp 286–291
Pan X, Shi J, Luo P, Wang X (2018) xiaoou Tang: spatial as deep: spatial cnn for traffic scene understanding. In: AAAI-18 AAAI Conference on artificial intelligence, pp 7276–7283
Paszke A, Chaurasia A, Kim S, Culurciello E (2016) Enet: a deep neural network architecture for real-time semantic segmentation. arXiv:1606.02147
Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch
Romera E, Alvarez JM, Bergasa LM, Arroyo R (2018) Erfnet: efficient residual factorized convnet for real-time semantic segmentation. IEEE Trans Intell Transp Syst 19(1):263–272
Salvaris M, Dean D, Tok WH (2018) Generative adversarial networks. arXiv:1406.2661, 187–208
Srinivasu PN, Bhoi AK, Jhaveri RH, Reddy GT, Bilal M (2021) Probabilistic deep q network for real-time path planning in censorious robotic procedures using force sensors. J Real-Time Image Proc 18(5):1773–1785
Srinivasu PN, SivaSai JG, Ijaz MF, Bhoi AK, Kim W, Kang JJ (2021) Classification of skin disease using deep learning neural networks with mobilenet v2 and lstm. Sensors 21(8):2852
Wang Y, Shen D, Teoh EK (2000) Lane detection using spline model. Pattern Recogn Lett 21(9):677–689
Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the european conference on computer vision (ECCV), pp 3–19
Xiao D, Yang X, Li J, Islam M (2020) Attention deep neural network for lane marking detection. Knowl Based Syst 194:105584
Xue X, Feng J, Gao Y, Liu M, Zhang W, Sun X, Zhao A, Guo S (2019) Convolutional recurrent neural networks with a self-attention mechanism for personnel performance prediction. Entropy 21(12):1227
Youjin T, Wei C, Xingguang L, Lei C (2018) A robust lane detection method based on vanishing point estimation. Procedia Comput Sci 131:354–360
Yu F, Koltun V (2016) Multi-scale context aggregation by dilated convolutions. In: ICLR 2016: International Conference on learning representations 2016
Zagoruyko S, Komodakis N (2017) Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. In: ICLR 2017: International Conference on learning representations 2017
Zhang H, Goodfellow I, Metaxas D, Odena A (2019) Self-attention generative adversarial networks. In: ICML 2019: thirty-Sixth international conference on machine learning, pp 7354–7363
Zhang R, Xiong Z (2019) Recurrent neural network model with self-attention mechanism for fault detection and diagnosis. In: 2019 Chinese Automation Congress (CAC), pp 4706–4711. IEEE
Zhang Z, Zhang X, Peng C, Xue X, Sun J (2018) Exfuse: enhancing feature fusion for semantic segmentation. In: Proceedings of the european conference on computer vision (ECCV), pp 273–288
Acknowledgments
This work is supported by Zhejiang Dahua Technology Co., Ltd.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yang, X., Yu, Y., Zhang, Z. et al. Lightweight lane marking detection CNNs by self soft label attention. Multimed Tools Appl 82, 5607–5626 (2023). https://doi.org/10.1007/s11042-022-13442-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-13442-6