Skip to main content
SpringerLink
Log in

Multi-scale object detection for high-speed railway clearance intrusion

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Clearance intrusion of foreign objects is a great threat to high-speed railway operation safety. An accurate and fast detection system of foreign object intrusion is important. In this paper, a multi-scale foreign object detection algorithm named feature fusion CenterNet with variable focus multi-scale augmentation (FFCN-VFMS) is proposed for the high-speed railway clearance scene. We render the ground truth with two-dimensional Gaussian distributions to generate the confidence score of each region in the image. In addition, a variable focus multi-scale augmentation (VFMS) method is proposed for multi-scale object detection, which takes detection results as prior knowledge to find the range of subsequent detection that contains most small objects. Moreover, feature fusion CenterNet (FFCN) adopts bidirectional iterative deep aggregation (BiIDA) to fuse the features in different convolutional layers and a spatial pyramid pooling (SPP) module to fuse feature maps extracted by different receptive fields. Our method was tested on public PASCAL VOC2007 datasets and our railway clearance intrusion (RCI) datasets. In comparison with related methods, FFCN-VFMS achieves better performance than comparison detectors with respect to accuracy and speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Wisultschew C, Mujica G, Lanza-Gutierrez JM, Portilla J (2021) 3D-LIDAR based object detection and tracking on the edge of IoT for railway level crossing. IEEE Access 9:35718–35729

    Article  Google Scholar 

  2. Catalano A, Bruno FA, Galliano C, Pisco M, Persiano GV, Cutolo A, Cusano A (2017) An optical fiber intrusion detection system for railway security. Sensors and Actuators A-Physical 253:91– 100

    Article  Google Scholar 

  3. Li Z, Zhang J, Wang M, Chai J, Wu Y, Peng F (2020) An anti-noise ϕ-OTDR based distributed acoustic sensing system for high-speed railway intrusion detection. Laser Phys 30 (8):085103

    Article  Google Scholar 

  4. Xie Z, Qin Y (2019) High-speed railway perimeter intrusion detection approach based on internet of things. Advances in Mechanical Engineering 11(2):1687814018821511

    Article  Google Scholar 

  5. Wang Y, Yu Z, Zhu L (2018) Foreground detection with deeply learned multi-scale spatial-temporal features. Sensors 18(4269):1–16

    Google Scholar 

  6. Tastimur C, Karakose M, Akin E (2017) Image processing based level crossing detection and foreign objects recognition approach in railways. International Journal of Applied Mathematics Electronics and Computers 2017 Special Issue(1), pp 19–23

  7. Guo B, Xingfang Z, Yingzi L, Liqiang Z, Zujun Y (2018) Novel registration and fusion algorithm for multimodal railway images with different field of views. J Adv Trans 2018(7836169):1–16

    Google Scholar 

  8. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Article  Google Scholar 

  9. Guo Y, Liu Y, Oerlemans A, Lao S, Wu S, Lew MS (2016) Deep learning for visual understanding: A review. Neurocomputing 187:27–48

    Article  Google Scholar 

  10. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: 26th Advances in neural information processing systems (NIPS 2012), pp 1106–1114. Lake Tahoe, Nevada, USA

  11. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd International conference on learning representations (ICLR 2015), pp 1–14. San Diego, CA, USA

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR 2016), pp 770–778. Las Vegas, Nevada, USA

  13. Huang G, Liu Z, Maaten LVD, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 2261–2269. Honolulu, HI, USA

  14. Tan M, Le Q (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. In: Proceedings of the 36th international conference on machine learning, pp 6105–6114. Long Beach, California, USA

  15. Sun C, Ai Y, Wang S, Zhang W (2020) Mask-guided SSD for small-object detection. Appl Intell

  16. Wang Y, Zhu L, Yu Z (2019) Foreground detection for infrared videos with multiscale 3-D fully convolutional network. IEEE Geosci Remote Sens Lett 16(5):712–716

    Article  Google Scholar 

  17. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR 2014), pp 580–587. Columbus, OH, USA

  18. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(9):1904–1916

    Article  Google Scholar 

  19. Girshick RB (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision (ICCV 2015), pp 1440–1448. Santiago, Chile

  20. Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(6):1137–1149

    Article  Google Scholar 

  21. He K, Gkioxari G, Dollar P, Girshick R (2020) Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence 42(2):386–397

    Article  Google Scholar 

  22. Cai Z, Vasconcelos N (2018) Cascade R-CNN: Delving into high quality object detection. In: Proceedings of the 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 6154–6162. Salt Lake City, UT, USA

  23. Redmon J, Divvala SK, Girshick RB, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR 2016), pp 779–788. Las Vegas, NV, USA

  24. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR 2017), pp 6517–6525. Honolulu, HI, USA

  25. Redmon J, Farhadi A (2018) YOLOv3: An incremental improvement. arXiv:1804.02767

  26. Liu W, Anguelov D, Erhan D, Szegedy C, Reed SE, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector. In: Proceedings of the European conference on computer vision (ECCV 2016), pp 21–37

  27. Fu C-Y, Liu W, Ranga A, Tyagi A, Berg AC (2017) DSSD : Deconvolutional single shot detector. arXiv:1701.06659

  28. Zhang S, Wen L, Bian X, Lei Z, Li SZ (2018) Single-shot refinement neural network for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR 2018), pp 4203–4212. Salt Lake City, UT, USA

  29. Tian Z, Shen C, Chen H, He T (2019) FCOS: Fully convolutional one-stage object detection. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 9627–9636. Seoul, Korea (South)

  30. Law H, Deng J (2020) Cornernet: Detecting objects as paired keypoints. Int J Comput Vis 128(3):642–656

    Article  Google Scholar 

  31. Duan K, Bai S, Xie L, Qi H, Huang Q, Tian Q (2019) CenterNet: Keypoint triplets for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV 2019), pp 6569–6578. Seoul, Korea

  32. Zhou X, Wang D, Krähenbühl P (2019) Objects as points. arXiv:1904.07850

  33. Lin TY, Goyal P, Girshick R, He K, Dollar P (2020) Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 42(2):318–327

    Article  Google Scholar 

  34. Zhou Y, Tao X, Yu Z, Fujita H (2019) Train-movement situation recognition for safety justification using moving-horizon TBM-based multisensor data fusion. Knowl-Based Syst 177:117– 126

    Article  Google Scholar 

  35. Li Z, Zhang J, Wang M, Zhong Y, Peng F (2020) Fiber distributed acoustic sensing using convolutional long short-term memory network: a field test on high-speed railway intrusion detection. Opt Express 28(3):2925–2938

    Article  Google Scholar 

  36. Zhangyu W, Guizhen Y, Xinkai W, Haoran L, Da L (2021) A camera and lidar data fusion method for railway object detection. IEEE Sensors J, pp 1–13

  37. Gao P, Zhang Q, Wang F, Xiao L, Fujita H, Zhang Y (2020) Learning reinforced attentional representation for end-to-end visual tracking. Inf Sci 517:52–67

    Article  Google Scholar 

  38. Gao P, Yuan R, Wang F, Xiao L, Fujita H, Zhang Y (2020) Siamese attentional keypoint network for high performance visual tracking. Knowl-Based Syst 193:105448

    Article  Google Scholar 

  39. Yang W, Liqiang Z, Zujun Y, Baoqing G (2016) High-speed railway clearance surveillance system based on convolutional neural networks. In: Proceedings of the 8th international conference on digital image processing (ICDIP 2016), pp 1–6. Chengu, China

  40. Huang H, Liang L, Zhao G, Yang Y, Ou K (2019) Railway clearance intrusion detection in aerial video based on convolutional neural network. In: Proceedings of the Chinese control and decision conference (CCDC 2019), pp 1644–1648. Nanchang, China

  41. Wang W, Zhu L, Guo B (2019) Reliable identification of redundant kernels for convolutional neural network compression. J Vis Commun Image Represent 63(102582):1–12

    Google Scholar 

  42. Yan X, Huiqing T, Lili HU (2020) Railway foreign body intrusion detection based on faster R-CNN network model. J China Railway Society 42(5):91–98

    Google Scholar 

  43. Xiaoying Y, Hongsheng S, Ze J, Yu D (2020) Detection method of railway intruding obstacle based on YOLO algorithm. J Lanzhou Jiaotong University 39(2):37–42

    Google Scholar 

  44. Guo B, Shi J, Zhu L, Yu Z (2019) High-speed railway clearance intrusion detection with improved SSD network. Appl Sci 9(15):1–20

    Google Scholar 

  45. Li Y, Dong H, Li H, Zhang X, Zhang B, Xiao Z (2020) Multi-block SSD based on small object detection for UAV railway scene surveillance. Chin J Aeronaut 33(6):1747–1755

    Article  Google Scholar 

  46. Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR 2017), pp 936–944. Honolulu, HI, USA

  47. Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR 2018), pp 8759–8768. Salt Lake City, UT, USA

  48. Tan M, Pang R, Le QV (2020) Efficientdet: Scalable and efficient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR 2020), pp 10781–10790. Seattle, WA, USA

  49. Yu F, Wang D, Shelhamer E, Darrell T (2018) Deep layer aggregation. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 2403–2412. Salt Lake City, UT, USA

  50. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR 2017), pp 6230–6239. Honolulu, HI, USA

  51. Huang Z, Wang J, Fu X, Yu T, Guo Y, Wang R (2020) DC-SPP-YOLO: Dense connection and spatial pyramid pooling based YOLO for object detection. Inf Sci 522:241–258

    Article  MathSciNet  Google Scholar 

  52. Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: Proceedings of the IEEE international conference on computer vision (ICCV 2017), pp 764–773

Download references

Author information

Authors and Affiliations

  1. School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing, 100044, People’s Republic of China

    Runliang Tian, Hongmei Shi, Baoqing Guo & Liqiang Zhu

  2. Key Laboratory of Vehicle Advanced Manufacturing, Measuring and Control Technology, (Beijing Jiaotong University), Ministry of Education, Beijing, People’s Republic of China

    Runliang Tian, Hongmei Shi, Baoqing Guo & Liqiang Zhu

Authors
  1. Runliang Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongmei Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baoqing Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liqiang Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongmei Shi.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the National Natural Science Foundation of China under Grant 62076022.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tian, R., Shi, H., Guo, B. et al. Multi-scale object detection for high-speed railway clearance intrusion. Appl Intell 52, 3511–3526 (2022). https://doi.org/10.1007/s10489-021-02534-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02534-9

Keywords

Search

Navigation