Skip to main content
SpringerLink
Log in

Unsupervised intrusion detection for rail transit based on anomaly segmentation

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Detecting intrusions in rail transit can be challenging using traditional supervised methods, as they only detect target categories present in the training dataset and require extensive manual annotations. This paper proposes an unsupervised method for railroad intrusion detection based on anomaly segmentation, called heterogeneous uninformed students network (HUS-Net). No obstacle data is needed for training with this method, and it does not restrict identified objects to specific categories. HUS-Net utilizes a pre-trained descriptive model as the teacher network and distils its knowledge into two heterogeneous students via multi-level feature pyramid matching and reconstruction techniques. The representation discrepancy between the students and the teacher is utilized to identify railroad intrusion events and locate anomalous objects. The model is evaluated on images captured by an onboard vision system in real rail transit operating environments. Experimental results demonstrate that HUS-Net can accurately and efficiently detect intrusion events on railroads and segment invading objects, achieving better performance than other anomaly segmentation methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Availability of data and materials

The data that support the findings of this study are available from the corresponding author, D.H., upon reasonable request.

References

  1. Jin, Z., He, D., Wei, Z.: Intelligent fault diagnosis of train axle box bearing based on parameter optimization VMD and improved DBN. Eng. Appl. Artif. Intell. (2022) https://doi.org/10.1016/j.engappai.2022.104713

  2. He, D., Liu, C., Jin, Z., Ma, R., Chen, Y., Shan, S.: Fault diagnosis of flywheel bearing based on parameter optimization variational mode decomposition energy entropy and deep learning. Energy (2022). https://doi.org/10.1016/j.energy.2021.122108

  3. Ristić-Durrant, D., Franke, M., Michels, K.: A review of vision-based on-board obstacle detection and distance estimation in railways. Sensors (2021). https://doi.org/10.3390/s21103452

  4. Xu, Y., Gao, C., Yuan, L., Tang, S., Wei, G.: Real-time obstacle detection over rails using deep convolutional neural network. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 1007–1012 (2019). https://doi.org/10.1109/ITSC.2019.8917091

  5. Ye, T., Zhao, Z., Wang, S., Zhou, F., Gao, X.: A stable lightweight and adaptive feature enhanced convolution neural network for efficient railway transit object detection. IEEE Trans. Intell. Transp. Syst. 23(10), 17952–17965 (2022). https://doi.org/10.1109/TITS.2022.3156267

    Article  Google Scholar 

  6. Ye, T., Zhang, X., Zhang, Y., Liu, J.: Railway traffic object detection using differential feature fusion convolution neural network. IEEE Trans. Intell. Transp. Syst. 22(3), 1375–1387 (2021). https://doi.org/10.1109/TITS.2020.2969993

    Article  Google Scholar 

  7. He, D., Qiu, Y., Miao, J., Zou, Z., Li, K., Ren, C., Shen, G.: Improved mask r-CNN for obstacle detection of rail transit. Measurement 190, 110728 (2022). https://doi.org/10.1016/j.measurement.2022.110728

    Article  Google Scholar 

  8. Cao, Z., Qin, Y., Xie, Z., Liu, Q., Zhang, E., Wu, Z., Yu, Z.: An effective railway intrusion detection method using dynamic intrusion region and lightweight neural network. Measurement 191, 110564 (2022). https://doi.org/10.1016/j.measurement.2021.110564

    Article  Google Scholar 

  9. Huang, H., Zhao, G., Bo, Y., Yu, J., Liang, L., Yang, Y., Ou, K.: Railway intrusion detection based on refined spatial and temporal features for UAV surveillance scene. Measurement 211, 112602 (2023). https://doi.org/10.1016/j.measurement.2023.112602

    Article  Google Scholar 

  10. Tao, X., Gong, X., Zhang, X., Yan, S., Adak, C.: Deep learning for unsupervised anomaly localization in industrial images: a survey. IEEE Trans. Instrum. Meas. 71, 1–21 (2022). https://doi.org/10.1109/TIM.2022.3196436

    Article  Google Scholar 

  11. Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Niethammer, M., Styner, M., Aylward, S., Zhu, H., Oguz, I., Yap, P.-T., Shen, D. (eds.) Information Processing in Medical Imaging, pp. 146–157. Springer, Cham (2017)

  12. Schlegl, T., Seeböck, P., Waldstein, S.M., Langs, G., Schmidt-Erfurth, U.: f-anogan: fast unsupervised anomaly detection with generative adversarial networks. Med. Image Anal. 54, 30–44 (2019). https://doi.org/10.1016/j.media.2019.01.010

    Article  PubMed  Google Scholar 

  13. Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: Ganomaly: semi-supervised anomaly detection via adversarial training. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) Computer Vision—ACCV 2018, pp. 622–637. Springer, Cham (2019)

    Chapter  Google Scholar 

  14. Shi, Y., Yang, J., Qi, Z.: Unsupervised anomaly segmentation via deep feature reconstruction. Neurocomputing 424, 9–22 (2021). https://doi.org/10.1016/j.neucom.2020.11.018

    Article  Google Scholar 

  15. Yu, J., Zheng, Y., Wang, X., Li, W., Wu, Y., Zhao, R., Wu, L.: Fastflow: unsupervised anomaly detection and localization via 2d normalizing flows. CoRR (2021). arXiv:2111.07677

  16. Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: Uninformed students: student-teacher anomaly detection with discriminative latent embeddings. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4182–4191. IEEE Computer Society, Los Alamitos (2020). https://doi.org/10.1109/CVPR42600.2020.00424 . https://doi.ieeecomputersociety.org/10.1109/CVPR42600.2020.00424

  17. Salehi, M., Sadjadi, N., Baselizadeh, S., Rohban, M.H., Rabiee, H.R.: Multiresolution knowledge distillation for anomaly detection. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14897–14907 (2021). https://doi.org/10.1109/CVPR46437.2021.01466

  18. Wang, G., Han, S., Ding, E., Huang, D.: Student-teacher feature pyramid matching for unsupervised anomaly detection. CoRR (2021). arXiv:2103.04257

  19. Deng, H., Li, X.: Anomaly detection via reverse distillation from one-class embedding. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9727–9736 (2022). https://doi.org/10.1109/CVPR52688.2022.00951

  20. Wang, W., Chang, F., Liu, C.: Mutuality-oriented reconstruction and prediction hybrid network for video anomaly detection. SIViP 16, 1747–1754 (2022). https://doi.org/10.1007/s11760-021-02131-w

    Article  Google Scholar 

  21. Hamilton, M., Zhang, Z., Hariharan, B., Snavely, N., Freeman, W.T.: Unsupervised semantic segmentation by distilling feature correspondences. CoRR (2022). arXiv:2203.08414

  22. Kim, W., Kanezaki, A., Tanaka, M.: Unsupervised learning of image segmentation based on differentiable feature clustering. IEEE Trans. Image Process. 29, 8055–8068 (2020). https://doi.org/10.1109/TIP.2020.3011269

    Article  ADS  Google Scholar 

  23. Wan, Q., Gao, L., Li, X., Wen, L.: Unsupervised image anomaly detection and segmentation based on pretrained feature mapping. IEEE Trans. Ind. Inf. 19(3), 2330–2339 (2023). https://doi.org/10.1109/TII.2022.3182385

    Article  Google Scholar 

  24. Wan, Q., Gao, L., Li, X.: Logit inducing with abnormality capturing for semi-supervised image anomaly detection. IEEE Trans. Instrum. Meas. 71, 1–12 (2022). https://doi.org/10.1109/TIM.2022.3205674

    Article  Google Scholar 

  25. Gasparini, R., D’Eusanio, A., Borghi, G., Pini, S., Scaglione, G., Calderara, S., Fedeli, E., Cucchiara, R.: Anomaly detection, localization and classification for railway inspection. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 3419–3426 (2021). https://doi.org/10.1109/ICPR48806.2021.9412972

  26. Wang, T., Zhang, Z., Tsui, K.-L.: A deep generative approach for rail foreign object detections via semisupervised learning. IEEE Trans. Ind. Inf. 19(1), 459–468 (2023). https://doi.org/10.1109/TII.2022.3149931

    Article  Google Scholar 

  27. Yang, P., Jin, W., Tang, P.: Anomaly detection of railway catenary based on deep convolutional generative adversarial networks. In: Xu, B. (ed.) Proceedings Of 2018 Ieee 3rd Advanced Information Technology, Electronic And Automation Control Conference (IAEAC 2018), pp. 1366–1370 (2018). IEEE. 3rd IEEE Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, Oct 12–14 (2018)

  28. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90

  29. Yamada, S., Kamiya, S., Hotta, K.: Reconstructed student-teacher and discriminative networks for anomaly detection. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2725–2732 (2022). https://doi.org/10.1109/IROS47612.2022.9981509

  30. Li, G., Fang, Q., Zha, L., Gao, X., Zheng, N.: Ham: Hybrid attention module in deep convolutional neural networks for image classification. Pattern Recogn. 129, 108785 (2022). https://doi.org/10.1016/j.patcog.2022.108785

    Article  Google Scholar 

  31. Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: Mvtec ad - a comprehensive real-world dataset for unsupervised anomaly detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9584–9592 (2019). https://doi.org/10.1109/CVPR.2019.00982

  32. Akcay, S., Ameln, D., Vaidya, A., Lakshmanan, B., Ahuja, N., Genc, U.: Anomalib: A deep learning library for anomaly detection. In: 2022 IEEE International Conference on Image Processing (ICIP), pp. 1706–1710 (2022). https://doi.org/10.1109/ICIP46576.2022.9897283

  33. Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 2011–2023 (2020). https://doi.org/10.1109/TPAMI.2019.2913372

    Article  PubMed  Google Scholar 

  34. Li, X., Wang, W., Hu, X., Yang, J.: Selective kernel networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 510–519 (2019). https://doi.org/10.1109/CVPR.2019.00060

  35. Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision—ECCV 2018, pp. 3–19. Springer, Cham (2018)

    Chapter  Google Scholar 

  36. Hou, Q., Zhou, D., Feng, J.: Coordinate attention for efficient mobile network design. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13708–13717 (2021). https://doi.org/10.1109/CVPR46437.2021.01350

Download references

Funding

The research was supported by National Natural Science Foundation of China [Grant No. 52072081], Major Project of Science and Technology of Guangxi Province of China [Grant No. Guike AA20302010], Guangxi Manufacturing Systems and Advanced Manufacturing Technology Key Laboratory Director Fund under [Grant No. 21-050-44-S015], and Innovation Project of Guangxi Graduate Education [Grant No. YCSW2023086].

Author information

Authors and Affiliations

  1. Guangxi Key Laboratory of Manufacturing System and Advanced Manufacturing Technology, School of Mechanical Engineering, Guangxi University, Nanning, 530004, Guangxi, China

    Yixin Shen, Deqiang He, Qi Liu, Zhenzhen Jin & Xianwang Li

  2. Nanning Rail Transit Co., Ltd., Nanning, 530029, Guangxi, China

    Chonghui Ren

Authors
  1. Yixin Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Deqiang He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenzhen Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xianwang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chonghui Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yixin Shen and Deqiang He contributed to conceptualization; Yixin Shen contributed to methodology; Yixin Shen and Qi Liu contributed to formal analysis and investigation; Yixin Shen contributed to writing—original draft preparation; Deqiang He, Zhenzhen Jin, and Xianwang Li contributed to writing—review and editing; Deqiang He and Xianwang Li contributed to funding acquisition; Deqiang He and Chonghui Ren contributed to resources; Yixin Shen, Qi Liu, and Zhenzhen Jin contributed to software; Zhenzhen Jin and Chonghui Ren contributed to data Curation.

Corresponding author

Correspondence to Deqiang He.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, Y., He, D., Liu, Q. et al. Unsupervised intrusion detection for rail transit based on anomaly segmentation. SIViP 18, 1079–1087 (2024). https://doi.org/10.1007/s11760-023-02791-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02791-w

Keywords

Search

Navigation