Skip to main content
SpringerLink
Log in

Improved small foreign object debris detection network based on YOLOv5

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In response to the challenges of detecting foreign object debris (FOD) on airport runways, where the objects are small in size and have indistinct features leading to false detections and missed detections, significant improvements were made to the YOLOv5 algorithm. First, the original YOLOv5-n model was optimized by incorporating multi-scale fusion and detection enhancements. To improve detection speed and reduce parameters, the detection head for large objects was removed. Second, the C3 module in the backbone network was replaced with the C2f module, resulting in enhanced gradient flow and improved feature representation capabilities. Additionally, the spatial pyramid pooling-fast (SPPF) module in the backbone network was refined to expand the receptive field and enhance the model’s perception of dependencies between targets and backgrounds. Furthermore, the coordinate attention (CA) mechanism was introduced in the neck layer to further enhance the model's perception of small FOD items. Lastly, the SCYLLA-IoU (SIoU) loss function was introduced to further improve the speed and accuracy of bounding box regression. Moreover, the nearest neighbor interpolation upsampling method was substituted with the lightweight Content-Aware ReAssembly of FEatures (CARAFE) upsampling operator to better exploit global information. Experimental results on the Fod_Tiny dataset, which consists of small FOD items in airports, demonstrated a significant 5.4% improvement over the baseline algorithm. To validate the generalizability of the algorithm, experiments were conducted on the Mirco_COCO dataset, resulting in a notable 1.9% improvement compared to the baseline algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Chen, W., Xu, Q., Ning, H., Wang, T., Li, J.: Foreign object debris surveilla-nce network for runway security. Aircraft Eng Aerospace Technol 83, 229–234 (2018). https://doi.org/10.1108/00022661111138648

    Article  Google Scholar 

  2. FAA.: AC 150/5220–24—foreign object debris detection equipment. 1–13 (2009) https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_150_5220-24.pdf

  3. Chen, J., Tang, X., Ji, X.: Multi-feature fusion for airport FOD detection. In: CICTP, pp. 198–208 (2020) https://doi.org/10.1061/9780784483053.017

  4. Maimunah M. A., Norhashila H.: Visible light imaging. In: Electromagnetic technologies in food science, pp. 337–362 (2021) https://doi.org/10.1002/9781119759522.ch14

  5. Wang, Y., Huang, H., Wang, J., Wang, P., Song, Q.: An image denoising method for arc-scanning SAR for airport runway foreign object debris detection. Electronics 12(4), 984 (2023). https://doi.org/10.3390/electronics12040984

    Article  Google Scholar 

  6. Noroozi, M., Shah, A.: Towards optimal foreign object debris detection in an airport environment. Expert Syst. Appl. (2023). https://doi.org/10.1016/j.eswa.2022.118829

    Article  Google Scholar 

  7. Chung, W.Y., Lee, I.H., Park, C.G.: Lightweight infrared small tar-get detection network using full-scale skip connection U-net. IEEE Geosci. Remote Sens. Lett. (2023). https://doi.org/10.1109/LGRS.2023.3276326

    Article  Google Scholar 

  8. Gong, Y., Zhang, Z., Wen, J., Lan, G., Xiao, S.: Small ship detection of SAR images based on optimized feature pyramid and sample augmentation. IEEE J Selected Topics Appl Earth Obs Remote Sens (2023). https://doi.org/10.1109/JSTARS.2023.3302575

    Article  Google Scholar 

  9. Wang, S., Wang, Y., Chang, Y., Zhao, R., She, Y.: EBSE-YOLO: high precision recognition algorithm for small target foreign object detection. IEEE Access (2023). https://doi.org/10.1109/ACCESS.2023.3284062

    Article  Google Scholar 

  10. Yang, L., et al.: An improving faster-RCNN with multi-attention ResN-et for small target detection in intelligent autonomous transport with 6G. IEEE Trans. Intell. Transp. Syst. (2023). https://doi.org/10.1109/TITS.2022.3193909

    Article  Google Scholar 

  11. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE conference on computer vision and pattern recognition. Las Vegas: IEEE, pp. 779–788 (2016) https://doi.org/10.1109/CVPR.2016.91

  12. Wei, L., Dragomir, A., Dumitru, E., et al.: SSD: single shot multibox detector. In: Computer vision-ECCV, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  13. Carion, N., Massa, F., Symnaeve, G., et al.: End-to-End object detection with transformers. In: European conference on computer vision, pp. 213–229 (2020) https://doi.org/10.1007/978-3-030-58452-8_13

  14. Lin, T.Y., Goyal, P., Girshick, R., et al.: Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell. (2017). https://doi.org/10.1109/TPAMI.2018.2858826

    Article  PubMed  Google Scholar 

  15. Ren, S.Q., He, K.M., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. (2017). https://doi.org/10.1109/TPAMI.2016.2577031

    Article  PubMed  Google Scholar 

  16. He, K., Gkioxari, G., Dollár, P., Girshick, R. et al.: Mask R-CNN.In: IEEE international conference on computer vision. Venice: IEEE, pp. 2980–2988 (2017) https://doi.org/10.1109/ICCV.2017.322

  17. Wang, C.Y., Bochkovskiy, A., Liao, H.Y.: YOLOv7: trainable bag-of-fre-ebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 7464–7475 (2023) https://doi.org/10.1109/CVPR52729.2023.00721

  18. Ultralytics.: YOLOv8. Github code (2023) https://github.com/ultralytics/ultralytics

  19. Chen, X., Fang, H., Lin, T. Y., et al.: Microsoft COCO captions: data collection and evaluation server. arxiv preprint (2015) https://doi.org/10.48550/arXiv.1504.00325

  20. Ultralytics.: YOLOv5. Github code (2020) https://github.com/ultralytics/YOLOv5

  21. Yu, Z., Huang, H., Chen, W., Su, Y., Liu, Y., Wang X.: YOLO-FaceV2: a scale and occlusion aware face detector. arxiv preprint (2022) https://doi.org/10.48550/arXiv.2208.02019

  22. Yang, L., Zhang, R,Y., Li, L., Xie, X.: SimAM : a simple, parameter-free attention module for convolutional neural networks. In: International conference on machine learning, pp. 11863–11874 (2021)

  23. Hou, Q., Zhou, D., Feng, J.: Coordinate attention for efficient mobile network design. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Nashville, TN, USA, pp. 13708–13717(2021) https://doi.org/10.1109/CVPR46437.2021.01350

  24. Gevorgyan, Z.: Slou loss: more powerful learning for bounding box regression. arxiv preprint (2022) https://doi.org/10.48550/arXiv.2205.12740

  25. Zheng, Z., Wang, P., Liu, W., et al.: Distance-loU loss: faster and better learning for bounding box regression. In: Proceedings of the AAAl conference on artificial intelligence. 34(07), 12993–13000 (2020) https://doi.org/10.1609/aaai.v34i07.6999

  26. Wang, J., Chen, K., Xu, R., et al.: CARAFE: content-aware reassembly of features. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 3007–3016(2019) https://doi.org/10.1109/ICCV.2019.00310

  27. Wan, Y., Liang, X., Bu, X., Liu, Y.: FOD detection method based on iterative adaptive approach for millimeter-wave radar. Sensors 21(4), 1241–1257 (2021). https://doi.org/10.3390/s21041241

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  28. Qin, F., Bu, X., Liu, Y., Liang, X., Xin, J.: Foreign object debris automatic target detection for millimeter-wave surveillance radar. Sensors 21(11), 3853 (2021). https://doi.org/10.3390/s21113853

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  29. Xiaojing, G., Haoda, S.: Application of improved yolov3 in foreign object detection of airport runway. Comput. Eng. Appl. 57(08), 249–255 (2021). https://doi.org/10.3778/j.issn.1002-8331.2007-0173

    Article  Google Scholar 

  30. Cao, Y et al.: FOD detection using a multi-channel information fusion method. In: 2022 12th international conference on CYBER technology in automation, control, and intelligent systems (CYBER), Baishan, China, pp. 785–790(2022) https://doi.org/10.1109/CYBER55403.2022.9907675

  31. Munyer., Travis, J.E. et al.: Foreign Object Debris Detection For Airport Pavement Images Based On Self-Supervised Localization And Vision Transformer. arxiv preprint (2022) https://doi.org/10.48550/arXiv.2210.16901

  32. Bo, Y., Qiuru, W.: Small target foreign object detection based on improved YOLO network. In: 2022 11th international conference of information and communication technology (ICTech), Wuhan, China, pp. 431–435 (2022) https://doi.org/10.1109/ICTech55460.2022.00092

  33. Taupik, J., Alamsyah, T., Wulandari, A., Armin, E.U. Hikmatur-okhman, A.: Airport runway foreign object debris (FOD) detection based on YOLOX architecture. In: 2023 international conference on computer science, information technology and engineering (ICCoSITE), Jakarta, Indonesia, pp. 40–43(2023) https://doi.org/10.1109/ICCoSITE57641.2023.10127676

  34. Zhang, H., Fu, W., Shao, J., Li, D., Wang, X.: Airport foreign object small target detection dataset. In: 2023 IEEE 7th information technology and mechatronics engineering conference (ITOEC), pp. 1495–1499(2023) https://doi.org/10.1109/ITOEC57671.2023.10291472

  35. Zhu, L., Wang, X., Ke, Z., Zhang, W., Lau, R.W.H.: BiFormer: vision transformer with bi-level routing attention. In: 2023 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 10323–10333 (2023) https://doi.org/10.1109/CVPR52729.2023.00995

  36. Li, Y., Yao, T., Pan, Y., Mei, T.: Contextual transformer networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 45(2), 1489–1500 (2022). https://doi.org/10.1109/TPAMI.2022.3164083

    Article  Google Scholar 

  37. Ouyang, D., He, S., Zhang, G., Luo, M., Guo, H., Zhan, J., Huang, Z.: Efficient multi-scale attention module with cross-spatial learning. In: ICASSP 2023–2023 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 1–5 (2023) https://doi.org/10.1109/ICASSP49357.2023.10096516

  38. Liu, Y., Shao, Z., Hoffmann, N.: Global attention mechanism: retain information to enhance channel-spatial interactions. arxiv preprint (2021) https://doi.org/10.48550/arXiv.2112.05561

  39. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., Hu, Q.: ECA-net: efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Seattle, WA, USA, pp. 11531–11539(2020) https://doi.org/10.1109/CVPR42600.2020.01155

  40. Li, X., Wang, W., Hu, X., Yang, J.: Selective Kernel networks. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Long Beach, CA, USA, pp. 510–519(2019. https://doi.org/10.1109/CVPR.2019.00060

  41. Rezatofighi, S.H., Tsoi, N., Gwak, J.Y., Sadeghian, A., Reid, I.D., Savarese, S.: Generalized intersection over union: a metric and a loss for bounding box regression. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 658–666 (2019) https://doi.org/10.1109/CVPR.2019.00075

  42. Zhang, Y.F., Ren, W., Zhang, Z., Jia, Z., Wang, L., Tan, T.: Focal and efficient IOU loss for accurate bounding box regression. Neurocomputing (2022). https://doi.org/10.1016/j.neucom.2022.07.042

    Article  PubMed  PubMed Central  Google Scholar 

  43. Tong, Z., Chen, Y., Xu, Z., Yu, R.: Wise-IoU: bounding box regression loss with dynamic focusing mechanism. arxiv preprint (2023). https://doi.org/10.48550/arXiv.2301.10051

  44. Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arxiv preprint (2018). https://doi.org/10.48550/arXiv.1804.02767

  45. Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., Nie, W., Li, Y.: YOLOv6: a single-stage object detection framework for industrial applications. arxiv preprint (2022). https://doi.org/10.48550/arXiv.2209.02976

  46. Ge, Z., et al.: Yolox: exceeding yolo series in 2021. arxiv preprint (2021) https://doi.org/10.48550/arXiv.2107.08430

  47. Lu, H., Liu, T., Zhang, J.: Hybrid attention module based on YOLOv5 for foreign object debris detection. In: International conference in communications, signal processing, and systems, pp. 266–272 (2022) https://doi.org/10.1007/978-981-99-2362-5_33

  48. Zhu, X., Lyu, S., Wang, X., Zhao, Q.: TPH-YOLOv5: improved YO-LOv5 based on transformer prediction head for object detection on drone-captured scenarios. In: 2021 IEEE/CVF international conference on computer vision workshops (ICCVW), pp. 2778–2788(2021) https://doi.org/10.1109/ICCVW54120.2021.00312

  49. Tian, Z., Shen, C., Chen, H., He, T.: FCOS: Fully convolutional one-stage object detection. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp. 9626–9635(2019) https://doi.org/10.1109/ICCV.2019.00972

  50. Munyer, T., Huang, P.C., Huang, C., et al.: FOD-A: a dataset for foreign object debris in airports. arxiv preprint (2021) https://doi.org/10.48550/arXiv.2110.03072

  51. Wang, J., Yang, W., Guo, H., Zhang, R., Xia, G.S.: Tiny object detection in aerial images. In: 2020 25th international conference on pattern recognition (ICPR), pp. 3791–3798 (2020) https://doi.org/10.1109/ICPR48806.2021.9413340

Download references

Author information

Authors and Affiliations

  1. School of Computer and Software Engineering, Xihua University, Chengdu, China

    Heng Zhang, Dong Li, Xiaoming Wang & Tengda Xu

  2. College of Chinese ASEAN Arts, Chengdu University, Chengdu, China

    Wei Fu

Authors
  1. Heng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tengda Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HZ was primarily responsible for the conceptualization of the manuscript, collection and creation of the Fod_Tiny dataset, execution of code experiments, and the writing and revision of the manuscript. WF was mainly responsible for reviewing and proofreading the paper. DL provided guidance throughout the research process. XW provided guidance in the initial stages of the manuscript. TX was responsible for the collection and creation of the Fod_Tiny dataset.

Corresponding author

Correspondence to Heng Zhang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Fu, W., Li, D. et al. Improved small foreign object debris detection network based on YOLOv5. J Real-Time Image Proc 21, 21 (2024). https://doi.org/10.1007/s11554-023-01399-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-023-01399-0

Keywords

Search

Navigation