Skip to main content
SpringerLink
Log in

Real-time detection and location of reserved anchor hole in coal mine roadway support steel belt

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Based on the current coal mine roadway using supporting steel belt steel belt auxiliary anchor support, the location of the supporting steel belt anchor holes is mainly done manually; if the location is not accurate, there will be large safety hazards and other problems. An intelligent real-time detection and location method of anchor holes in coal mine roadway support steel belt based on deep learning model and depth camera is proposed. First, to reduce the influence of water mist and dust on the camera and improve the image quality of the camera, the image is pre-processed using contrast limited adaptive histogram equalization. Second, the YOLOv5s model is improved by adding SPD-Conv and coordinate attention mechanisms to improve the detection capability of the model. Third, a real-time depth map restoration method that fully preserves object edge features is proposed to avoid errors caused by areas with depth values of 0 in the depth map when locating anchor holes in combination with a depth camera. Finally, the improved YOLOv5s model proposed in this paper combined with the repaired depth map was used to achieve the detection and location of anchor holes in a laboratory simulated tunnel with a location error of less than 5 mm and an average FPS of 28.9. In summary, the real-time target detection and location method based on a deep learning model and a depth camera is feasible in an unstructured environment in coal mines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Kang, H.: Support technologies for deep and complex roadways in underground coal mines: a review. J. Int. J. Coal Sci. Technol. 1(3), 261–277 (2014)

    Article  Google Scholar 

  2. Jamal, R., Sair, K., Ali, N., et al.: Rock characterization while drilling and application of roof bolter drilling data for evaluation of ground conditions. J. Rock Mech. Geotech. Eng. 7(3), 273–281 (2015)

    Article  Google Scholar 

  3. Meng, Q., Han, L., Sun, J., et al.: Experimental study on the bolt–cable combined supporting technology for the extraction roadways in weakly cemented strata. J. Int. J. Mining Sci. Technol. 25(1), 113–119 (2015)

    Article  Google Scholar 

  4. Zhang, Y., Shao, J., Sun, D., et al.: Study on location parameter adjusting system for anchor drilling in coal mine roadways. In: Proceedings of 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC 2018). Chongqing, China. pp. 1159-1162 (2018)

  5. Liu, Q., Zha, Y., Liu, T., et al.: Research on adaptive control of air-borne bolting rigs based on genetic algorithm optimization. J. Mach. 9(10), 240 (2021)

    Google Scholar 

  6. Pan, H.G., Shi, Y.H., Lei, X.Y., et al.: Fast identification model for coal and gangue based on the improved tiny YOLO v3. J. Real-Time Image Proc. 19(3), 687–701 (2022)

    Article  Google Scholar 

  7. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. J. Commun. ACM. 60(6), 84–90 (2017)

    Article  Google Scholar 

  8. Yang, S.D., Chen, Z.H., Ma, X.M., et al.: Real-time high-precision pedestrian tracking: a detection-tracking-correction strategy based on improved SSD and Cascade R-CNN. J. Real-Time Image Proc. 19(2), 287–302 (2021)

    Article  Google Scholar 

  9. Girshick, R., Donahue, J., Darrell, T., et al.: Rich feature hierarchies for accurate object detection and semantic segmenta-tion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA; pp. 580–587 (2014)

  10. Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, pp. 1440–1448 (2015)

  11. Ren, S., He, K., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada, 7–12 December; Cor-tes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R. (eds.). Curran Associates, Inc.: Red Hook, NY, USA, Volume 28 (2015)

  12. He, K., Gkioxari, G., Dollár, P., et al. Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, pp. 2961–2969 (2017)

  13. Liu, W., Anguelov, D., Erhan, D., et al.: Ssd: single shot multibox detector. In: Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, pp. 21–37 (2016)

  14. Redmon, J., Divvala, S., Girshick, R., et al.: You only look once: unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 779–788 (2016)

  15. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition IEEE, Honolulu, HI, USA, pp. 6517–6525 (2017)

  16. Redmon, J., Farhadi, A.: YOLOv3: An incremental improvement. In: Proceedings of the CVPR 2018: IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake, UT, USA, 18–22 (2018)

  17. Bochkovskiy, A., Wang, C.Y., Liao, H.: YOLOv4: optimal speed and accuracy of object detection. Available online: https://arxiv.org/abs/2004.10934. Accessed 3 Dec 2022

  18. Ultralytics. YOLOv5 in PyTorch. 2021. Available online: https://github.com/ultralytics/yolov5. Accessed 3 Dec 2022

  19. Huang, L., Yang, Y., Deng, Y., et al.: Densebox: unifying landmark localization with end to end object detection. arXiv preprint arXiv:1509.04874 (2015) (Accessed on 3 Dec 2022).

  20. Guo, X., Liu, X., Zhou, H., Stanislawski, R., Królczyk, G., Li, Z.: Belt tear detection for coal mining conveyors. J. Micromach. 13(3), 449 (2022)

    Article  Google Scholar 

  21. Zhang, M., Shi, H., Zhang, Y., et al.: Deep learning-based damage detection of mining conveyor belt. J. Measurement. 175, 109130 (2021)

    Article  Google Scholar 

  22. Li, M., Duan, Y., He, X., et al.: Image positioning and identification method and system for coal and gangue sorting robot. J. Int. J. Coal Prep. Util. 42(6), 1759–1777 (2022)

    Article  Google Scholar 

  23. Cao, X.G., Fei, J.H., Wang, P.: Study on coal-gangue sorting method based on multi-manipulator collaboration. J. Coal Sci. Technol. 47(4), 7–12 (2019)

    Google Scholar 

  24. Chi, H., Gong, B.: Development and application of underground intelligent explosive vehicles. 2019 2nd World Conference on Mechanical Engineering and Intelligent Manufacturing (WCMEIM), Shanghai, China, pp. 635–638 (2019)

  25. Bonchis, A., Duff, E., Roberts, J., et al.: Robotic explosive charging in mining and construction applications. Autom. Sci. Eng. IEEE Trans. IEEE 11(1), 245–250 (2014)

    Article  Google Scholar 

  26. Reza, A.M.: Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. J. VLSI Signal Process. Syst. Signal Image Video Technol. 38, 35–44 (2004)

    Article  Google Scholar 

  27. Sunkara, R., Luo, T.: No more strided convolutions or pooling: A new CNN building block for low-resolution images and small objects. arXiv preprint arXiv:2208.03641 (2022). Accessed 1 Oct 2022

  28. Fukui, H., Hirakawa, T., Yamashita, T., et al.: Attention branch network: Learning of attention mechanism for visual explanation, in Proc, pp. 10705–10714. CVPR, Long Beach, CA, USA (2019)

    Google Scholar 

  29. Liu, G., Guo, J.: Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing 337, 325–338 (2019)

    Article  Google Scholar 

  30. Hou, Q., Zhou, D., Feng, J.: Coordinate attention for efficient mobile network design. arXiv preprint, arXiv:2103.02907 (2021) (Accessed on 8 May 2022).

  31. Wang, D., Li, W., Liu, X., et al.: UAV environmental perception and autonomous obstacle avoidance: A deep learning and depth camera combined solution. J. Comput. Electron. Agric. 175, 105523 (2020)

    Article  Google Scholar 

  32. Liu, Z., Shi, S., Duan, Q., et al.: Salient object detection for RGB-D image by single stream recurrent convolution neural network. J. Neurocomput. 363, 46–57 (2019)

    Article  Google Scholar 

  33. Nealen, A., Igarashi, T., Sorkine, O., et al.: Fibermesh: designing freeform surfaces with 3d curves. In ACM SIGGRAPH 2007 Papers, SIGGRAPH ’07, New York, NY, USA. ACM (2007).

  34. Jaritz, M., de Charette, R., Wirbel, E., et al.: Sparse and dense data with CNNs: depth completion and semantic segmentation. In: 2018 International Conference On 3D Vision (3DV), pp. 52–60. Verona, Italy (2018)

  35. Kopf, J., Cohen, M.F., Lischinski, D., et al.: Joint bilateral upsampling. ACM Trans. Gr. 26(3), 96–99 (2007)

    Article  Google Scholar 

  36. Tomasi C, Manduchi R.: Bilateral filtering for gray and color images. The 6th International Conference on Computer Vision, Washington DC, USA, 839–846 (1998)

Download references

Acknowledgements

This research was funded by National Key Research and Development Program of China (Grant number 2020YFB1314004), Bidding Project of Shanxi Province (Grant number 20201101008), National Key Research and Development Program of Shanxi Province (Grant number 202102100401015).

Author information

Authors and Affiliations

  1. College of Mechanical and Vehicle Engineering, Taiyuan University of Technology, Taiyuan, 030024, China

    Hongwei Wang

  2. College of Safety and Emergency Management Engineering, Taiyuan University of Technology, Taiyuan, 030024, China

    Haoran Wang

  3. College of Mining Engineering, Taiyuan University of Technology, Taiyuan, 030024, China

    Fujing Zhang, Zhenglong Li & Yuheng Wang

  4. Shanxi Engineering Research Center for Coal Mine Intelligent Equipment, Taiyuan University of Technology, Taiyuan, 030024, China

    Hongwei Wang, Fujing Zhang, Haoran Wang, Zhenglong Li & Yuheng Wang

  5. Postdoctoral Workstation, Shanxi Coking Coal Group Co., Ltd, Taiyuan, 030024, China

    Hongwei Wang

Authors
  1. Hongwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fujing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haoran Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenglong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: HW, FZ, HW, ZL and YW; methodology: HW and FZ; software: HW and FZ; validation: HW, FZ, HW, ZL, and YW; formal analysis: HW and FZ; investiga-tion: HW and FZ; resources: HW; data curation: HW and FZ; writing original draft prep-aration: HW and FZ; writing—review and editing: HW, FZ, HW, ZL and YW; visualiza-tion: HW and FZ; supervision: HW and FZ; project administration: HW; funding acquisition: HW. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Fujing Zhang.

Ethics declarations

Conflict of interest

The authors have no conflict of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Zhang, F., Wang, H. et al. Real-time detection and location of reserved anchor hole in coal mine roadway support steel belt. J Real-Time Image Proc 20, 89 (2023). https://doi.org/10.1007/s11554-023-01347-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-023-01347-y

Keywords

Search

Navigation