Skip to main content
SpringerLink
Log in

An adaptive focused target feature fusion network for detection of foreign bodies in coal flow

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In the process of conveying raw coal to the surface on conveyor belts, the raw coal is generally blended with foreign bodies, such as large pieces of gangue and damaged bolts, which can affect the quality of mined coal, damage the transportation equipment and even jam the coal conduit, seriously reducing the coal conveying efficiency. To handle to the existing problems of underground complex environment, low detection accuracy and poor real-time performance in coal flow foreign bodies detection, we propose an adaptive focused target feature fusion network (AFFNet) based on YOLOX. The multi-transformer parallel (MTRP) module is used to expand the receptive field and fuse the features under different receptive fields with the transformer encoder to enhance the feature extraction ability. The cross stage partial transformer (CSPTR) with transformer encoder module is designed to capture the global context information of the feature maps in the network, and improve the location prediction in the detection. In the feature fusion channel of different scales, the learnable weight parameters are added to learn the spatial weight of feature map fusion adaptively, and the feature expression ability of different scales is optimized. SCYLLA-IoU (SIoU) loss and varifocal loss are used to obtain more accurate bounding boxes and deal with the sample category imbalance problem, respectively. The experimental results show that AFFNet can achieve a detection speed of 48 frame per second (FPS) and a mean average precision (mAP50) of 95.6%, 6.7% higher than YOLOX-s on the dataset of foreign body in coal flow. It can balance both the detection speed and detection accuracy, and can be used to improve the efficiency of detecting foreign bodies in the coal flow.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data is not available.

References

  1. Sun Y-Q, Xiao K, Wang X-D, Lv Z-H, Mao M (2021) Evaluating the distribution and potential ecological risks of heavy metal in coal gangue. Environ Sci Pollut Res 28(15):18604–18615

    Article  Google Scholar 

  2. Li D-Y, Wang G-F, Zhang Y, Wang S (2022) Coal gangue detection and recognition algorithm based on deformable convolution yolov3. IET Image Process 16(1):134–144

    Article  Google Scholar 

  3. Shang D, Zhang L, Niu Y (2022) Design and key technology analysis of coal gangue sorting robot. Coal Sci Technol 50(3):7

    Google Scholar 

  4. Zhao Y, Yang X, Luo Z, Duan C, Song S (2014) Progress in developments of dry coal beneficiation. Int J Coal Sci Technol 1(1):103–112

    Article  Google Scholar 

  5. Guofa W, Feng L, Xiangjun M, Jingdao F, Huaiwei R (2019) Research and practice on intelligent coal mine construction (primary stage). Coal Sci Technol 47(8):1–36

    Google Scholar 

  6. Luo Y, Zhang B, Lang LL (2021) Technology and realization of coal gangue recognition based on deep learning. Coal Sci Technol 49(12):7

    Google Scholar 

  7. Ge Z, Liu S, Wang F, Li Z, Sun J (2021 ) Yolox: exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430

  8. Liu S, Huang D, Wang Y (2019) Learning spatial fusion for single-shot object detection. arXiv preprint arXiv:1911.09516

  9. Zhang H, Wang Y, Dayoub F, Sünderhauf N (2020) Varifocalnet: an iou-aware dense object detector. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  10. Gevorgyan Z (2022) Siou loss: More powerful learning for bounding box regression. arXiv preprint arXiv:2205.12740

  11. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly SA (2021) An image is worth 16 \(\times\) 16 words: transformers for image recognition at scale. In: International conference on learning representations

  12. Tao Y, Jun Z, Zhi-hao Z, Yi Z, Fu-qiang Z, Xiao-zhi G (2022) Fault detection of train mechanical parts using multi-mode aggregation feature enhanced convolution neural network. Int J Mach Learn Cybern 13(6):1781–1794

    Article  Google Scholar 

  13. Cheng B, Wei Y, Shi H, Feris R, Xiong J, Huang T (2018) Revisiting rcnn: on awakening the classification power of faster rcnn. In: 2018 European conference on computer vision (ECCV)

  14. Ren S, He K, Girshick R, Sun J (2016) Faster r-cnn: towards real-time object detection with region proposal networks. In: NIPS

  15. Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR)

  16. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2015) Ssd: single shot multibox detector

  17. Jiang P, Ergu D, Liu F, Cai Y, Ma B (2022) A review of yolo algorithm developments. Procedia Comput Sci 199:1066–1073

    Article  Google Scholar 

  18. Li C, Li L, Jiang H, Weng K, Geng Y, Li L, Ke Z, Li Q, Cheng M, Nie W et al (2022) Yolov6: a single-stage object detection framework for industrial applications. arXiv preprint arXiv:2209.02976 (2022)

  19. Wang C-Y, Bochkovskiy A, Liao H-YM (2022) Yolov7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696

  20. Zou X, Wu Z, Zhou W, Huang J (2022) Yolox-pai: an improved yolox version by pai. arXiv preprint arXiv:2208.13040

  21. Guo Y, Zhongshan YU, Yichang LU (2019) Research on photoelectric intelligent separation technology of coal and gangue based on np-fsvm with the pso algorithm. Coal Sci Technol 47(4):13–19

    Google Scholar 

  22. Shang D, Wang Y, Yang Z, Wang J, Liu Y (2020) Study on comprehensive calibration and image sieving for coal-gangue separation parallel robot. Appl Sci 10(20):7059

    Article  Google Scholar 

  23. Li M, Duan Y, He X, Yang M (2020) Image positioning and identification method and system for coal and gangue sorting robot. Int J Coal Prep Util 4:1–19

    Google Scholar 

  24. Fan Z, Chen N, Huang Y, Zhang L, Li Y (2021) Coal gangue recognition based on support vector machine and multiple features. J Jinan Univ (Nat Sci Ed) 19(3):13–18

    Google Scholar 

  25. Sun Z, Huang L, Jia R (2021) Coal and gangue separating robot system based on computer vision. Sensors 21(4):1349

    Article  Google Scholar 

  26. Han C, Zhu X, Fu L (2021) Improve the recognition of coal gangue based on the fusion of spatial channel attention and residual. J Xi’an Univ Sci Technol 6:1113–1121

    Google Scholar 

  27. Lv Z, Wang W, Xu Z, Zhang K, Lv H (2020) Cascade network for detection of coal and gangue in the production context. Powder Technol 377:361–371

    Article  Google Scholar 

  28. Liu Q, Li J, Li Y, Gao M (2021) Recognition methods for coal and coal gangue based on deep learning. IEEE Access PP(99):1–1

    Article  Google Scholar 

  29. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  30. Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR)

  31. Jin Z, Yu D, Song L, Yuan Z, Yu L (2022) You should look at all objects. arXiv preprint arXiv:2207.07889

  32. Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell PP(99):2999–3007

    Google Scholar 

  33. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767

  34. Zhu X, Lyu S, Wang X, Zhao Q (2021) Tph-yolov5: improved yolov5 based on transformer prediction head for object detection on drone-captured scenarios. In: 2021 IEEE/CVF international conference on computer vision (ICCV)

  35. Tian Z, Shen C, Chen H, He T (2020) Fcos: fully convolutional one-stage object detection. In: 2019 IEEE/CVF international conference on computer vision (ICCV)

Download references

Acknowledgements

We thank China University of Mining and Technology(Beijing) for providing the experimental hardware platform.This work was supported by the State Key Laboratory of Coal Mining and Clean Utilization, China (2021-CMCU-KF012), National Science Foundation of China (52121003), and Fundamental Research Funds for the Central Universities (2022YJSJD01 and Grant 2022YQJD04).

Author information

Authors and Affiliations

  1. School of Mechanical Electronic and Information Engineering, China University of Mining and Technology(Beijing), Beijing, 100083, China

    Tao Ye, Zhikang Zheng, Yunwang Li, Xi Zhang, Xiangpeng Deng, Yu Ouyang & Zongyang Zhao

  2. State Key Laboratory of Coal Mining and Clean Utilization, Beijing, 100083, China

    Tao Ye, Zhikang Zheng, Xiangpeng Deng, Yu Ouyang & Zongyang Zhao

  3. Key Laboratory of Intelligent Mining and Robotics, Ministry of Emergency Management, Beijing, 100083, China

    Tao Ye, Zhikang Zheng, Yunwang Li, Xi Zhang, Xiangpeng Deng, Yu Ouyang & Zongyang Zhao

  4. School of Computing, University of Eastern Finland, Kuopio, Finland

    Xiaozhi Gao

Authors
  1. Tao Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhikang Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yunwang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiangpeng Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yu Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zongyang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xiaozhi Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Ye.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ye, T., Zheng, Z., Li, Y. et al. An adaptive focused target feature fusion network for detection of foreign bodies in coal flow. Int. J. Mach. Learn. & Cyber. 14, 2777–2791 (2023). https://doi.org/10.1007/s13042-023-01798-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01798-6

Keywords

Search

Navigation