Abstract
The longitudinal tear of conveyor belts is the most common accident occurring at the workplace. Given the limitations on accuracy and stability of current single-modal approaches to detecting the longitudinal tear of conveyor belts, a solution is proposed in this paper through Audio-Visual Fusion. According to this method, a linear CCD camera is used to capture the images of the conveyor belt and a microphone array for the acquisition of sound signals from the operating belt conveyor. Then, the visual data is inputted into an improved Shufflenet_V2 network for classification, while the preprocessed sound signals are subjected to feature extraction and classification using a CNN-LSTM network. Finally, decision fusion is performed in line with Dempster-Shafer theory for image and sound classification. Experimental results show that the method proposed in this paper achieves an accuracy of 97% in tear detection, which is 1.2% and 2.8% higher compared to using images or sound alone, respectively. Apparently, the method proposed in this paper is effective in enhancing the performance of the existing detection methods.



















Similar content being viewed by others
References
Andrejiova, M., Grincova, A., & Marasova, D. (2016). Measurement and simulation of impact wear damage to industrial conveyor belts. Wear, 368, 400–407.
He, D., Pang, Y., & Lodewijks, G. (2017). Green operations of belt conveyors by means of speed control. Applied Energy, 188, 330–341.
Cao, H. (2015). Study and analysis on tear belt and break belt of belt conveyor in coal mine. Coal Science and Technology, 43(S2), 130–134.
Peng, X. (2013). A novel image-based method for conveyor belt rip detection. In IEEE International Conference on Signal Processing.
Zakharov, A., Geike, B., Grigoryev, A., & Zakharova, A. (2020). Analysis of devices to detect longitudinal tear on conveyor belts. In E3S Web of Conferences; EDP Sciences: Kemerovo, Russia, volume 174, p. 03006.
Dobrota, D. (2015). Vulcanization of rubber conveyor belts with metallic insertion using ultrasounds. In Katalinic, B. (Ed.) 25th Daaam International Symposium on Intelligent Manufacturing and Automation, 2014, pp. 1160–1166.
Kozłowski, T., Błażej, R., Jurdziak, L., & Kirjanów-Błażej, A. (2019). Magnetic methods in monitoring changes of the technical condition of splices in steel cord conveyor belts. Engineering Failure Analysis, 104, 462–470.
Kozłowski, T., Wodecki, J., Zimroz, R., Błażej, R., & Hardygóra, M. (2020). A diagnostics of conveyor belt splices. Applied Sciences, 10, 6259.
Yang, Y., Miao, C., Li, X., & Mei, X. (2014). On-line conveyor belts inspection based on machine vision. Optik—International Journal for Light and Electron Optics, 125, 5803–5807.
Qiao, T., Li, X., Pang, Y., Lu, Y., Wang, F., & Jin, B. (2017). Research on conditional characteristics vision real-time detection system for conveyor belt longitudinal tear. IET Science, Measurement & Technology, 11, 11955–11960.
Li, J., & Miao, C. (2016). The conveyor belt longitudinal tear on-line detection based on improved SSR algorithm. Optik, 127(19), 8002–8010.
Xianguo, L., Lifang, S., Zixu, M., Can, Z., & Hangqi, J. (2018). Laser-based on-line machine vision detection for longitudinal rip of conveyor belt. Optik (Stuttg)., 168, 360–369. https://doi.org/10.1016/j.ijleo.2018.04.053
Yang, Y. L., Qiao, T. Z., Pang, T. Z., & Yan, S. (2020). Infrared spectrum analysis method for detection and early warning of longitudinal tear of mine conveyor belt. Measurement, 165, 107856.
Zhang, M., Shi, H., Zhang, Y., Yu, Y., & Zhou, M. (2021). Deep learning-based damage detection of mining conveyor belt. Measurement, 175, 1–9.
Miao, D., Wang, Y., & Li, S. (2022). Sound-based improved DenseNet conveyor belt longitudinal tear detection. IEEE Access, 10, 123801–123808. https://doi.org/10.1109/ACCESS.2022.3224430
Poria, S., Peng, H., Hussain, A., Howard, N., & Cambria, E. (2017). Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis. Neurocomputing, 261, 217–230. https://doi.org/10.1016/j.neucom.2016.09.117
Rahmani, M. H., Almasganj, F., & Seyyedsalehi, S. A. (2018). Audio-visual feature fusion via deep neural networks for automatic speech recognition. Digital Signal Processing, 82, 54–63.
Shrivastava, K., Kumar, S., & Jain, D. K. (2019). An effective approach for emotion detection in multimedia text data using sequence based convolutional neural network. Multimedia Tools And Applications, 78(20), 29607–29639.
Zhang, J., Wen, X., Cho, A., & Whang, M. (2021). An empathy evaluation system using spectrogram image features of audio. Sensors, 21, 7111. https://doi.org/10.3390/s21217111
Reinolds, F., Neto, C., & Machado, J. (2022). Deep learning for activity recognition using audio and video. Electronics, 11, 782. https://doi.org/10.3390/electronics11050782
Liu, Y., Miao, C., & Li, X. (2021). Research on the fault analysis method of belt conveyor idlers based on sound and thermal infrared image features. Measurement, 186, 110177.
de Donato, L., Flammini, F., & Marrone, S. (2022). A survey on audio-video based defect detection through deep learning in railway maintenance. IEEE Access, 10, 65376–65400. https://doi.org/10.1109/ACCESS.2022.3183102
Ma, N. N., Zhang, X. Y., Zheng, H. T., & Sun, J. (2018). ShuffleNet V2: Practical guidelines for efficient CNN architecture design. arXiv:1807. 11164v1 [cs.CV].
Sanghyun, W., Jongchan, P., Joon-Young, L., & Kweon, I. S. CBAM: Convolutional block attention module. arXiv:1807.06521v2 [cs.CV].
Qi, J., Wang, D., Jing, Y., & Liu, R. S. (2013). Auditory features based on Gammatone filters for robust speech recognition. In IEEE International Symposium on Circuits and Systems, pp. 305–308.
Gupta, V., Saxena, N. K., Kanungo, A., et al. (2022). PCA as an effective tool for the detection of R-peaks in an ECG signal processing. International Journal of System Assurance Engineering and Management. https://doi.org/10.1007/s13198-022-01650-0
Zou, L., Xia, L., & Ding, Z. (2019). Reinforcement learning to optimize long-term user engagement in recommender systems: ACM. https://doi.org/10.1145/3292500.3330668[P].
Wang, Y. M., Miao, C. Y., Liu, Y., & Meng, D. J. (2022). Research on a sound-based method for belt conveyor longitudinal tear detection. Measurement, 190, 110787.
Chen, M., & Hernández, A. (2022). Towards an explainable model for sepsis detection based on sensitivity analysis. IRBM, 43(1), 75–86.
Pouard, P., & Collaange, V. (2007). Neuromonitoring par la spectroscopie dans le proche infrarouge en chirurgie cardiaque pédiatrique: Neuromonitoring by near infrared spectroscopy in paediatric cardiac surgery. IRPM, 28, 1959–2318.
Gupta, V., Mittal, M., & Mittal, V. (2022). A novel FrWT based arrhythmia detection in ECG signal using YWARA and PCA. Wireless Personal Communications, 124, 1229–1246.
Gupta, V., Mittal, M., & Mittal, V. (2021). FrWT-PPCA-based R-peak detection for improved management of healthcare system. IETE Journal of Research, 69(8), 5064–5078.
Gupta, A., Gupta, V., Mittal, M., & Mittal, V. (2022). An efficient AR modelling-based electrocardiogram signal analysis for health informatics. International Journal of Medical Engineering and Informatics, 14(1), 74.
Gupta, V., Mittal, M., Mittal, V., et al. (2022). Detection of R-peaks using fractional Fourier transform and principal component analysis. Journal of Ambient Intelligence and Humanized Computing. https://doi.org/10.1007/s12652-021-03484-3
Gupta, V., Mittal, M., & Mittal, V. (2021). Spectrogram as an emerging tool in ECG signal processing. Wireless Personal Communications, 114(4), 0929–6212.
Gupta, V., Mittal, M., & Mittal, V. (2022). A simplistic and novel technique for ECG signal pre-processing. IETE Journal of Research. https://doi.org/10.1080/03772063.2022.2135622
Ebad, S. A. (2022). Lessons learned from offline assessment of security-critical systems: The case of microsoft’s active directory. International Journal of System Assurance Engineering and Management. https://doi.org/10.1007/s13198-021-01236-2
Amanbek, N., Mamayeva, L. A., & Rakhimzhanova, G. M. (2021). Results of a comprehensive assessment of the quality of services to the population with the use of statistical methods. International Journal of Systems Assurance Engineering and Management. https://doi.org/10.1007/s13198-021-01278
Alketbi, A., Nasir, Q., & Abu, T. (2020). Novel blockchain reference model for government services: Dubai government case study. International Journal of System Assurance Engineering and Management, 11(6), 1170–1191.
Gupta, S., Gupta, P., & Parida, A. (2017). Modeling lean maintenance metric using incidence matrix approach. International Journal of System Assurance Engineering and Management, 8(4), 799–816.
Ye, W., Wang, H., & Zhong, Y. (2022). Optimization of network security protection situation based on data clustering. International Journal of System Assurance Engineering and Management. https://doi.org/10.1007/s13198-021-01529-6
Xu, Q., Wu, D., Jiang, C., et al. (2022). A composite quantile regression long short-term memory network with group lasso for wind turbine anomaly detection. Journal of Ambient Intelligence and Humanized Computing, 14(3), 2261–2274. https://doi.org/10.1007/s12652-022-04484-7
Son, Y., Zhang, X., Yoon, Y., et al. (2022). LSTM–GAN based cloud movement prediction in satellite images for PV forecast. Journal of Ambient Intelligence and Humanized Computing, 14(9), 12373–12386. https://doi.org/10.1007/s12652-022-04333-7
Gundu, V., & Simon, S. P. (2021). PSO–LSTM for short term forecast of heterogeneous time series electricity price signals. Journal of Ambient Intelligence and Humanized Computing, 12(2), 2375–2385. https://doi.org/10.1007/s12652-020-02353-9
Reznikov, I., Chuprakov, D., & Bekerov, I. (2023). Analytical model of 2D leakoff in waterflood-induced fractures. Journal of Rock Mechanics and Geotechnical Engineering, 15(7), 1713–1733.
Zeng, L., Zhang, H., Han, Q., et al. (2021). An LSTM-based driving operation suggestion method for riding comfort-oriented critical zone. Journal of Ambient Intelligence and Humanized Computing. https://doi.org/10.1007/s12652-021-03327-1
Ubaid, A. M., & Dweiri, F. T. (2020). Business process management (BPM): Terminologies and methodologies unified. International Journal of System Assurance Engineering and Management, 11, 1046–1064.
Acknowledgements
This work was supported by Science and technology think tank youth talent plan (Grant No. 20220615ZZ07110010); National Natural Science Foundation of China-Shanxi coal-based low-carbon joint fund (Grant No. U1810121) and the Natural Science Foundation of Shanxi (Grant: No. 201801D121180).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, Y., Du, Y., Miao, C. et al. Longitudinal tear detection method for conveyor belt based on multi-mode fusion. Wireless Netw 30, 2839–2854 (2024). https://doi.org/10.1007/s11276-024-03693-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11276-024-03693-6