Skip to main content
SpringerLink
Log in

An improved U-Net-based network for multiclass segmentation and category ratio statistics of ore images

  • Application of soft computing
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The ratio of various types of ores on a conveyor belt indicates how well the ores are blended, which can play a crucial role in determining the quality of steel. However, the conventional method of artificial naked eye detection based on experience is risky, time-consuming, and inaccurate. To address this issue, this study proposes an automatic ore category percentage statistics method based on multiclass semantic segmentation. The first step involves creating a clean dataset comprising five different categories of ores collected at a mine site. Subsequently, a novel U-Net-based network is introduced to segment the ore images. Specifically, a pre-trained VGG16 is adopted to replace the original backbone to enhance the feature extraction ability. A pyramid pooling module is also simplified to effectively aggregate the multiscale information. In addition, a W-shaped architecture, which bridges feature maps between the first and second encoders through addition, is utilized to generate the final probability map in a predict-refine manner. The experimental results demonstrate that the proposed model achieves a pixel-wise accuracy of 92.45% and a mean intersection over union of 72.42%, outperforming ten other state-of-the-art segmentation algorithms. Furthermore, the proposed method is applied to predict images of unprocessed ores from a mine site to determine their category proportions, thus providing valuable guidance for optimizing the ore blending process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Algorithm 1
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Availability of data and materials

The dataset generated during this study is available in the GitHub repository, https://github.com/wangweiustb/W_Ore_Net.

References

  • Azhari F, Sennersten CC, Lindley CA, et al (2023) Deep learning implementations in mining applications: a compact critical review. Artif Intell Rev, pp 1–36

  • Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  PubMed  Google Scholar 

  • Baldeon CM, Lai YS (2018) Res-Unet: residual convolutional neural network for prostate MRI segmentation. In: IIE Annual Conference Proceedings, Institute of Industrial and Systems Engineers (IISE), pp 731–736

  • Bochkovskiy A, Wang CY, Liao HYM (2020) YOLOv4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934

  • Chen L, Papandreou G, Kokkinos I, et al (2014) Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv preprint arXiv:1412.7062

  • Chen L, Zhu Y, Papandreou G, et al (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 801–818

  • Chen W, Zhang Y, He J, et al (2019) Prostate segmentation using 2D bridged U-Net. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp 1–7

  • Chen W, Su L, Chen X, et al (2023) Rock image classification using deep residual neural network with transfer learning. Front Earth Sc-Switz 10(1079):447

  • Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1251–1258

  • Das S, Deka A, Iwahori Y, et al (2019) Contour-aware residual W-Net for nuclei segmentation. Preced Comput Sci 159:1479–1488

  • Ding X, Zhang X, Ma N, et al (2021) RepVGG: making VGG-style convnets great again. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 13733–13742

  • Gao S, Cheng M, Zhao K, et al (2019) Res2Net: a new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell 43(2):652–662

  • Gharehchobogh BK, Kuzekanani ZD, Khiavi AM et al (2023) Flotation froth image segmentation using Mask R-CNN. Miner Eng 192(107):959

    Google Scholar 

  • Guo MH, Lu CZ, Hou Q et al (2022) SegNext: rethinking convolutional attention design for semantic segmentation. NeurIPS 35:1140–1156

    Google Scholar 

  • Guo Q, Yang S, Wang Y et al (2023) Size measurement of blasted rock fragments based on FRRSNet+. Measurement 285(113):207

    Google Scholar 

  • Han K, Wang Y, Tian Q, et al (2020) GhostNet: more features from cheap operations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1580–1589

  • He K, Zhang X, Ren S, et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778

  • Howard AG, Zhu M, Chen B, et al (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

  • Hu J, Shen L, Sun G (2018) Squeeze-and-Excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7132–7141

  • Huang Z, Wang X, Huang L, et al (2019) CCNet: criss-cross attention for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 603–612

  • Huang H, Lin L, Tong R, et al (2020) U-Net3+: a full-scale connected U-Net for medical image segmentation. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 1055–1059

  • Iglovikov V, Shvets A (2018) TernausNet: U-Net with VGG11 encoder pre-trained on ImageNet for image segmentation. arXiv preprint arXiv:1801.05746

  • Jha D, Riegler MA, Johansen D, et al (2020) Double-Unet: a deep convolutional neural network for medical image segmentation. In: 2020 IEEE 33rd International Symposium on Computer-based Medical Systems (CBMS), pp 558–564

  • Lahmyed R, Ansari ME, Kerkaou Z (2022) Automatic road sign detection and recognition based on neural network. Soft Comput 26:1743–1764

    Article  Google Scholar 

  • Li C, Cheng G, Wang G et al (2023) Instance-aware distillation for efficient object detection in remote sensing images. IEEE T Geosci Remote 61:1–11

    Google Scholar 

  • Liu W, Anguelov D, Erhan D, et al (2016) SSD: single shot multibox detector. In: European Conference on Computer Vision (ECCV), pp 21–37

  • Liu X, Wang H, Jing H et al (2020) Research on intelligent identification of rock types based on Faster R-CNN method. IEEE Access 8:21804–21812

    Article  Google Scholar 

  • Liu Z, Mao H, Wu CY, et al (2022) A ConvNet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11976–11986

  • Liu Z, Lin Y, Cao Y, et al (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 10012–10022

  • Liu Y, Wang X, Zhang Z et al (2023) LOSN: lightweight ore sorting networks for edge device environment. Eng Appl Artif Intel 123(106):191

    Google Scholar 

  • Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3431–3440

  • Luan H, Xu H, Tang W et al (2023) Coal and gangue classification in actual environment of mines based on deep learning. Measurement 211(112):651

    Google Scholar 

  • Luo X, Wu H, Wang Z et al (2021) A novel approach to large-scale dynamically weighted directed network representation. IEEE Trans Pattern Anal Mach Intell 44(12):9756–9773

    Article  Google Scholar 

  • Luo X, Wu H, Li Z (2022) NeuLFT: a novel approach to nonlinear canonical polyadic decomposition on high-dimensional incomplete tensors. IEEE Trans Knowl Data Eng 35(6):6148–6166

    Google Scholar 

  • Lv Z, Wang W, Xu Z et al (2021) Fine-grained object detection method using attention mechanism and its application in coal-gangue detection. Appl Soft Comput 113(107):891

    Google Scholar 

  • Oktay O, Schlemper J, Folgoc LL, et al (2018) Attention U-Net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999

  • Qin X, Fan D, Huang C, et al (2021) Boundary-aware segmentation network for mobile and web applications. arXiv preprint arXiv:2101.04704

  • Rani G, Thakkar P, Verma A et al (2022) KUB-UNet: segmentation of organs of urinary system from a KUB X-ray image. Comput Meth Prog Biol 224(107):031

    Google Scholar 

  • Redmon J, Divvala S, Girshick R, et al (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 779–788

  • Ren S, He K, Girshick R et al (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  PubMed  Google Scholar 

  • Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp 234–241

  • Sandler M, Howard A, Zhu M, et al (2018) MobileNetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4510–4520

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  • Tan M, Le Q (2019) EfficientNet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp 6105–6114

  • Wan B, Zhou X, Zheng B et al (2023) LFRNet: localizing, focus, and refinement network for salient object detection of surface defects. IEEE T Instrum Meas 72:1–12

    Google Scholar 

  • Wang X, Girshick R, Gupta A, et al (2018) Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7794–7803

  • Wang Z, Wang H, Liu X et al (2022) An effective lightweight measurement model to enable coal dust size distribution analysis. IEEE T Instrum Meas 72:1–14

    CAS  Google Scholar 

  • Wang CY, Bochkovskiy A, Liao HYM (2023a) YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7464–7475

  • Wang W, Li Q, Zhang D et al (2023b) Image segmentation of adhesive ores based on MSBA-Unet and convex-hull defect detection. Eng Appl Artif Intell 123(106):185

    Google Scholar 

  • Wang W, Li Q, Zhang D et al (2023c) A survey of ore image processing based on deep learning. Chin J Eng 45(4):621–631

    Google Scholar 

  • Woo S, Debnath S, Hu R, et al (2023) ConvNextv2: co-designing and scaling convnets with masked autoencoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 16133–16142

  • Wu Z, Zhang X, Li F et al (2023) W-Net: a boundary-enhanced segmentation network for stroke lesions. Expert Syst Appl 230(120):637

    Google Scholar 

  • Xia X, Kulis B (2017) W-Net: a deep model for fully unsupervised image segmentation. arXiv preprint arXiv:1711.08506

  • Xia X, Chai X, Li Z et al (2023) MTYOLOX: multi-transformers-enabled YOLO for tree-level apple inflorescences detection and density mapping. Comput Electron Agr 209(107):803

    Google Scholar 

  • Xu Z, Ma W, Lin P et al (2021) Deep learning of rock images for intelligent lithology identification. Comput Geosci UK 154(104):799

    Google Scholar 

  • Xue Z, Chen L, Liu Z et al (2021) Rock segmentation visual system for assisting driving in TBM construction. Mach Vis Appl 32(4):1–12

    Article  Google Scholar 

  • Yu C, Wang J, Peng C, et al (2018) Learning a discriminative feature network for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1857–1866

  • Zhao H, Shi J, Qi X, et al (2017) Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2881–2890

  • Zhou W, Wang H, Wan Z (2022) Ore image classification based on improved CNN. Comput Electr Eng 99(107):819

    Google Scholar 

  • Zhou Y, Wong LNY, Tse KKC (2023) Novel rock image classification: the proposal and implementation of HKUDES_Net. Rock Mech Rock Eng 56(5):3825–3841

Download references

Funding

This work was supported by the Fundamental Research Funds for the Central Universities under Grant (FRF-TP-20-016A1).

Author information

Authors and Affiliations

  1. School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Wei Wang, Qing Li, Chengyong Xiao & Zihan Wang

  2. Key Laboratory of Knowledge Automation for Industrial Processes, University of Science and Technology Beijing, Beijing, 100083, China

    Wei Wang, Qing Li, Chengyong Xiao & Zihan Wang

  3. Financial Technology Innovation Department, Postal Savings Bank of China, Beijing, 100808, China

    Peng Chen

  4. School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Dezheng Zhang

  5. Beijing Key Laboratory of Knowledge Engineering for Materials Science, University of Science and Technology Beijing, Beijing, 100083, China

    Dezheng Zhang

Authors
  1. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dezheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chengyong Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zihan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by [Wei Wang], [Peng Chen] and [Zihan Wang]. Supervision and project administration were performed by [Qing Li], [Dezheng Zhang] and [Chengyong Xiao]. The first draft of the manuscript was written by [Wei Wang]. All the authors read and approved the final manuscript.

Corresponding author

Correspondence to Qing Li.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, W., Li, Q., Chen, P. et al. An improved U-Net-based network for multiclass segmentation and category ratio statistics of ore images. Soft Comput 28, 4725–4741 (2024). https://doi.org/10.1007/s00500-023-09131-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-023-09131-7

Keywords

Search

Navigation