Surface defect detection of steel strips based on improved YOLOv4

doi:10.1016/j.compeleceng.2022.108208

Computers and Electrical Engineering

Volume 102, September 2022, 108208

https://doi.org/10.1016/j.compeleceng.2022.108208 Get rights and content

Highlights

•
The modified YOLOv4 algorithm for defect detection of industrial strip steel.
•
Add a Convolutional Block Attention Module to the YOLOv4’s backbone network.
•
Receptive Field Block replaced the path aggregation network.
•
The outperforms most of the mainstream target detection networks.

Abstract

During the production and processing of steel strips, the production process and external factors lead to surface defects that negatively impact the strips’ integrity and functionality. However, traditional manual defect detection algorithms cannot meet modern accuracy requirements. Therefore, we propose a steel strip surface defect detection method based on the improved you-only-look-once version 4 (YOLOv4) algorithm. The attention mechanism is embedded in the backbone network structure, and the path aggregation network is modified into a customised receptive field block structure, which strengthens the feature extraction functionality of the network model. From the final experimental results, relative to the original YOLOv4 algorithm, the proposed algorithm's mean average precision values in the detection of four types of steel strip defects is improved by 3.87%, reaching 85.41%, thereby providing a new detection method for daily steel strip surface defects.

Graphical abstract

Introduction

Strip steel is an indispensable material for national construction and relates to the safety of urban infrastructure. During the production and processing of steel strips, owing to the influence of modern production technology and external factors, their surfaces may have various defects, such as plaques, inclusions, and scratches. These defects compromise their integrity and functionality, leading to construction accidents and endangering personal safety. Therefore, a speedy and accurate detection of surface defects in steel strips is critical. The traditional defect detection method involves random manual visual inspections. This method has the disadvantages of slow speed, low efficiency, high cost, poor detection environment, and potential safety hazards. Manual sampling is overly subjective, which leads to the low credibility of the results. Therefore, surface defect detection based on computer vision and deep learning is increasingly being used in modern production to solve these problems.

This paper is organised as follows. Section 2 presents the related work on computer vision and deep learning in the field of steel defect detection. In Section 3, information regarding the original you-only-look-once YOLO version 4 (YOLOv4) algorithm and the loss function is described. Detailed algorithm improvements are provided in Section 4. Section 5 presents the experimental results, and the conclusions are presented in Section 6.

Our contributions are as follows:

•
We propose a modified YOLOv4 algorithm for defect detection of industrial strip steel.
•
We design a convolutional block attention module (CBAM) for the backbone network and a receptive field block (RFB)-like structure replaces an enhance path aggregation network (PANet) to empower the information acquisition and feature extraction capability of the network.
•
The experimental results verify the superiority of the proposed algorithm.

Section snippets

Related work

As traditional defect detection methods have numerous drawbacks, computer vision-based methods are gradually gaining traction. Ref. [1] proposed a global adaptive percentile threshold method based on a gradient image. This method selectively segments the defect area and preserves defect features regardless of its size. Ref. [2] proposed a new algorithm based on wavelet anisotropic diffusion filtering. This method is effective at filtering out unwanted textural backgrounds in cold-rolled strips

YOLOV4 network structure

YOLOv4 is an updated and reinforced version of the YOLOv3 target-detection algorithm. The network structure is displayed in Fig. 1. First, the backbone network applies the cross-stage partial network (CSPnet) [16] to Darknet53, a 53-layer-deep CNN [17], and adds a CSPnet [16] structure to each group of resblock_body, continuously carrying out downsampling to obtain higher-level semantic information, which not only reduces the amount of computation but also enhances the gradient information.

A model's robustness improvement of YOLOv4

The improved YOLOv4 algorithm network architecture proposed in this study is marked in red boxes in Fig. 2, and the details are given in the subsequent two sections.

Experiment and result analysis

In this section, we describe the dataset, evaluation indicators, comparison objects, and methods, and finally dissect the experimental results to confirm the practicality of the improved model.

Conclusions

In response to the multiple difficulties faced in the task of steel strip defect detection, this study proposed a detection algorithm based on an improved YOLOv4. The value of this algorithm lies in the following:

•
Its YOLOv4 backbone network is embedded with CBAM.
•
Its YOLOv4 feature extraction network SPP module is improved to the customised RFB structure to deepen the network.
•
Among the four defect types in the NEU-DET dataset, our model achieved a 3.87% improvement in mAP over the original

CRediT authorship contribution statement

Mengjiao Li: Conceptualization, Methodology, Software, Investigation, Formal analysis, Writing – original draft. Hao Wang: Data curation, Writing – original draft. Zhibo Wan: Funding acquisition, Supervision, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (29)

B. Cao et al.
Defect detection of nickel-plated punched steel strip based on improved least square method[J]
Opt. Int. J. Light Electron Opt.
(2020)
X. Kou et al.
Development of a YOLO-V3-based model for detecting defects on steel strip surface[J]
Measurement
(2021)
N. Neogi et al.
Defect detection of steel surfaces with global adaptive percentile thresholding of gradient image[J]
J. Inst. Eng.
(2017)
J. Sun et al.
An effective method of weld defect detection and classification based on machine vision[J]
IEEE Trans. Ind. Inf.
(2019)
W. Liu et al.
SSD: Single Shot MultiBox Detector[J]
(2016)
T.Y. Lin et al.
Focal loss for dense object detection[J]
IEEE Trans. Pattern Anal. Mach. Intell.
(2017)
J. Redmon et al.
You Only Look Once: Unified, Real-Time Object Detection[J]
(2016)
J. Redmon et al.
YOLO9000: Better, Faster, Stronger
(2017)
Redmon J, Farhadi A. Yolov3: An incremental improvement[J]. arXiv preprint arXiv:1804.02767,...
Bochkovskiy A, Wang C Y, Liao H Y M. Yolov4: Optimal speed and accuracy of object detection[J]. arXiv preprint...

Ge Z, Liu S, Wang F, et al. Yolox: Exceeding yolo series in 2021[J]. arXiv preprint arXiv:2107.08430,...

S. Ren et al.

Faster R-CNN: towards real-time object detection with region proposal networks[J]

IEEE Trans. Pattern Anal. Mach. Intell.

(2017)

Cai Z, Vasconcelos N. Cascade r-cnn: Delving into high quality object detection[C]//Proceedings of the IEEE conference...

K. He et al.

Mask R-CNN [J]

IEEE Trans. Pattern Anal. Mach. Intell.

(2017)

Cited by (46)

A detection network for small defects of steel surface based on YOLOv7
2024, Digital Signal Processing: A Review Journal
Defect detection is crucial in steel production to prevent safety risks and prolong lifespan. However, current methods still face challenges in accurately detecting small and vague targets. In response to this problem, we propose a model named SRN-YOLO for steel surface detection based on YOLOv7. Firstly, a split residual convolution network (SResNet) is designed to capture gradient feature information. Then, a re-fusion feature pyramid network (RFPN) is built to minimize the loss of features. Furthermore, a location regression loss function named NWD-CIOU combined the Normalized Wasserstein Distance (NWD) and Complete Intersection Over Union (CIOU) is employed to increase the number of positive samples of small targets. The experiment results show that SRN-YOLO has excellent detection performance for steel surface defects. Specifically, the recognition accuracies of the model on NEU-DET and GC10-DET reach 81.2 mAP and 71.6 mAP, respectively, which are improved by 5.8% and 3.8% compared to YOLOv7.
Depth feature fusion based surface defect region identification method for steel plate manufacturing
2024, Computers and Electrical Engineering
Computers and electrical engineering have made great strides in steel plate manufacturing. Defect recognition techniques have also evolved. However, due to the large scale of defects, diverse features and sample imbalance problems of steel plates, the general algorithms often suffer from low recognition accuracy and weak robustness in practical detection. Aiming at the problems in recognition, this study proposes an improved defect segmentation network with coder-decoder structure to realize multi-scale interaction of features. Using the split-attention feature extraction module, defect features are learned adaptively. Meanwhile, combined with the group normalization module, a surface defect region recognition model based on depth feature fusion is established. The model was trained for comparative ablation using a migration learning approach. The experimental results confirm the efficiency of the technique. 89.11 % IoU and 94.24 % Dice can be achieved on the Severstal dataset using this method. The research can be applied as an intelligent system for quality monitoring throughout the production process, guiding its rational decision-making and control to realize the improvement of strip steel product quality.
LDD-Net: Lightweight printed circuit board defect detection network fusing multi-scale features
2024, Engineering Applications of Artificial Intelligence
The current printed circuit board (PCB) defect detection model is difficult to balance accuracy and computational cost and cannot satisfy the requirements of practical applications. In this regard, a lightweight PCB image defect detection network (LDD-Net) with high accuracy is proposed in this study. First, LDD-Net designs a novel lightweight feature extraction network (LFEN), which adds convolutional layers for extracting large-scale feature maps and uses an efficient downsampling method to extract richer defect features. LFEN not only greatly reduces resource requirements, but also improves detection accuracy. Second, LDD-Net devises multi-scale aggregation network (MAN) to facilitate information sharing among feature maps at different scales. In addition, MAN incorporates an efficient attention module to suppress similar background interference. Finally, LDD-Net uses the lightweight decoupling head to detect three groups of shallow feature maps with rich information of the small object. Experimental results on the PCB defect dataset ,and VisDrone public dataset show that the mean average precision (mAP), recall, and precision of LDD-Net outperform state-of-the-art lightweight object detection models. The mAP, computation, number of parameters, and detection speed of LDD-Net are 95.90%, 21.48 G, 5.13 M, and 78.13 frames per second, respectively. LDD-Net effectively balances accuracy, computational cost, and detection speed for applications in edge systems or resource-constrained embedded devices.
Revolutionising the Sustainability of Steel Manufacturing Using Computer Vision
2024, Procedia Computer Science
The pressure of the global sustainability problem is growing faster than ever after the social and economic havoc wreaked by the pandemic, as well as the present time nearing to the 2050 net zero carbon goal. In 2021 the steel industry had a turnover of around €125 billion and was directly responsible for providing at least 300000 jobs. In 2022, it was responsible for roughly 5% of CO₂ emissions in the EU and 7% globally, and was responsible for at least 20278 injuries and 90 deaths. Environmental solutions for mitigating negative consequences of the steel industry exist such as hydrogen-based steelmaking, carbon capture, utilisation and storage and electrolysis, whilst safety regulations and personal protective equipment have been used to improve safety. However, little has been done to combat the problem using the most powerful emerging technology of the fourth industrial revolution, artificial intelligence. In particular, computer vision has already shown great competence in a range of applications related to steelmaking, but without seriously considering sustainability, resulting in limited awareness of the potential benefits computer vision can bring to industry. A lack of this awareness leads to missed opportunities for sustainable development. This paper aims to address the gap in research that discusses computer vision capabilities for enhancing the sustainability of steel production by providing a literature review covering recent advances in computer vision, as well as an industry 4.0 approach for integrating computer vision systems with steelworks. Research presented here successfully exposes the untapped potential of computer vision in the steel industry and paves the way for future developments by exhibiting a blueprint for simultaneously elevating sustainability and technological advancement.
Hypertuned-YOLO for interpretable distribution power grid fault location based on EigenCAM
2024, Ain Shams Engineering Journal
Ensuring the reliability of electrical distribution networks is a pressing concern, especially given the power outages due to surface contamination on insulating components. Surface contamination can elevate surface conductivity, thereby resulting in failures that can lead to power shutdowns. Addressing this challenge, this paper proposes an approach for real-time monitoring of electrical distribution grids to prevent such incidents. A hypertuned version of the you only look once (YOLO) model is tailored for this application. We refine the model's hyperparameters by integrating a genetic algorithm to maximize its detection performance. The EigenCAM technique enhances the visual interpretability of the model's outcomes, providing operators with actionable insights for maintenance and monitoring tasks. Benchmark tests reveal that the proposed Hypertuned-YOLO outperforms Detectron (Masked R-CNN), YOLOv5, and YOLOv7 models. The Hypertuned-YOLO achieves an F1-score of 0.867 and a [email protected] of 0.922, validating its robustness and efficacy.
An automated defect detection method for optimizing industrial quality inspection
2024, Engineering Applications of Artificial Intelligence
Detecting defects in industrial-quality inspection is an important task, but defect detection remains challenging due to limited size, various types of defects, and imbalanced samples in images. In this paper, we propose a novel automatic defect detection network that efficiently detects defects. We introduce the Automated Sample Assignment (SCID) algorithm, which automatically divides the distribution of positive and negative samples based on similarity scores using the Gaussian mixture model. The Automated Sample Assignment (SCID) algorithm solves the problem of sample imbalance in defect detection networks caused by manually setting the intersection over the union (IOU) threshold and assigning positive and negative labels to samples. Then, instead of the pooling method used by most backbone networks, we use a loss fusion pool named LE-Pool to retain more detailed information. Finally, we use the soft weighted attention mechanism of the detection head (S-head) to identify noise information. Experimental results on multiple defect detection datasets show that the proposed method is superior to other detection methods in terms of accuracy.

View all citing articles on Scopus

View full text

Surface defect detection of steel strips based on improved YOLOv4

Highlights

Abstract

Graphical abstract

Introduction

Section snippets

Related work

YOLOV4 network structure

A model's robustness improvement of YOLOv4

Experiment and result analysis

Conclusions

CRediT authorship contribution statement

Declaration of Competing Interest

Opt. Int. J. Light Electron Opt.

Measurement

Defect detection of steel surfaces with global adaptive percentile thresholding of gradient image[J]

J. Inst. Eng.

An effective method of weld defect detection and classification based on machine vision[J]

IEEE Trans. Ind. Inf.

SSD: Single Shot MultiBox Detector[J]

Focal loss for dense object detection[J]

IEEE Trans. Pattern Anal. Mach. Intell.

You Only Look Once: Unified, Real-Time Object Detection[J]

YOLO9000: Better, Faster, Stronger

Faster R-CNN: towards real-time object detection with region proposal networks[J]

IEEE Trans. Pattern Anal. Mach. Intell.

Mask R-CNN [J]

IEEE Trans. Pattern Anal. Mach. Intell.