Abstract
The mura defects of thin film transistor-liquid crystal display (TFT-LCD) panels have low contrast and random locations, which makes it impossible for us to correctly evaluate the number and type of mura defects on the image in the field inspection. In response to the above problems, this paper proposes a lightweight YOLO-ADPAM detection method based on an attention mechanism. First, we designed a K-means-ciou++ clustering algorithm using the Complete-Intersection-Over-Union loss function to cluster the anchor box size of the display defect dataset, making the bounding box regression more accurate and stable and improving the recognition and positioning accuracy of the algorithm. Second, we design a parallel attention module, combining the advantages of the channel and spatial attention mechanisms to effectively extract helpful information from feature maps. The channel attention branch can compensate for the defect information lost by global average pooling to a certain extent, and selecting a larger convolution kernel in the spatial attention branch is beneficial to retain crucial spatial information. Third, using atrous spatial pyramid pooling and depthwise separable convolution in the Neck network can further improve the receptive field of the feature map and improve the detection accuracy of the network. The experimental results show that the mAP of our proposed YOLO-ADPAM algorithm in TFT-LCD defect detection reaches 98.20%, and the detection speed reaches 83.23 FPS, which meets the detection accuracy and real-time requirements of TFT-LCD defect detection tasks.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv:2004.10934
Çelik, A., Küçükmanisa, A., Sümer, A., et al. (2020). A real-time defective pixel detection system for LCDS using deep learning based object detectors. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-020-01704-9
Chen, M., Chen, P., Wang, S., et al. (2022). TFT-LCD mura defect visual inspection method in multiple backgrounds. Journal of the Society for Information Display. https://doi.org/10.1002/jsid.1171
Chen, L. C., Papandreou, G., Kokkinos, I., et al. (2017). Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFS. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834–848. https://doi.org/10.1109/TPAMI.2017.2699184
Cui, Y., Wang, S., Wu, H., et al. (2021). Liquid crystal display defects in multiple backgrounds with visual real-time detection. Journal of the Society for Information Display, 29(7), 547–560. https://doi.org/10.1002/jsid.997
Deng, Y., Pan, X., Wang, X., et al. (2019). Vison-based 3D shape measurement system for transparent microdefect characterization. IEEE Access, 7, 105721–105733. https://doi.org/10.1109/ACCESS.2019.2931194
Dong, H., Song, K., He, Y., et al. (2019). Pga-net: Pyramid feature fusion and global context attention network for automated surface defect detection. IEEE Transactions on Industrial Informatics, 16(12), 7448–7458. https://doi.org/10.1109/TII.2019.2958826
Dong, X., Taylor, C. J., & Cootes, T. F. (2020). A random forest-based automatic inspection system for aerospace welds in X-ray images. IEEE Transactions on Automation Science and Engineering, 18(4), 2128–2141. https://doi.org/10.1109/TASE.2020.3039115
Ge, Z., Liu, S., Wang, F., et al. (2021) Yolox: Exceeding yolo series in 2021. arXiv:2107.08430.
He, K., Zhang, X., Ren, S., et al. (2015). Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9), 1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824
Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, (pp 7132–7141)
Kim, M., Lee, M., An, M., et al. (2020). Effective automatic defect classification process based on CNN with stacking ensemble model for TFT-LCD panel. Journal of Intelligent Manufacturing, 31(5), 1165–1174. https://doi.org/10.1007/s10845-019-01502-y
Kwak, J., Lee, K. B., Jang, J., et al. (2019). Automatic inspection of salt-and-pepper defects in OLED panels using image processing and control chart techniques. Journal of Intelligent Manufacturing, 30(3), 1047–1055. https://doi.org/10.1007/s10845-017-1304-8
Le, N. T., Wang, J. W., Shih, M. H., et al. (2020). Novel framework for optical film defect detection and classification. IEEE Access, 8, 60,964-60,978. https://doi.org/10.1109/ACCESS.2020.2982250
Lin, T. Y., Goyal, P., Girshick, R., et al. (2017). Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, (pp. 2980–2988)
Luo, R., Chen, R., Jia, F., et al. (2022). RBD-NET: Robust breakage detection algorithm for industrial leather. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-022-01962-9
Mei, S., Cheng, J., He, X., et al. (2021). A novel weakly supervised ensemble learning framework for automated pixel-wise industry anomaly detection. IEEE Sensors Journal, 22(2), 1560–1570. https://doi.org/10.1109/JSEN.2021.3131908
Ming, W., Zhang, S., Liu, X., et al. (2021). Survey of mura defect detection in liquid crystal displays based on machine vision. Crystals, 11(12), 1444. https://doi.org/10.3390/cryst11121444
Pan, Y., Lu, R., & Zhang, T. (2020). FPGA-accelerated textured surface defect segmentation based on complete period Fourier reconstruction. Journal of Real-Time Image Processing, 17(5), 1659–1673. https://doi.org/10.1007/s11554-019-00927-1
Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv:1804.02767
Ren, S., He, K., Girshick, R., et al. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in neural information processing systems. https://doi.org/10.1109/tpami.2016.2577031
Schlosser, T., Friedrich, M., Beuth, F., et al. (2022). Improving automated visual fault inspection for semiconductor manufacturing using a hybrid multistage system of deep neural networks. Journal of Intelligent Manufacturing, 33(4), 1099–1123. https://doi.org/10.1007/s10845-021-01906-9
Sun, Y., Li, X., & Xiao, J. (2019). A cascaded mura defect detection method based on mean shift and level set algorithm for active-matrix OLED display panel. Journal of the Society for Information Display, 27(1), 13–20. https://doi.org/10.1002/jsid.743
Tan, M., Pang, R., & Le, Q. V. (2020). Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10781–10790). https://doi.org/10.48550/arXiv.1911.09070
Wang, C.Y., Bochkovskiy, A., & Liao, H. Y. M. (2021). Scaled-yolov4: Scaling cross stage partial network. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition (pp. 13029–13038). https://doi.org/10.1109/CVPR46437.2021.01283
Wang, C. Y., Liao, H. Y. M., Wu, Y. H., et al. (2020a). CSPNET: A new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (pp 390–391). https://doi.org/10.1109/CVPRW50498.2020.00203
Wang, Q., Wu, B., Zhu, P., et al. (2020b). Supplementary material for ECA-NET: Efficient channel attention for deep convolutional neural networks. In Proceedings of the 2020 IEEE/CVF conference on computer vision and pattern recognition, IEEE, Seattle, WA, USA (pp. 13–19) https://doi.org/10.1109/CVPR42600.2020.01155
Woo, S., Park, J., Lee, J. Y., et al. (2018). CBAM: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV) (pp. 3–19) https://doi.org/10.1007/978-3-030-01234-2_1
Yang, H., Chen, Y., Song, K., et al. (2019). Multiscale feature-clustering-based fully convolutional autoencoder for fast accurate visual inspection of texture surface defects. IEEE Transactions on Automation Science and Engineering, 16(3), 1450–1467. https://doi.org/10.1109/TASE.2018.2886031
Yang, H., Zhou, Q., Song, K., et al. (2020). An anomaly feature-editing-based adversarial network for texture defect visual inspection. IEEE Transactions on Industrial Informatics, 17(3), 2220–2230. https://doi.org/10.1109/TII.2020.3015765
Zheng, Z., Wang, P., Liu W., et al. (2020). Distance-IOU loss: Faster and better learning for bounding box regression. In: Proceedings of the AAAI conference on artificial intelligence (pp. 12993–13000) https://doi.org/10.1609/aaai.v34i07.6999
Zhi, Z., Jiang, H., Yang, D., et al. (2022). An end-to-end welding defect detection approach based on titanium alloy time-of-flight diffraction images. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01905-w
Zhou, X., Wang, D., & Krähenbühl, P. (2019). Objects as points. arXiv:1904.07850
Zhu, .X, Lyu, S., Wang, X., et al. (2021). Tph-yolov5: Improved yolov5 based on transformer prediction head for object detection on drone-captured scenarios. In: Proceedings of the IEEE/CVF international conference on computer vision (pp. 2778–2788) https://doi.org/10.1109/ICCVW54120.2021.00312
Funding
Funding was provided by Natural Science Foundation of China (Grant Nos. 51965029 and 52065035).
Author information
Authors and Affiliations
Contributions
PC methodology, manuscript drafting, conceptualization, manuscript revision, experimental data curation, manuscript review, and supervision. SW conceptualization, manuscript revision, experimental data curation, and manuscript review. YS experimental data curation and manuscript review. MC experimental data curation and manuscript review. XM supervision. YC supervision. ZC supervision. SC supervision. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, P., Chen, M., Wang, S. et al. Real-time defect detection of TFT-LCD displays using a lightweight network architecture. J Intell Manuf 35, 1337–1352 (2024). https://doi.org/10.1007/s10845-023-02110-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-023-02110-7