Abstract
Reliable industrial defect inspection is one of the main challenges in manufacturing scenarios, especially for the inspection of structural defects. However, the cost of missing a defect is much higher than the cost of misclassifying a qualified sample, which is seldom emphasized in previous work. Thus, the purpose of our work has two folds: reduce the omission rate of defective samples; classify industrial samples correctly. To that end, in this paper, we first define a position tag for each sample, where samples with the same position tag describe the same product information. We also design the multi-position weighted-resampling (MPWR) method for extracting paired data with identical tags. Then, in order to fully learn from the paired data, we propose a multiple position-based bi-branch (MPB3) neural network architecture to perform similarity measurements and multi-classifications simultaneously. Experimental results demonstrate the effectiveness of our method and generalization capacity to data from unknown tags by comparing with other methods. For example, the proposed method achieves 2.77\(\%\)/1.00\(\%\) omission rates and 96.81\(\%\)/99.03\(\%\) weighted F-Scores on the SMT defect dataset and the motor brush holder dataset, respectively. In addition, the average running time of the method only needs 9.6 ms, which meets requirements of cycle time in manufacturing industries. In conclusion, the omission rate of defective samples can be reduced effectively by the position-based method that consists of MPWR method and MPB3 structure, which greatly improves productivity in real production lines.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and material
Not applicable.
References
Acciani, G., Brunetti, G., & Fornarelli, G. (2006). Application of neural networks in optical inspection and classification of solder joints in surface mount technology. IEEE Transactions on Industrial Informatics, 2(3), 200–209. https://doi.org/10.1109/tii.2006.877265
Bradford, J. P., Kunz, C., Kohavi, R., Brunk, C., & Brodley, C. E. (1998). Pruning decision trees with misclassification costs. European Conference on Machine Learning, 1398, 131–136. https://doi.org/10.1007/bfb0026682
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357. https://doi.org/10.1613/jair.953
Chen, X., & He, K. (2021). Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 15750–15758). arXiv:2011.10566.
Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., & Tian, Q. (2019). Centernet: Keypoint triplets for object detection. In 2019 IEEE/CVF international conference on computer vision (ICCV) (pp. 6568–6577). https://doi.org/10.1109/iccv.2019.00667.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In 2016 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 770–778). https://doi.org/10.1109/cvpr.2016.90.
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020). Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(8), 2011–2023. https://doi.org/10.1109/TPAMI.2019.2913372
Jiang, X., Pan, S., Long, G., Chang, J., Jiang, J., & Zhang, C. (2018). Cost-sensitive hybrid neural networks for heterogeneous and imbalanced data. In 2018 International joint conference on neural networks (IJCNN) (pp 1–8). https://doi.org/10.1109/ijcnn.2018.8489420.
Knoll, U., Nakhaeizadeh, G., & Tausend, B. (1994). Cost-sensitive pruning of decision trees. European Conference on Machine Learning, 784, 383–386. https://doi.org/10.1007/3-540-57868-4_79
Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2), 318–327. https://doi.org/10.1109/TPAMI.2018.2858826
Liu, X. Y., Wu, J., & Zhou, Z. H. (2009). Exploratory undersampling for class-imbalance learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 39(2), 539–550. https://doi.org/10.1109/tsmcb.2008.2007853
Molchanov, P., Tyree, S., Karras, T., Aila, T., & Kautz, J. (2016). Pruning convolutional neural networks for resource efficient transfer learning. CoRR. arXiv:1611.06440.
Motiian, S., Piccirilli, M., Adjeroh, D. A., & Doretto, G. (2017). Unified deep supervised domain adaptation and generalization. In Proceedings of the IEEE international conference on computer vision (pp. 5715–5725). https://doi.org/10.1109/iccv.2017.609.
Noble, W. S. (2006). What is a support vector machine? Nature Biotechnology, 24(12), 1565–1567. https://doi.org/10.1038/nbt1206-1565
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. In 2017 IEEE international conference on computer vision (ICCV) (pp. 618–626). https://doi.org/10.1007/s11263-019-01228-7.
Sun, B., & Saenko, K. (2016). Deep coral: Correlation alignment for deep domain adaptation. European Conference on Computer Vision, 9915, 443–450. https://doi.org/10.1007/978-3-319-49409-8_35
Tao, P., Yi, H., Wei, C., Ge, L. Y., & Xu, L. (2013). A method based on weighted F-score and SVM for feature selection. In 2013 25th Chinese control and decision conference (CCDC) (pp. 4287–4290). https://doi.org/10.1109/ccdc.2013.6561705.
Tayeh, T., Aburakhia, S., Myers, R., & Shami, A. (2020). Distance-based anomaly detection for industrial surfaces using triplet networks. In 2020 11th IEEE annual information technology, electronics and mobile communication conference (IEMCON) (pp. 0372–0377). https://doi.org/10.1109/iemcon51383.2020.9284921.
Van Rijsbergen, C. J. (1974). Foundation of evaluation. Journal of Documentation, 30(4), 365–373. https://doi.org/10.1108/eb026584
Wang, T., Chen, Y., Qiao, M., & Snoussi, H. (2018). A fast and robust convolutional neural network-based defect detection model in product quality control. The International Journal of Advanced Manufacturing Technology, 94(9), 3465–3471. https://doi.org/10.1007/s00170-017-0882-0
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612. https://doi.org/10.1109/tip.2003.819861
Wu, F., & Zhang, X. (2011). Feature-extraction-based inspection algorithm for IC solder joints. IEEE Transactions on Components, Packaging and Manufacturing Technology, 1(5), 689–694. https://doi.org/10.1109/tcpmt.2011.2118208
Wu, F., & Zhang, X. (2014). An inspection and classification method for chip solder joints using color grads and Boolean rules. Robotics and Computer-Integrated Manufacturing, 30(5), 517–526. https://doi.org/10.1016/j.rcim.2014.03.003
Yoo, J. C., & Han, T. H. (2009). Fast normalized cross-correlation. Circuits, Systems and Signal Processing, 28, 819–843. https://doi.org/10.1007/s00034-009-9130-7
Zadrozny, B., Langford, J., & Abe, N. (2003). Cost-sensitive learning by cost-proportionate example weighting. In Third IEEE international conference on data mining (pp 435–442). https://doi.org/10.1109/icdm.2003.1250950.
Zhou, S., & Zhang, Y. (2016). Active learning for cost-sensitive classification using logistic regression model. In 2016 IEEE international conference on big data analysis (ICBDA) (pp. 1–4). https://doi.org/10.1109/icbda.2016.7509840.
Funding
The work is supported by Anhui Center for Applied Mathematics, the National Science Foundation of China (No. 11871447), the Special Project of Strategic Leading Science and Technology of Chinese Academy of Sciences (No. XDC08010100), and the National Key Research and Development Program of Ministry of Science and Technology of China (No. 2018AAA0101001).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflict of interest to declare that are relevant to the content of this article.
Code availability
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, F., Yang, Z., Huang, Z. et al. A multiple position-based bi-branch model for structural defect inspection. J Intell Manuf 34, 1601–1614 (2023). https://doi.org/10.1007/s10845-021-01870-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-021-01870-4