Abstract
Metal packaging is an alternative technology to guarantee the environmental resistance and the performance reliability of ICs. Surface defect inspection of IC metal packages is an indispensable process during manufacturing. Here, a statistical modeling framework is proposed based on a GAN for surface defect inspection of IC metal packages, which involves several adaptive schemes. To the best of our knowledge, we first introduce the GAN to establish a machine vision based method for surface defect inspection of IC metal packages. IC metal package images are automatically acquired by an AOI system and employed for inspection via the proposed framework. To tackle the problem of imbalanced data in real industries, the framework only utilizes qualified samples to train the GAN template generator, which can characterize the intrinsic pattern of qualified samples. Then, a weight mask scheme is proposed to suppress the interference pixels in the difference image corresponding to qualified samples. Next, an adaptive thresholding scheme is proposed to adaptively determine an appropriate threshold for each inspected sample. Finally, an image patch-based defect evaluation scheme is designed to local-to-global evaluate the surface qualities of IC metal packages. Comparison experiments indicate that the proposed framework achieves better inspection performance in terms of 3.16% error rate and 0.89% mission rate at a reasonable inspection time of 119.86 ms per sample, which is superior to some existing deep learning based inspection methods for surface defect inspection of IC metal packages.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akcay, S., Atapour-Abarghouei, A., & Breckon, T. P. (2019a). GANomaly: Semi-supervised anomaly detection via adversarial training. In Computer Vision-ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part III 14 (pp. 622–637). Springer International Publishing. https://doi.org/10.1007/978-3-030-20893-6_39
Akcay, S., Atapour-Abarghouei, A., & Breckon, T. P. (2019b). Skip-GANomaly: Skip connected and adversarially trained encoder-decoder anomaly detection. In 2019b International Joint Conference on Neural Networks (IJCNN) (pp. 1–8). IEEE. https://doi.org/10.1109/IJCNN.2019.8851808
Chen, S. H., Chiou, A. H., & Wang, C. C. (2018). A NMF-based image restoration scheme with applications to LED integrated substrate defect detection. IEEE Transactions on Semiconductor Manufacturing, 31(4), 486–494. https://doi.org/10.1109/TSM.2018.2867840
Chen, S. H., & Perng, D. B. (2016). Automatic optical inspection system for IC molding surface. Journal of Intelligent Manufacturing, 27(5), 915–926. https://doi.org/10.1007/s10845-014-0924-5
Chen, X., Chen, J., Han, X., Zhao, C., Zhang, D., Zhu, K., & Su, Y. (2020). Alight-weighted CNN model for wafer structural defect detection. IEEE Access, 8, 24006–24018. https://doi.org/10.1109/ACCESS.2020.2970461
Du, D. Y., & Shi, Z. (2020, November). A wafer map defect pattern classification model based on deep convolutional neural network. In 2020 IEEE 15th International Conference on Solid-State & Integrated Circuit Technology (ICSICT) (pp. 1–3). IEEE. https://doi.org/10.1109/ICSICT49897.2020.9278021
Kumar, B., Tiwari, U. K., Kumar, S., Tomer, V., & Kalra, J. (2020). Comparison and performance evaluation of boundary fill and flood fill algorithm. International Journal of Innovative Technology and Exploring Engineering, 8, 9–13. https://doi.org/10.35940/ijitee.L1002.10812S319
Lancaster, A., & Keswani, M. (2018). Integrated circuit packaging review with an emphasis on 3D packaging. Integration, 60, 204–212. https://doi.org/10.1016/j.vlsi.2017.09.008
Li, J., Cai, N., Mo, Z., Zhou, G., & Wang, H. (2021). IC solder joint inspection via generator-adversarial-network based template. Machine Vision and Applications, 32(4), 96. https://doi.org/10.1007/s00138-021-01218-1
Li, Y., Srinath, P. K. M., & Goyal, D. (2016). A review of failure analysis methods for advanced 3D microelectronic packages. Journal of Electronic Materials, 45, 116–124. https://doi.org/10.1007/s11664-015-4165-y
Lin, H. D., Chung, C. Y., & Lin, W. T. (2008). Principal component analysis based on wavelet characteristics applied to automated surface defect inspection. WSEAS Transactions on Computer Research, 3(4), 193–202. https://doi.org/10.5555/1466874.1466875
Lin, H., Li, B., Wang, X., Shu, Y., & Niu, S. (2019). Automated defect inspection of LED chip using deep convolutional neural network. Journal of Intelligent Manufacturing, 30, 2525–2534. https://doi.org/10.1007/s10845-018-1415-x
Liu, Q., Wang, M., Wang, G., Gao, R., & Hanajima, N. (2020). Detection algorithm of porosity defect on surface of micro-precision glass encapsulated electrical connectors. Journal of Robotics Networking and Artificial Life, 7(3), 212–216. https://doi.org/10.2991/jrnal.k.200909.015
Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Paul Smolley, S. (2017). Least squares generative adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2813–2821). https://doi.org/10.1109/ICCV.2017.304
Minhas, M. S., & Zelek, J. (2020). Semi-supervised anomaly detection using autoencoders. arXiv preprint https://doi.org/10.48550/arXiv.2001.03674
Pecht, M., Agarwal, R., McCluskey, F. P., Dishongh, T. J., Javadpour, S., & Mahajan, R. (2017). Electronic packaging materials and their properties. CRC Press.
Rahman, F., Forte, D., & Tehranipoor, M. M. (2016). Reliability vs. security: Challenges and opportunities for developing reliable and secure integrated circuits. In 2016 IEEE International Reliability Physics Symposium (IRPS) (4C-6). IEEE. https://doi.org/10.1109/IRPS.2016.7574542
Rudolph, M., Wandt, B., & Rosenhahn, B. (2020). Same same but DifferNet: Semi-supervised defect detection with normalizing flows. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 1907–1916). https://doi.org/10.48550/arXiv.2008.12577
Tsai, D. M., Fan, M. S. K., Huang, Y. Q., & Chiu, W. Y. (2019). Saw-mark defect detection in heterogeneous solar wafer images using GAN-based training samples generation and CNN classification. Proceedings of the 14th International Joint Conference on Computer Vision, Prague, Czech Republic, 5, 234–240. https://doi.org/10.5220/0007306602340240
Tu, K. N. (2011). Reliability challenges in 3D IC packaging technology. Microelectronics Reliability, 51(3), 517–523. https://doi.org/10.1016/j.microrel.2010.09.031
Wan, Y. J., Li, G., Yao, Y. M., Zeng, X. L., & Sun, R. (2020). Recent advances in polymer-based electronic packaging materials. Composites Communications, 19, 154–167. https://doi.org/10.1016/j.coco.2020.03.011
Wang, M., Joo, H., Kim, J.-S., & Park P. (2009, December). A fast algorithm for segmenting defects on the surface of QFN packages. In 2009 International Conference on Information Engineering and Computer Science (pp. 1–4). IEEE. https://doi.org/10.1109/ICIECS.2009.5366301
Yi, J., & Yoon, S. (2020). Patch SVDD: Patch-level SVDD for anomaly detection and segmentation. In Proceedings of the Asian Conference on Computer Vision. https://doi.org/10.48550/arXiv.2006.16067
Zhong, Z., & Ma, Z. (2021). A novel defect detection algorithm for flexible integrated circuit package substrates. IEEE Transactions on Industrial Electronics, 69(2), 2117–2126. https://doi.org/10.1109/TIE.2021.3057026
Acknowledgements
This work was in part supported by the National Natural Science Foundation of China (Nos. 62171142 and 61901123), the Research Fund for Colleges and Universities in Huizhou (No. 2019HZKY003), the Project of Jihua Laboratory (No. X190071UZ190) and the National Natural Science Foundation of Guangdong Province, China (No. 2021A1515011908).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wu, Z., Cai, N., Chen, K. et al. GAN-based statistical modeling with adaptive schemes for surface defect inspection of IC metal packages. J Intell Manuf 35, 1811–1824 (2024). https://doi.org/10.1007/s10845-023-02146-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-023-02146-9