Abstract
The classification of defect types during LCD panel production is very important because it is closely related to deciding whether a defect panel is restorable. But since defect areas are very small compared to the panel area, it is hard to classify defect types by images. Therefore, we need to eliminate the background pattern of the panel, but this is not an easy task because the brightness and saturation of the background varies, even in a single image. In this paper, we propose an indicator that can distinguish between defect and background area, which is robust to brightness change and minor noises. With this indicator, we got useful defect information and images with patterns eliminated to make a more efficient defect classifier. The convolutional neural network with stacked ensemble techniques played a great role in improving defect classification performance, when various information from image preprocessing was combined.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Badmos, O., Kopp, A., Bernthaler, T., & Schneider G. (2019). Image-based defect detection in lithium-ion battery electrode using convolutional neural networks. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-019-01484-x.
Cheng, H., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., et al. (2016). Wide and deep learning for recommender systems. arXiv:1606.07792v1.
Clevert, D., Unterthiner, T., & Hochreiter, S. (2015). Fast and accurate deep network learning by exponential linear units (ELUs). arXiv:1511.07289v5.
Das, R., Turkoglu, I., & Sengur, A. (2009). Effective diagnosis of heart disease through neural network ensembles. Expert Systems with Applications, 36, 3976–3982.
Faghih-Roohi, S., Hajizadeh, S., Nunez, A., Babuska R., & De Schutter, B. (2016). Deep convolutional neural networks for detection of rail surface defects. In International joint conference on neural networks (IJCNN).
He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition. arXiv:1512.03385v1 [cs.CV].
Holliday, A., Barekatain, M., Laurmaa, J., Kandaswamy, C., & Prendinger, H. (2017). Speedup of deep learning ensembles for semantic segmentation using model compression technique. Computer Vision and Image Understanding IJC, 164, 16–26.
Ju, C., Bibaut, A., & van der Laan, M. J. (2017). The relative performance of ensemble methods with deep convolutional neural networks for image classification. ArXiv:1704.01664v1.
Kameyama, K., Kosugi, Y., Okahahi, T., & Izumita, M. (1998). Automatic defect classification in visual inspection of semiconductors using neural networks. IEICE Transactions on Information and Systems, 81(11), 1261–1271.
Kim, J., Kim, S., Kwon, N., Kang, H., Kim Y., & Lee, C. (2018). Deep learning based automatic defect classification in through-silicon via process: FA: Factory automation. In 29th annual SEMI advanced semiconductor manufacturing conference (ASMC).
Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). ImageNet classification with deep convolutional neural networks. In NIPS12 proceedings of the 25th international conference on neural information processing systems (Vol. 1).
Kwon, O., Kim, H., Ham, H., Kim, W., Kim, G., Cho, J., et al. (2018). A deep neural network for classification of melt-pool images in metal additive manufacturing. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-018-1451-6.
Lee, K., Chang, M., & Park, P. (2007). Periodic comparison method for defects inspection of TFT-LCD panel. In Proceedings of the 7th WSEAS international conference on robotics, control & manufacturing technology, Hangzhou, China
Lee, K., Ko, M., Lee, J., Koo, T., & Park, K. (2004). Defect detection method for TFT-LCD panel based on saliency map model. In IEEE region 10 conference TENCON.
Li, T., Tsai, J., Chang, R., Ho, L., & Yang, C. (2013). Pretest gap Mura on TFT LCDs using the optical interference pattern sensing method and neural network classification. IEEE Transactions on Industrial Electronics, 60(9), 3976–3982.
Lim, T. Y., Ratnam, M. M., & Khalid, M. (2007). Automatic classification of weld defects using simulated data and MLP neural network. Insight, 49(3), 154–159.
Lin, H., Li, B., Wang, X., Shu, Y., & Niu, S. (2018). Automated defect inspection of LED chip using deep convolutional neural network. Journal of Intelligent Manufacturing, 30(6), 2525–2534.
Lin, M., Chen, Q., & Yan, S. (2013). Network in network. arXiv:1312.4400.
Lu, C., & Tsai, D. (2005). Automatic defect inspection for LCDs using singular value decomposition. The International Journal of Advanced Manufacturing Technology, 25(1–2), 53–61.
Masci, J., Meier, U., Ciresan, D., Schmidhuber, J., & Fricout, G. (2012). Steel defect classification with max-pooling convolutional neural networks. In The 2012 international joint conference on neural networks (IJCNN), Brisbane, QLD, pp. 1–6. https://doi.org/10.1109/IJCNN.2012.6252468.
Nakazawa, T., & Kulkarni, D. V. (2018). Wafer map defect pattern classification and image retrieval using convolutional neural network. IEEE Transactions on Semiconductor Manufacturing, 31(2), 309–314.
Park, E., Han, X., Berg, T. L., & Berg, A. C. (2016). Combining multiple sources of knowledge in deep CNNs for action recognition. In IEEE winter conference on applications of computer vision (WACV).
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556.
Song, Y., Choi, D., & Park, K. (2004). Morphological Blob-Mura defect detection method for TFT-LCD panel inspection. In KES: knowledge-based intelligent information and engineering systems (pp. 862–868).
Tsai, D. M., & Hung, C. Y. (2005). Automatic defect inspection of patterned thin film transistor-liquid crystal display (TFT-LCD) panels using one-dimensional Fourier reconstruction and wavelet decomposition. International Journal of Production Research, 43(21), 4589–4607 .
Tsai, K., & Luo, J. (2017). An inverse model for injection molding of optical lens using artificial neural network coupled with genetic algorithm. Journal of Intelligent Manufacturing, 28(2), 473–487.
Wang, T., Chen, Y., Qiao, M., & Snoussi, H. (2017). A fast and robust convolutional neural network-based defect detection model in product quality control. The International Journal of Advanced Manufacturing Technology, 94(9–12), 3465–3471.
Weiner, D., Thamer, H., & Scholz-Reiter, B. (2013). Learning defect classifiers for textured surfaces using neural networks and statistical feature representations. Procedia CIRP, 7, 347–352.
Yanh, H., Mei, S., Song, K., Tao, B., & Yin, Z. (2018). Transfer-learning-based online Mura defect classification. IEEE Transactions on Semiconductor Manufacturing, 31(1), 116–123.
Zhang, Y. F., & Bresee, R. R. (1995). Fabric defect detection and classification using image analysis. Textile Research Journal, 65(1), 1–9.
Acknowledgements
This research was supported by Aim Systems Inc. by providing us with LCD panel data and labeling class of the defects.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
A.1 Defect feature list and descriptions
The defect features list and descriptions extracted from binary thresholding on 2.1.4 are in Table 6.
Rights and permissions
About this article
Cite this article
Kim, M., Lee, M., An, M. et al. Effective automatic defect classification process based on CNN with stacking ensemble model for TFT-LCD panel. J Intell Manuf 31, 1165–1174 (2020). https://doi.org/10.1007/s10845-019-01502-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-019-01502-y