Abstract
The inspection of conductive particles after Anisotropic Conductive Film (ACF) bonding is a common and crucial step in the TFT-LCD manufacturing process since the number of high-quality conductive particles is a key indicator of ACF bonding quality. However, manual inspection under microscope is a time-consuming, tedious, and error-prone. Therefore, there is an urgent demand in industry for the automatic conductive particle inspection system. It is challenging for automatic conductive particle quality inspection due to the existence of complex background noise and diversified particle appearance, including shape, size, clustering and overlapping, etc. As a result, it lacks an effective automatic detection method to handle all the complex particle patterns. In this paper, we propose a U-shaped deep residual neural network (i.e., U-ResNet), which can learn features of particle from massively labeled data. The experimental results show that the proposed method achieves high detection accuracy and recall rate, which exceedingly outperforms the previous work. Also, our system is very efficient and can work in real time.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828.
Cao, K., & Jain, A. K. (2019). Automated latent fingerprint recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(4), 788–800.
Drozdzal, M., Vorontsov, E., Chartrand, G., Kadoury, S., & Pal, C. (2016). The importance of skip connections in biomedical image segmentation. Deep learning and data labeling for medical applications (pp. 179–187). Cham: Springer.
Ferguson, M., Ak, R., Lee, Y. T. T., & Law, K. H. (2017). Automatic localization of casting defects with convolutional neural networks. In 2017 IEEE international conference on big data (big data) (pp. 1726–1735). IEEE.
Gonzalez, R. C., & Woo, R. E. (2017). Digital image processing (4th ed.). London: Pearson.
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. Cambridge: MIT Press.
Graves, A., Mohamed, A. R., & Hinton, G. (2013). Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing. IEEE. arXiv:1303.5778.
He, K., Zhang, X., Ren, S., & Sun, J. (2015a). Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385.
He, K., Zhang, X., Ren, S., & Sun, J. (2015b). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision (pp. 1026–1034).
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).
Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504–507.
Jia, L., Sheng, X., Xiong, Z., Wang, Z., & Ding, H. (2014). Particle on bump (POB) technique for ultra-fine pitch chip on glass (COG) applications by conductive particles and adhesives. Microelectronics Reliability, 54(4), 825–832.
Kingma, D. P., & Ba, J. L. (2015). Adam: A method for stochastic optimization. In International conference on learning representations (pp. 1–13).
Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, & K. Q. Weinberger (Eds.), Advances in neural information processing systems 25 (pp. 1097–1105). New York: Curran Associates.
Kwon, O., Kim, H. G., Ham, M. J., Kim, W., Kim, G. H., Cho, J. H., Kim, N. I., & Kim, K. (2018). A deep neural network for classification of melt-pool images in metal additive manufacturing. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-018-1451-6.
LeCun, Y. (1988). A theoretical framework for back-propagation. In D. Touretzky, G. Hinton, and T. Sejnowski (Eds.), Proceedings of the 1988 connectionist models summer school, pages 21–28, CMU, Pittsburgh, Pa, 1988. Morgan Kaufmann.
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.
Lin, C. S., Huang, K. H., Lin, T. C., Shei, H. J., & Tien, Chuen-Lin. (2011). An automatic inspection method for the fracture conditions of anisotropic conductive film in the tft-lcd assembly process. International Journal of Optomechatronics, 5(3), 286–298.
Lin, H., Li, B., Wang, X., Shu, Y., & Niu, S. (2018). Automated defect inspection of led chip using deep convolutional neural network. Journal of Intelligent Manufacturing, 30(6), 2525–2534.
Long, J., Shelhamer, E., & Darrell, T. (2015a). Fully convolutional networks for semantic segmentation. In 2015 IEEE conference on computer vision and pattern recognition (CVPR), (pp. 3431–3440).
Long, J., Shelhamer, E., & Darrell, T. (2015b). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431–3440).
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781.
Ni, G., Liu, L., Xiaohui, D., Zhang, J., Liu, J., & Liu, Y. (2017). Accurate aoi inspection of resistance in lcd anisotropic conductive film bonding using differential interference contrast. Optik-International Journal for Light and Electron Optics, 130, 786–796.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. . In N. Navab, J. Hornegger, W. M. Wells, & A. F. Frangi (Eds.), International conference on medical image computing and computer-assisted intervention (pp. 234–241). Berlin: Springer.
Russakovsky, O., Deng, J., Hao, S., Krause, J., Satheesh, S., & Ma, S. (2015). Imagenet large scale visual recognition challenge. International Journal of Computer Vision (IJCV), 115(3), 211–252.
Sezgin, M., & Sankur, B. (2004). Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic imaging, 13(1), 146–166.
Sheng, X., Jia, L., Xiong, Z., Wang, Z., & Ding, H. (2013). Acf-cog interconnection conductivity inspection system using conductive area. Microelectronics Reliability, 53(4), 622–628.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In 2015 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1–9).
Tabernik, D., Šela, S., Skvarč, J., & Skočaj, D. (2019). Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-019-01476-x.
Tran, N., Zhang, X., Xin, L., Shan, B., & Li, M. (2017). De novo peptide sequencing by deep learning. Proceedings of the National Academy of Sciences, 114(31), 8247–8252.
Wang, T., Qiao, M., Zhang, M., Yang, Y., & Snoussi, H. (2018). Data-driven prognostic method based on self-supervised learning approaches for fault detection. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-018-1431-x
Wirges, S., Hartenbach, F., & Stiller, C. (2018). Evidential occupancy grid map augmentation using deep learning. In 2018 IEEE intelligent vehicles symposium (IV) (pp. 668–673). arXiv:1801.05297.
Yen, Y. W., & Lee, C. Y. (2011). ACF particle distribution in COG process. Microelectronics Reliability, 51(3), 676–684.
Yu-ye, C., Ke, X., Zhen-xiong, G., Jun-jie, H., Chang, L., & Song-yan, C. (2017). Detection of conducting particles bonding in the circuit of liquid crystal display. Chinese Journal of Liquid Crystals and Displays, 32(7), 553–559.
Zan, T., Liu, Z., Wang, H., Wang, M., & Gao, X. (2019). Control chart pattern recognition using the convolutional neural network. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-019-01473-0
Zhang, Z., Liu, Q., & Wang, Y. (2018). Road extraction by deep residual u-net. IEEE Geoscience and Remote Sensing Letters, 15(5), 749–753.
Zhang, Z., Luo, P., Loy, C. C., & Tang, X. (2016). Learning deep representation for face alignment with auxiliary attributes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(5), 918–930.
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017). Pyramid scene parsing network. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2881–2890).
Zhao, Z., Li, Y., Liu, C., & Gao, J. (2019). On-line part deformation prediction based on deep learning. Journal of Intelligent Manufacturing (pp. 1–14).
Acknowledgements
Eryun Liu and Zhiyu Xiang’s research were supported by the National Natural Science Foundation of China under Grant No. U1709214 and by the Fundamental Research Funds for the Central Universities.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, E., Chen, K., Xiang, Z. et al. Conductive particle detection via deep learning for ACF bonding in TFT-LCD manufacturing. J Intell Manuf 31, 1037–1049 (2020). https://doi.org/10.1007/s10845-019-01494-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-019-01494-9