Abstract
Manual inspection is still widely used for defect detection in optical isolator factories. The manual method is not only inefficient, but also low reliability, and easy to be disturbed by visual fatigue. This paper proposes an automatic defect detection algorithm for optical isolators, which is an extension to a previous conference paper, with more algorithms and detailed description of the detection algorithm added. In the focusing part of the algorithm, an improved Laplace operator which increases the attention of oblique gradient is adopted. In the detection part of the algorithm, the image-enhancement based on adaptive threshold, the line and ring detection based on Hough transform, and process the detection results by clustering method are used. Experiments show that the scheme proposed in this paper can inspect a series of defects on the crystal area of the optical isolator in real time, the manpower demand can be reduced to less than 10 % and the detection accuracy is 93 %.
Zusammenfassung
Zur Fehlererkennung in Fabriken für optische Isolatoren wird häufig immer noch eine manuelle Inspektion eingesetzt. Das manuelle Verfahren ist nicht nur ineffizient, sondern auch wenig zuverlässig und kann leicht durch visuelle Ermüdung gestört werden. Dieser Beitrag schlägt einen automatischen Fehlererkennungsalgorithmus für optische Isolatoren vor, der eine Erweiterung eines früheren Konferenzpapiers darstellt, wobei weitere Algorithmen und eine detaillierte Beschreibung des Erkennungsalgorithmus hinzugefügt wurden. Im Fokussierungsteil des Algorithmus wird ein verbesserter Laplace-Operator verwendet, der die Aufmerksamkeit des schiefen Gradienten erhöht. Im Detektionsteil des Algorithmus werden die Bildverbesserung basierend auf der adaptiven Schwelle, die Linien- und Ringerkennung basierend auf der Hough-Transformation und die Verarbeitung der Detektionsergebnisse durch das Clustering-Verfahren verwendet. Experimente zeigen, dass das in diesem Dokument vorgeschlagene Schema eine Reihe von Defekten auf der Kristallfläche des optischen Isolators in Echtzeit untersuchen kann, der Personalbedarf auf weniger als 10 % reduziert werden kann und die Erkennungsgenauigkeit 93 % beträgt.
About the authors
Tian QIU received the B. Eng. degree in measurement and instrumentation, and M. Sc. and Ph. D degree in circuits and systems, all from University of Science and Technology of China in 2000, 2003 and 2006, respectively. He had worked as an engineer and senior engineer in Samsung Electronics in South Korea from 2006 to 2009; worked as a research associate in University of Kent, UK, from 2009 to 2012; and worked as a leading engineer and research engineer in Imagination Technologies, UK, from 2012 to 2016. He is currently a contract professor in Wuyi University with image processing at the School of Intelligent Manufacture.He has published about 40 papers in academic journals such as IEEE Transactions and the Journal of circuits and systems, most of which have been retrieved by SCI or EI. Among them, one paper published by Google academic display has been cited internationally more than 120 times. 18 patents have been applied for, including 3 authorized U. S. patents. His research interests include image processing and image analysis.
Zhiquan LIN received his B. Eng degree in traffic engineering from Wuyi University in 2019. He is pursuing his M. Sc degree in electronic information in Wuyi University.He has published 2 academic papers. His research interests include image processing, image analysis and computer vision.
Dr. Tsai is a veteran researcher with more than 25 years of experience and more than 60 patents granted in the US, Mainland China and Taiwan. Dr Chen Jung Tsai joined Hong Kong ASTRI in 2005. In recent years, he has led the development of award-winning applications, including head-mounted display, automatic optical inspection, and AI-based defect detection. He holds a PhD in Applied Mechanics and a Master in Applied Mechanics from the National Taiwan University, and a Bachelor in Agriculture from the National Chung-Hsin University in Taiwan.
Chi Shing WONG is a senior lead engineer at one of the biggest Research Institute in Hong Kong named Hong Kong Applied Science and Technology Research Institute. He received bachelor’s degree and Mphil’s degree in Electronic and Information Engineering from The Hong Kong Polytechnic University. He has been working in AOI and automation industry related development for more than eight years.He is interested in machine vision, computer vision, image processing, object segmentation, object recognition and deep learning analysis.
Professor Xin Zhang is working with the Department of Intelligent Manufacturing of Wuyi University. He has undertaken more than 30 national, provincial scientific research projects, won two provincial third prizes for scientific and technological progress, one provincial scientific and technological achievement appraisal, and provincial new products. He has published more than 60 academic papers, of which nearly 50 have been indexed by SCI, EI and ISTP. He also published one textbook and one translation technology book. He is with more than ten patents granted.
Shuaiqi Liu received his Ph. D. degree in Institute of Information Science from Beijing Jiaotong University at 2014 and got B. S. degree from Shandong University of Science and Technology at 2009. At present, he is associate professor in college of Electronic and Information Engineering, Hebei University. And he is also now conducting postdoctoral research in National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences. His research interests include image processing and signal processing.
Honglong Ning is a professor of China South University of Technology. He received BS in metal materials and fabrication in 1993, MS in powder metallurgy in 1999, and PhD in electronic material & packaging in 2004. He worked in Korean Samsung from 2004 to 2013 as a principle researcher, he had published more than 200 research papers and applied more than 200 patents, and his current research interests concern AMLCD, OLED and E-Paper including flexible, print display materials and devices.
References
1. Du, Q., T. Fakhrul, Y. Zhang, J. Hu and C.A. Ross. 2018. Monolithic magneto-optical oxide thin films for on-chip optical isolation. MRS Bulletin 43(6): 413–418.10.1557/mrs.2018.127Search in Google Scholar
2. Xia, C.-C., X.-B. Yan and F. Gao. 2019. Ideal optical isolator with a two-cavity optomechanical system. Optics Communications 451: 197–201.10.1016/j.optcom.2019.06.059Search in Google Scholar
3. Yamaguchi R. and T. Mizumoto. 2018. Low-loss waveguide optical isolator with tapered mode converter and magneto-optical phase shifter for TE mode input. Optics Express 26(16): 21271–21278.10.1364/OE.26.021271Search in Google Scholar PubMed
4. Lin, S., S. Silva and D. Talbayev. 2018. A one-way mirror: high-performance terahertz optical isolator based on magnetoplasmonics. Advanced Optical Materials 6(19): 1800572.10.1002/adom.201800572Search in Google Scholar
5. Amemiya, T., H. Shimizu, M. Yokoyama, P.N. Hai and Y. Nakano. 2007. 1.54-microm TM-mode waveguide optical isolator based on the nonreciprocal-loss phenomenon: device design to reduce insertion loss. Applied Optics 46(23): 5784–5791.10.1364/AO.46.005784Search in Google Scholar
6. Kim, S.A., et al. 2001. Change of visual function and lacrimation among male near-work inspectors in a TV manufacturing plant. Korean Journal of Occupational and Environmental Medicine 13(4): 436–448.10.35371/kjoem.2001.13.4.436Search in Google Scholar
7. Yang, F.S., C.C. Ho and L.C. Chen. 2021. Automated optical inspection system for O-ring based on photometric stereo and machine vision. Applied Sciences 11(6): 2601.10.1109/IS3C50286.2020.00083Search in Google Scholar
8. Abd Al Rahman, M. and A. Mousavi. 2020. A review and analysis of automatic optical inspection and quality monitoring methods in electronics industry. IEEE Access 8: 183192–183271.10.1109/ACCESS.2020.3029127Search in Google Scholar
9. Ye, R., M. Chang, C.S. Pan, C.A. Chiang and J.L. Gabayno. 2018. High-resolution optical inspection system for fast detection and classification of surface defects. International Journal of Optomechatronics 12(1): 1–10.10.1080/15599612.2018.1444829Search in Google Scholar
10. Minami, H., J. Mori, S. Iwai, H. Moriya and N. Watanabe. 2011. Manufacturing and inspection equipment for efficient production of large LCDs. Hitachi Review 60(5): 229.Search in Google Scholar
11. Liao, Q., Y. Dai and J. Liu. 2017, October. Optimal design of high performance linear guideway based on orifice restriction. In: AOPC 2017: 3D measurement technology for intelligent manufacturing. Vol. 10458. International Society for Optics and Photonics, p. 104581P.Search in Google Scholar
12. Li, J., M.D. Levine, X. An, X. Xu and H. He. 2012. Visual saliency based on scale-space analysis in the frequency domain. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(4): 996–1010.10.1109/TPAMI.2012.147Search in Google Scholar PubMed
13. Vijayalakshmi, D., M.K. Nath and O.P. Acharya. 2020. A comprehensive survey on image contrast enhancement techniques in spatial domain. Sensing and Imaging 21(1): 1–40.10.1007/s11220-020-00305-3Search in Google Scholar
14. Remeseiro, B., J. Tarrío-Saavedra, M. Francisco-Fernández, M.G. Penedo, S. Naya and R. Cao. 2019. Automatic detection of defective crankshafts by image analysis and supervised classification. The International Journal of Advanced Manufacturing Technology 105(9): 3761–3777.10.1007/s00170-019-03819-7Search in Google Scholar
15. Li, N., M. Shepperd and Y. Guo. 2020. A systematic review of unsupervised learning techniques for software defect prediction. Information and Software Technology 122: 106287.10.1016/j.infsof.2020.106287Search in Google Scholar
16. Rong-sheng, L., W. Ang, Z. Teng-da and W. Yong-hong. 2018. Review on automated optical (visual) inspection and its applications in defect inspection. Acta Optica Sinica 38: 0815002.10.3788/AOS201838.0815002Search in Google Scholar
17. Wahyuni, S. and R.W. Sembiring. 2020. Digital image processing for detecting yellow lines (patch yellow) in palm oil plant using Sobel algorithm. Journal of Statistical Mechanics: Theory and Experiment 1471(1): 012067.10.1088/1742-6596/1471/1/012067Search in Google Scholar
18. Chen, M.Y. and H.T. Wu. 2021. Real-time intelligent image processing for the internet of things. J Real-Time Image Proc 18: 997–998.10.1007/s11554-021-01149-0Search in Google Scholar
19. Park, I.S., Y. Park, S.H. Oh, J. Yang and S. Chung. 2018. Multifunctional liquid lens for variable focus and zoom. Sensors and Actuators A-physical 273: 317–323.10.1016/j.sna.2018.02.017Search in Google Scholar
20. Fang, C., Y. Cao, D. Jiang and C. Zhang. 2020. Triboelectric effect-modulated varifocal liquid lens. Microsystems and Nanoengineering 6(1): 1773–1788.10.1038/s41378-020-0174-ySearch in Google Scholar PubMed PubMed Central
21. Wang, X., Y. Zhu and J. Ji. 2021. A zoom tracking algorithm based on defocus difference. Journal of Real-Time Image Processing 18(6): 2417–2428.10.1007/s11554-021-01133-8Search in Google Scholar
22. Guangmang, C., Z. Ke-qi, M.C. Lei, X. Zhi-hai and F. Hua-jun. 2019. Micro-image definition evaluation using multi-scale decomposition and gradient absolute value. Opto-Electronic Engineering 46: 180531.Search in Google Scholar
23. Śliwiński, P., P. Wachel and A. Gałęziowski. 2019, March. Stochastic framework for contrast-detection autofocusing. In: Workshop on stochastic models, statistics and their application. Springer, Cham, pp. 385–392.10.1007/978-3-030-28665-1_29Search in Google Scholar
24. Memon, F. and S. Memon. 2015. Image quality assessment for performance evaluation of focus measure operators. Mehran University Research Journal of Engineering and Technology 34(4): 379–386.Search in Google Scholar
25. Yan, X., J. Lei and Z. Zhao. 2020. Multidirectional gradient neighbourhood-weighted image sharpness evaluation algorithm. Mathematical Problems in Engineering 2020.10.1155/2020/7864024Search in Google Scholar
26. Peddigari, V. and N. Kehtarnavaz. 2007. Real-time predictive zossssom tracking for digital still cameras. J Real-Time Image Proc 2: 45–54.10.1007/s11554-007-0036-ySearch in Google Scholar
27. Yu, L., D. Zhang, N. Peng and X. Liang. 2021. Research on the application of binary-like coding and Hough circle detection technology in PCB traceability system. Journal of Ambient Intelligence and Humanized Computing 1–11.10.1007/s12652-020-02655-ySearch in Google Scholar
28. Pan, M.S. and Q. Xiong. 2021. Iris location method based on mathematical morphology and improved Hough transform. Biomedical Engineering: Applications, Basis and Communications 33(01): 2150001.10.4015/S1016237221500010Search in Google Scholar
29. Ta, Q.B. and J.T. Kim. 2020. Monitoring of corroded and loosened bolts in steel structures via deep learning and Hough transforms. Sensors 20(23): 6888.10.3390/s20236888Search in Google Scholar PubMed PubMed Central
30. Chen, W., S. Chen, H. Guo and X. Ni. 2020. Welding flame detection based on color recognition and progressive probabilistic Hough transform. Concurrency and Computation: Practice and Experience 32(19): e5815.10.1002/cpe.5815Search in Google Scholar
31. Yanxue, W.A.N.G. 2020. U. S. Patent No. 10,740,889. Washington, DC: U. S. Patent and Trademark Office.Search in Google Scholar
32. Chen, B., B. Ding and J. Wang. 2020, August. Application of an improved Hough transform and image correction algorithm in ACC. Journal of Physics: Conference Series 1621(1): 012044. IOP Publishing.10.1088/1742-6596/1621/1/012044Search in Google Scholar
33. Khadanga, G. and K. Jain. 2020. Tree census using circular Hough transform and grvi. Procedia Computer Science 171: 389–394.10.1016/j.procs.2020.04.040Search in Google Scholar
© 2022 Walter de Gruyter GmbH, Berlin/Boston
