Robust minimization of lighting variation for real-time defect detection
Introduction
When an image is captured by a camera, its quality depends on a number of factors, such as illumination condition, exposure time of the film or sensor, optical aberration, and sensor noise [1]. Ordinarily, for consumer photography, the aberration and noise can be kept minimal. Therefore, two pictures taken under similar illumination and exposure would appear fairly identical. Yet for machine vision, when such images need to be further processed or combined together, the conditions are often more rigid and we need to pay attention to the difference in image capture conditions.
One particularly challenging application that further complicates the issue is in industrial inspection. Our interest here is in semiconductor manufacturing, but the technique we introduce can potentially be used in other industries as well [2]. Semiconductor manufacturing is a complicated task with very stringent requirements [3], [4]. Scrupulous inspection is needed to avoid malfunctioning of the integrated circuits (IC) due to misplacement, contamination, or missing components, a process generally known as defect detection. This is usually performed after every major step in the manufacturing process with automated visual inspection (AVI) techniques by machine vision. While such techniques constantly need to be refined and improved to meet with the ever-shrinking feature size and ever-increasing complexity of the IC, a basic setup can be described as follows. A reference IC is first carefully examined to ensure that it is free from defect. A test IC is then compared against the reference, and we observe if they have any visual difference. This difference indicates the potential areas of defect [5]. Usually, some post-processing steps are required to analyze and classify the difference image, which would eventually result in a report detailing the existence and attributes of the defects. Clearly, the success of this defect detection step depends on an accurate comparison between the test and the reference. This comparison needs to be robust against variations in the image capture conditions. As usually a large volume of ICs has to be inspected, the vision algorithm also needs to be real-time.
Illumination and exposure differences are the two main variations in capture conditions that need to be accounted for [6]. The intensity of an image depends on the nature of illumination, the reflectance of the subject, and the camera setting. We explore the details of the lighting variation and its model in Section 2. This brings us to the need of an illumination normalization scheme [7]. We explore two alternatives, first with a least-squares method in Section 2.1, and then with a linear method in Section 2.2. The latter is seen to be more robust against the existence of defects, and is capable of high speed, due to the efficiency of linear programming. Simulations are provided in Section 3 followed by some concluding remarks.
Section snippets
Lighting variation minimization
Consider that we have taken two images: the first one is with the reference die which we know to be defect-free, while the second one is with the test die that we are going to inspect. Let denote the wavelength of electromagnetic radiation. The first die has reflectance and is subject to illumination at location (x,y), while the second die has reflectance and is subject to illumination at location . The subscripts r and t refer to reference
Simulations
We apply the method described above on bump inspection, which is a critical process in die bonding in semiconductor assembly [3]. Bumps are the electrical and mechanical connection between the die and the substrate, and are formed from processes such as paste-deposition and electroplating. As such, the shape of the bumps may vary, and may have a few potential defects such as missing bumps, bridged bumps, contaminants on or between bumps, and incorrect bump volumes or heights [14].
We test in
Conclusions
In this paper, we have described a linear programming formulation to minimize the lighting variation between two images. This method is particularly useful in inspection for manufacturing processes, where defects could impose significant effects on the mapping of intensity values. Simulation results on bump images in semiconductor assembly have shown support for our approach.
Acknowledgements
The financial support by ASM Assembly Automation Ltd and the Innovation and Technology Fund of the Hong Kong Special Administrative Region Government for this work is gratefully acknowledged.
References (15)
Image restoration in digital photography
IEEE Transactions on Consumer Electronics
(2003)- Lai S-H, Fang M. A novel illumination compensation algorithm for industrial inspection. In: Machine vision applications...
Microchip fabricationa practical guide to semiconductor processing
(2000)- et al.
Semiconductor manufacturing technology
(2001) - et al.
A color vision inspection system for integrated circuit manufacturing
IEEE Transactions on Semiconductor Manufacturing
(1992) Simultaneous homographic and comparametric alignment of multiple exposure-adjusted pictures of the same scene
IEEE Transactions on Image Processing
(2003)- Matsushita Y, Nishino K, Ikeuchi K, Sakauchi M. Illumination normalization with time-dependent intrinsic images for...
Cited by (4)
Holographic Classifier: Deep Learning in Digital Holography for Automatic Micro-objects Classification
2020, IEEE International Conference on Industrial Informatics (INDIN)Non-negative matrix factorization for images with laplacian noise
2008, IEEE Asia-Pacific Conference on Circuits and Systems, Proceedings, APCCASAutomatic white balancing in digital photography
2008, Single-Sensor Imaging: Methods and Applications for Digital CamerasSimultaneous photometric correction and defect detection in semiconductor manufacturing
2006, Proceedings of SPIE - The International Society for Optical Engineering