Elsevier

Real-Time Imaging

Volume 10, Issue 6, December 2004, Pages 365-370
Real-Time Imaging

Robust minimization of lighting variation for real-time defect detection

https://doi.org/10.1016/j.rti.2004.08.001Get rights and content

Abstract

In machine vision applications that involve comparing two images, it is necessary to match the capture conditions, which can affect their graylevels. Illumination and exposure are two important causes for lighting variation that we should compensate for in the resulting images. A standard technique for this purpose is to map one of the images to achieve the smallest mean square error (MSE) between the two. However, applications in defect detection for manufacturing processes are more challenging, because the existence of defects would affect the mapping significantly. In this paper, we present a robust method that is more tolerant to defects, and discuss its formulation as a linear programming to achieve fast implementations. This algorithm is also flexible and capable of incorporating further constraints, such as ensuring non-negativity of the pixel values.

Introduction

When an image is captured by a camera, its quality depends on a number of factors, such as illumination condition, exposure time of the film or sensor, optical aberration, and sensor noise [1]. Ordinarily, for consumer photography, the aberration and noise can be kept minimal. Therefore, two pictures taken under similar illumination and exposure would appear fairly identical. Yet for machine vision, when such images need to be further processed or combined together, the conditions are often more rigid and we need to pay attention to the difference in image capture conditions.

One particularly challenging application that further complicates the issue is in industrial inspection. Our interest here is in semiconductor manufacturing, but the technique we introduce can potentially be used in other industries as well [2]. Semiconductor manufacturing is a complicated task with very stringent requirements [3], [4]. Scrupulous inspection is needed to avoid malfunctioning of the integrated circuits (IC) due to misplacement, contamination, or missing components, a process generally known as defect detection. This is usually performed after every major step in the manufacturing process with automated visual inspection (AVI) techniques by machine vision. While such techniques constantly need to be refined and improved to meet with the ever-shrinking feature size and ever-increasing complexity of the IC, a basic setup can be described as follows. A reference IC is first carefully examined to ensure that it is free from defect. A test IC is then compared against the reference, and we observe if they have any visual difference. This difference indicates the potential areas of defect [5]. Usually, some post-processing steps are required to analyze and classify the difference image, which would eventually result in a report detailing the existence and attributes of the defects. Clearly, the success of this defect detection step depends on an accurate comparison between the test and the reference. This comparison needs to be robust against variations in the image capture conditions. As usually a large volume of ICs has to be inspected, the vision algorithm also needs to be real-time.

Illumination and exposure differences are the two main variations in capture conditions that need to be accounted for [6]. The intensity of an image depends on the nature of illumination, the reflectance of the subject, and the camera setting. We explore the details of the lighting variation and its model in Section 2. This brings us to the need of an illumination normalization scheme [7]. We explore two alternatives, first with a least-squares method in Section 2.1, and then with a linear method in Section 2.2. The latter is seen to be more robust against the existence of defects, and is capable of high speed, due to the efficiency of linear programming. Simulations are provided in Section 3 followed by some concluding remarks.

Section snippets

Lighting variation minimization

Consider that we have taken two images: the first one is with the reference die which we know to be defect-free, while the second one is with the test die that we are going to inspect. Let λ denote the wavelength of electromagnetic radiation. The first die has reflectance Rr(λ;x,y) and is subject to illumination Ir(λ;x,y) at location (x,y), while the second die has reflectance Rt(λ;x^,y^) and is subject to illumination It(λ;x^,y^) at location (x^,y^). The subscripts r and t refer to reference

Simulations

We apply the method described above on bump inspection, which is a critical process in die bonding in semiconductor assembly [3]. Bumps are the electrical and mechanical connection between the die and the substrate, and are formed from processes such as paste-deposition and electroplating. As such, the shape of the bumps may vary, and may have a few potential defects such as missing bumps, bridged bumps, contaminants on or between bumps, and incorrect bump volumes or heights [14].

We test in

Conclusions

In this paper, we have described a linear programming formulation to minimize the lighting variation between two images. This method is particularly useful in inspection for manufacturing processes, where defects could impose significant effects on the mapping of intensity values. Simulation results on bump images in semiconductor assembly have shown support for our approach.

Acknowledgements

The financial support by ASM Assembly Automation Ltd and the Innovation and Technology Fund of the Hong Kong Special Administrative Region Government for this work is gratefully acknowledged.

References (15)

  • E.Y. Lam

    Image restoration in digital photography

    IEEE Transactions on Consumer Electronics

    (2003)
  • Lai S-H, Fang M. A novel illumination compensation algorithm for industrial inspection. In: Machine vision applications...
  • P. Van Zant

    Microchip fabricationa practical guide to semiconductor processing

    (2000)
  • M. Quirk et al.

    Semiconductor manufacturing technology

    (2001)
  • M. Barth et al.

    A color vision inspection system for integrated circuit manufacturing

    IEEE Transactions on Semiconductor Manufacturing

    (1992)
  • F.M. Candocia

    Simultaneous homographic and comparametric alignment of multiple exposure-adjusted pictures of the same scene

    IEEE Transactions on Image Processing

    (2003)
  • Matsushita Y, Nishino K, Ikeuchi K, Sakauchi M. Illumination normalization with time-dependent intrinsic images for...
There are more references available in the full text version of this article.

Cited by (4)

View full text