Efficient template matching for multi-channel images

doi:10.1016/j.patrec.2010.12.004

Pattern Recognition Letters

Volume 32, Issue 5, 1 April 2011, Pages 694-700

https://doi.org/10.1016/j.patrec.2010.12.004 Get rights and content

Abstract

Template matching is a computationally intensive problem aimed at locating a template within a image. When dealing with images having more than one channel, the computational burden becomes even more dramatic. For this reason, in this paper we investigate on a methodology to speed-up template matching on multi-channel images without deteriorating the outcome of the search. In particular, we propose a fast, exhaustive technique based on the Zero-mean Normalized Cross-Correlation (ZNCC) inspired from previous work related to grayscale images. Experimental testing performed over thousands of template matching instances demonstrates the efficiency of our proposal.

Research highlights

► A methodology to speed-up template matching on multi-channel images without approximating the outcome of the search is investigated. ► A fast, exhaustive technique based on the Zero-mean Normalized Cross-Correlation (ZNCC) inspired from previous work related to grayscale images is proposed. ► An extensive experimental validation of the proposed algorithm is performed over 9000 template matching instances, demonstrating the effectiveness of our proposal.

Introduction

Template matching (Zitová and Flusser, 2003, Goshtasby, 2005) is a computationally intensive problem that aims at locating a template T within an image I. The basic approach, or Full-Search (FS), simply computes a similarity measure between T and each possible portion of the image, referred to as the image subwindow or candidate, of the same size as T. Then, the candidate reporting the highest similarity locates the position of the template in the image.

While numerous exhaustive techniques have been recently proposed to speed-up this task based on grayscale (i.e. single-channel) images (Mattoccia et al., 2008a, Mattoccia et al., 2008b, Tombari et al., 2009, Wei and Lai, 2007, Mahmood and Khan, 2007a, Mahmood and Khan, 2007b, Pan and Wei, 2008, Alkhansari, 2001), to the best of our knowledge the problem of performing fast exhaustive template matching with multi-channel images has not been addressed yet. Today, multi-channel images are used in an increasing number of computer vision tasks. Two typical examples are represented by color images and multi-spectral images: the former are commonly deployed for tasks such as defect detection and industrial quality control (e.g. Tsai et al., 2003), the latter for tasks such as medical image registration and remote sensing data alignment (Guo et al., 2009, Kern and Pattichis, 2007, Pope and Theiler, 2003).

In this paper we propose a fast template matching technique for multi-channel images based on the Zero-mean Normalized Cross-Correlation (ZNCC), a measure which is widely used for grayscale template matching (Mattoccia et al., 2008b, Mahmood and Khan, 2007a, Mahmood and Khan, 2007b). The proposed technique is exhaustive, i.e. it guarantees to find always the global maximum of the correlation function, and it is inspired by the ZEBC technique (Mattoccia et al., 2008b), previously proposed for grayscale images. The basic idea is to find efficiently computable upper bounding functions of the ZNCC that allow rapid detection of mismatching candidates.

The paper is structured as follows. Section 2 introduces the notation and the definitions used throughout the paper. Then, Sections 3 Upper-bounding functions, 4 The proposed algorithm describe the mathematical grounds and the proposed algorithm. Section 5 shows experimental results on a vast dataset composed of color images. Finally, conclusions are drawn in Section 6.

Section snippets

Notation

Let T and I(x, y) be respectively a template and an image subwindow with C channels, the template size being M × N: $T = [T_{1}, \dots, T_{k}, \dots, T_{c}]^{T} .$ $I (x, y) = [I_{1} (x, y), \dots, I_{k} (x, y) \dots, I_{c} (x, y)]^{T},$ where $T_{k} = [T_{k} (1, 1), \dots, T_{k} (M, N)]^{T},$ $I_{k} (x, y) = [I_{k} (x + 1, y + 1), \dots, I_{k} (x + N, y + M)]^{T} .$ The ZNCC on multi-channel images can be defined as follows: $ζ (x, y) ≜ \frac{\sum_{k = 1}^{C} (I_{k} (x, y) - μ_{I} (x, y)) \circ (T_{k} - μ_{T})}{\sqrt{\sum_{k = 1}^{C} ‖ I_{k} (x, y) - μ_{I} (x, y)) ‖_{2}^{2}} \cdot \sqrt{\sum_{k = 1}^{C} ‖ T_{k} - μ_{T} ‖_{2}^{2}}}$ with ∘ denoting the dot product between two M × N vectors, and μ_T, μ_I(x, y) the mean values of respectively the template and the

Upper-bounding functions

Given the definition of partial correlation term $ψ_{ZC} (x, y) |_{ρ}^{θ}$ in (13), we can determine two different upper-bounds of this term that can be usefully deployed to establish sufficient conditions aimed at efficient detection of mismatching candidates, that is, candidates that can not improve the current ZNCC maximum.

The proposed algorithm

This Section proposes a novel algorithm for fast and exhaustive template matching on multi-channel images inspired by the ZEBC approach described in (Mattoccia et al., 2008b). First of all, we partition image subwindow and template as proposed in (Mattoccia et al., 2008b), that is, we divide them into an equal number of r rectangular regions (assuming that N is a multiple of r).¹ Each region is characterized

Experimental results

This section presents an experimental evaluation aimed at assessing the effectiveness of the proposed algorithm in speeding-up ZNCC-based template matching on multi-channel images. In particular, we tested the performance of the proposed algorithm on a dataset of color images of various contents. In particular, they were chosen from 3 different image databases: one concerning mainly indoor, urban and natural environments (MIT-CSAIL database of objects and scenes, 2005), one concerning medical

Conclusions

We have presented an approach that performs fast exhaustive template matching with multi-channel images based on the ZNCC function. The proposed method generalizes the ZEBC technique (Mattoccia et al., 2008b) to the case of images with a generic number of channels. Moreover, though not shown here for the sake of brevity, the proposed method can be easily adapted to work with the NCC function. Experimental results have demonstrated the effectiveness of the proposed bounding functions to rapidly

References (19)

D. Tsai et al.
The evaluation of normalized cross correlations for defect detection
Pattern Recognit. Lett.
(2003)
B. Zitová et al.
Image registration methods: A survey
Image Vision Comput.
(2003)
M. Alkhansari
A fast globally optimal algorithm for template matching using low-resolution pruning
IEEE Trans. Image Process.
(2001)
F. Crow
Summed-area tables for texture mapping
Comput. Graphics
(1984)
Lukas Corpus Medical Image database, 2002....
A. Goshtasby
2D and 3D Image Registration for Medical Remote, Sensing and Industrial Applications
(2005)
Guo, J., Sun, H., Zhu, C., Xiao, S., 2009. Multispectral remote sensing image registration based on maximally stable...
J. Kern et al.
Robust multispectral image registration using mutual-information models
IEEE Trans. Geosci. Remote Sens.
(2007)
Mahmood, A., Khan, S., 2007a. Early termination algorithms for correlation coefficient based block matching. In:...

There are more references available in the full text version of this article.

Cited by (33)

A vision-based method for estimating shallow cable tension via vibrational properties
2022, Measurement: Journal of the International Measurement Confederation
Citation Excerpt :
The camera calibration will be introduced in Section 3.1 for obtaining the scaling factor that realizes the conversion of image coordinates to real-world coordinates. A template matching algorithm [33] was optimized to track the vibrational displacement of vision targets from each frame of images as shown in Fig. 4. Two load cells are installed at both ends of the cable to compare the cable tension value with the data calculated from the camera.
Shallow cable is the component of cable-supported glass roof systems, which has been widely used in modern building constructions. In this research, a vision-based method is proposed for estimating tension values of shallow cables subjected to harmonic excitations. To validate the accuracy of the proposed low-cost, easy-to-operate, and multipoint synchronous measurement method, vision-based results are compared with the data obtained from contact sensors, and the error is within 2%. In addition, the data-driven stochastic subspace identification (Data-SSI) is employed to identify the first-order modal shapes of the shallow cable by vision-based data, and the results have a precise match with the computed shape of the numerical analysis. To investigate the practicality of the proposed method, the uncertainty evaluation for shallow cable tensions obtained by vision-based data is conducted based on the Monte Carlo method (MCM). These results demonstrate that the method proposed in this research is effective for capturing vibrational properties and estimating shallow cable tension.
Defect detection methods using boolean functions and the φ-coefficient between bit-plane slices
2021, Optics and Lasers in Engineering
Citation Excerpt :
In particular, automated defect detection (ADD) systems using 2-D signals (images) are badly required to meet the urgent needs for modern industries and to accelerate productions, while maintaining the highest possible degree of accuracy, see e.g. [1–3]. In recent years, many ADD systems have been established in various industries such as: ceramic tiles [4,5], fabrics [6,7], paper industry [8], solar wafers [9], OCR [10], wood industry [11] and printed circuit boards (PCBs) [12–18]. The normalized cross correlation (NCC) has been extensively considered as a principal tool in various AOI systems, see e.g. [3,10,19,20] and the references cited therein.
We establish two automated defect detection (ADD) algorithms and apply them to detect faults in printed circuit boards (PCBs). Both techniques are referential and are implemented on the binary bit-plane images of the PCBs. In the first algorithm we measure the association between reference and inspected images via the $φ$ -correlation coefficient of percentage statistics, while in the second approach we apply a carefully selected boolean function, as well as a smoothing median filter. Both techniques show a high accuracy, which is comparable to the state-of-the-art techniques, but in a notably faster time. For instance, the boolean function approach is faster than the use of the normalized cross correlation (NCC) by $1700 %$ and it is faster than the use of a fast form of the NCC by $700 %,$ while the use of the $φ$ -correlation coefficient accelerates the procedure by $500 %$ and $200 %$ for the use of the NCC and some of its fast forms respectively.
Photovoltaic panel extraction from very high-resolution aerial imagery using region–line primitive association analysis and template matching
2018, ISPRS Journal of Photogrammetry and Remote Sensing
Citation Excerpt :
After tuning, the parameter is 0.1. Inputs C1 and C2 of SSIM used in the template matching were 0.3 and 0.7, respectively, which are the default settings used in (Mattoccia et al., 2011). In each experimental area, we set the similarity threshold from small to large with a step of 0.05 and draw the accuracy curves.
In object-based image analysis (OBIA), object classification performance is jointly determined by image segmentation, sample or rule setting, and classifiers. Typically, as a crucial step to obtain object primitives, image segmentation quality significantly influences subsequent feature extraction and analyses. By contrast, template matching extracts specific objects from images and prevents shape defects caused by image segmentation. However, creating or editing templates is tedious and sometimes results in incomplete or inaccurate templates. In this study, we combine OBIA and template matching techniques to address these problems and aim for accurate photovoltaic panel (PVP) extraction from very high-resolution (VHR) aerial imagery. The proposed method is based on the previously proposed region–line primitive association framework, in which complementary information between region (segment) and line (straight line) primitives is utilized to achieve a more powerful performance than routine OBIA. Several novel concepts, including the mutual fitting ratio and best-fitting template based on region–line primitive association analyses, are proposed. Automatic template generation and matching method for PVP extraction from VHR imagery are designed for concept and model validation. Results show that the proposed method can successfully extract PVPs without any user-specified matching template or training sample. High user independency and accuracy are the main characteristics of the proposed method in comparison with routine OBIA and template matching techniques.
Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection – A review
2018, Engineering Structures
Citation Excerpt :
Computer vision-based displacement sensors are primarily enabled by the template matching technique, one of the most effective image processing techniques for tracking objects. Template matching is a computationally intensive process that aims at locating a template within an image [55]. As illustrated in Fig. 2, the technique involves two primary components: (1) the template image T; and (2) the source image I in which a match to the template image is expected to find.
To address the limitations of current sensor systems for field applications, the research community has been actively exploring new technologies that can advance the state-of-the-practice in structural health monitoring (SHM). Thanks to the rapid advances in computer vision, the camera-based noncontact vision sensor has emerged as a promising alternative to conventional contact sensors for structural dynamic response measurement and health monitoring. Significant advantages of the vision sensor include its low cost, ease of setup and operation, and flexibility to extract displacements of any points on the structure from a single video measurement. This review paper is intended to summarize the collective experience that the research community has gained from the recent development and validation of the vision-based sensors for structural dynamic response measurement and SHM. General principles of the vision sensor systems are firstly presented by reviewing different template matching techniques for tracking targets, coordinate conversion methods for determining calibration factors to convert image pixel displacements to physical displacements, measurements by tracking artificial targets vs. natural targets, measurements in real time vs. by post-processing, etc. Then the paper reviews laboratory and filed experimentations carried out to evaluate the performance of the vision sensors, followed by a discussion on measurement error sources and mitigation methods. Finally, applications of the measured displacement data for SHM are reviewed, including examples of structural modal property identification, structural model updating, damage detection, and cable force estimation.
An accelerating CPU based correlation-based image alignment for real-time automatic optical inspection
2016, Computers and Electrical Engineering
Citation Excerpt :
Though the several efficient methods are used, the computation burden is still too expensive to calculate the similarity between a template and a block in search image. To simplify the computation procedure, an elimination strategy based on the upper bounds has been applied in the full exhaustive search template matching procedure to rapidly skip the dissimilar candidates in monochrome level images [2] and color images [3]. For the elimination strategy performance, tight bounds are proposed to speed up the correlation computation in [4].
This paper proposes an accelerating correlation-based image alignment using CPUs for time-critical applications in automatic optical inspection (AOI). In order to improve computation efficiency, the image pyramid search scheme is combined with the parallel computation. The image pyramid search scheme is employed first to quickly find certain objects in both monochrome and color images with rotation, translation and scaling. Sub-pixel accuracy is then used to attain the more accurate results at the sub-pixel level. In our experimental results, rotation accuracy is smaller than 0.218°, and the speed is increased between 277 and 20,841 times. According to translation, rotation and scaling tests, the errors of rotation, translation and scaling are 0.2°, 2.07 pixel and 0.55%, respectively. These results show that the proposed method is suitable for dealing with correlation-based image alignment for time-critical applications in automatic optical inspection.
Conveyor belt automatic vaccine injection system (AVIS) for flatfish based on computer vision analysis of fish shape
2013, Aquacultural Engineering
Citation Excerpt :
In the case of the template matching vaccine algorithm, in particular, the users need to perform complicated tasks such as designations of search groups and needle insertion lengths, and need many template images of flatfish to deliver the correct vaccine injections. Moreover, image processes using template matching or pattern matching techniques (Kim, 2010; Li et al., 2010; Mattoccia et al., 2011) need a considerable amount of image processing time because they perform a comparative analysis with respect to specific templates or patterns in all the areas of the captured images. The proposed conveyor belt AVIS has resolved issues, e.g., when flatfish are caught by a needle, and has vaccinated by reciprocating motion of servo on one line.
Traditionally, vaccination of fish is carried out manually which requires considerable cost, time and health risk. In particular, because a person has to inject a number of fish in a such short period of time, the injector is under stress and in exposed to risk of self-injection, in which he/she may insert a syringe into his/her own hand. In order to resolve this issue, fast, accurate and automated vaccination systems have been developed in countries with an advanced aquaculture industry, including Norway, Denmark, the Netherlands, and the United States. However, this development is limited to the development of systems for round fish because round fish like salmon are mostly farmed in these countries. Thus, different automated vaccination systems for flatfish need to be developed in countries where flatfish aquaculture is prevalent. In this study, we have demonstrated a conveyor belt automatic vaccine injection system (AVIS) for flatfish based on a vaccination algorithm utilizing computer vision analysis of fish shape. This vaccination algorithm is designed according to correlations derived from a statistical analysis of flatfish's body height, length, and thickness, and the location of the vaccine injection so that it can be used for vaccine injection without any additional separate presets. The proposed AVIS allowed for an injection rate of 2800 per h on average. The vaccination of flatfish with a body length of up to 500 mm was possible, even when the orientation of the fish was arbitrary. The injection errors in various sizes of flatfish ranged from 0 to 1.9 mm.

View all citing articles on Scopus

View full text

Efficient template matching for multi-channel images

Abstract

Research highlights

Introduction

Section snippets

Notation

Upper-bounding functions

The proposed algorithm

Experimental results

Conclusions

Pattern Recognit. Lett.

Image Vision Comput.

A fast globally optimal algorithm for template matching using low-resolution pruning

IEEE Trans. Image Process.

Summed-area tables for texture mapping

Comput. Graphics

2D and 3D Image Registration for Medical Remote, Sensing and Industrial Applications

Robust multispectral image registration using mutual-information models

IEEE Trans. Geosci. Remote Sens.