Robust visible-infrared image matching by exploiting dominant edge orientations

Highlights

• Propose a rotation invariant descriptor for visible and infrared image matching.

• Encode edge information into the descriptor statistically based on multi-orientation and multi-scale Log-Gabor filters.

• Estimate the dominant orientation of the descriptor by using accumulated edge orientations.

• The descriptor is robust to the discrepancies in the electromagnetic wavelengths between visible and infrared images.

Abstract

Finding the correspondences between visible and infrared images is a challenging task due to the image spectral inconsistency which leads to large differences of gradient distributions between these images. To alleviate this problem, we propose a novel feature descriptor for visible and infrared image matching based on Log-Gabor filters. The descriptor employs multi-orientation and multi-scale Log-Gabor filters to encode the edge information statistically. Furthermore, the descriptor provides rotation invariance by estimating the dominant orientation which is based on accumulated edge orientations. The experimental results demonstrate the effectiveness of the proposed rotation invariant descriptor for matching visible and long wave infrared images as compared with state-of-the-art descriptors.

Introduction

Automatic image matching, as a fundamental problem in computer vision and image processing, has been studied and made great progress in recent decades [16], [24], [26], [29], [30], [31], [32], [35]. The matching of visible and infrared images can provide complementary information and play an important role in extensive applications [13], such as video surveillance, urban monitoring, driver assistance systems. However, it still remains some difficulties in matching visible and infrared images automatically because of their discrepancies in the electromagnetic wavelengths. Visible images capture reflected light, whereas infrared images depict thermal radiation. The dramatic appearance changes between the visible and infrared image pairs usually include large gradient differences or missing texture details in one of the images, which will cause the inconsistent description for the correspondences. For instance, Fig. 1 illustrates a visible and infrared image pair exhibiting large changes in appearance. It reveals that SIFT [16], which is highly invariant to a range of geometric and photometric transformations, performs poorly in such intractable situations as shown in Fig. 1.

The traditional approaches addressing the large appearance differences between images usually use the mutual information [28] or the sum of squared differences (SSD) [21], [25], which are known as the area-based image matching methods. These methods can match image patches accurately as all pixels are taken into account. However, these methods are sensitive to geometric distortions and computationally expensive.

It would be more challenging for identifying two image patches when there exists a rotation transformation between them. For visible image matching, extracting feature vectors [16] (a.k.a descriptors) from images is effective and computationally efficient. The rotation differences between image patches are solved by estimating the dominant orientations from the image patches. The dominant orientations can be used to re-arrange the descriptors, which would make the descriptors invariant with respect to image rotations.

However, different from visible image matching, the low-level information (e.g., gradients) will be affected by the differences between visible images and infrared images, thus causing incorrect dominant orientation estimations and mis-matches. For example, Fig. 2a shows the similar gradient distributions between visible images which could estimate the similar dominant orientations, leading to good matching performance [16]. While in Fig. 2b, the disparate dominant orientations estimated by gradients between the visible and infrared image pair are misleading for correspondence recognition. Other feature descriptors based on either straight line segments [11], [15] or edges [1], [2], [20] show improved matching accuracy in some extent comparing with the gradient based methods. However, those descriptors are not robust to image rotation invariance.

By investigating visible and infrared images, we notice that although they have dissimilar gradients at pixel level, the global structure and shape features such as edges tend to remain reasonably invariant and the orientation of edges can be a cue to establish the similarity between visible and infrared images. For the corresponding image patches between visible and infrared images, the distributions of local edge orientations are similar. For instance, pixels on horizontal edges in one image also lie on the corresponding horizontal edges in the other image, which can establish the similar relations between corresponding pixels. We describe the pixel using the orientation of edges instead of intensity or gradient and construct the orientation histograms of pixels statistically to design the descriptor. When there exist relatively rotation distortions between visible and infrared images, all the edges in one image have the same rotation angle comparing with the corresponding edges in the other image. Among the different orientations within a local image patch, we use the accumulated maximum orientation as the dominant orientation. Multi-orientation and multi-scale Log-Gabor filters can be a good tool to extract edge information [17].

Motivated by the observation mentioned above, we propose a new rotation invariant descriptor for visible and infrared image matching on the basis of multi-orientation and multi-scale Log-Gabor filters. Our proposed descriptor can handle the rotations between visible and infrared images by estimating the reliable dominant orientation. We evaluate and compare the proposed descriptor with several state-of-the-art descriptors in challenging datasets.

The rest of this paper is organized as following: Section 2 gives a brief overview of related work. Section 3 presents the proposed rotation invariant descriptor for matching visible and infrared images in details, followed by the experiments in Section 4. Finally, conclusions are presented in Section 5.