A dynamic generating graphical model for point-sets matching

Abstract

This paper presents a new dynamic generating graphical model for point-sets matching. The existing algorithms on graphical models are quite robust to noise but are susceptible to the effects of outliers. We investigated the influences of separators on point-sets matching in inference of graphical models theoretically and found that the separators that consist of outliers will interfere with message-passing, which will directly lead to failure of the existing methods. Because of this, in order to minimize the outliers in the separators, we propose a new algorithm for generating a graphical model and a corresponding Junction Tree for point-sets matching. A bi-mapping algorithm is also introduced to solve the problem of multi-mapping caused by outliers. Experiments were carried out on both synthetical data and real-world data with point-sets extracted by the Harris corner detector. The results show that the proposed algorithm is significantly more stable and possesses higher accuracy of point-sets matching, which can overcome the limitation of the sensitivity of outliers in the existing graphical models.

Introduction

Point-sets matching is a fundamental method for graph matching, which is a task of pivotal importance in high level computer vision with many applications such as image retrieval [1], [2], object tracking [3], panoramic mosaic [4], [5], object recognition [6], [7], computational biology and computational chemistry [8], [9].

Point-sets matching consists of finding correspondences between two sets of interest points, which are extracted from two partially overlapping images taken from different views. In this paper, the problem of point-sets matching is defined as follows. Assume we have two images to be matched, D and T, which are defined as the data image and target image. The set of interest points derived from the data image is denoted as ${\mathit{S}}_1=\{D_1\text{,}{D}_2\text{,}\dots \text{,}{D}_{N_{\text{D}}}\}$ and that from the target image as ${\mathit{S}}_2=\{T_1\text{,}{T}_2\text{,}\dots \text{,}{T}_{N_{\text{T}}}\}$, with N_D and N_T as the number of points in each set. The goal is to find the best mapping between a subset of S₁ and a subset of S₂. Here we follow the principle that it is better to have a small number of matches than to have wrong matches [10].

Many researchers have attempted to obtain more accurate results and reduce the complexity of the computation. In the computer vision literature there have been several categories of solutions including spectral methods [11], [12], [13], tree search [14], [15], [16], [17], relaxation labeling [18], [19], [20], [21] and various others [22], [23], [24]. Based on the spectral graph theory [25], [26], the main idea behind spectral methods is that the eigenvalues and the eigenvectors of the adjacency matrix of a graph are invariant with respect to node permutations [27]. An important limitation of spectral methods is that when there are spurious nodes and edges in the graphs under study, the performance will soon break down. Tree search methods generate a matching tree by iteratively adding new pairs of corresponding nodes from the initial tree (usually empty) and make comparison after each addition of nodes to eliminate incorrect assignment. The number of possible assignments is very large if there are many nodes to be matched. Relaxation labeling algorithms assign each node in the data image a label of probability, which can indicate which node in the target image is the best matching result for it. The initial probability is computed by local information of each node, and then it is iteratively updated taking into account the neighboring nodes until a global maximum probability of each node is reached. The final probability will determine for each node in the data image which node of the other image it corresponds to. Relaxation labeling methods have a much lower computational complexity than the tree search method.

Our work is based on the theory of probabilistic graphical models. Inference in graphical models is a method of global optimality and is optimal in a maximum a posteriori sense, but is limited in application due to its NP-complete [28] complexity. Recently, Caetano et al. [29] presented a probabilistic graphical model for point-sets matching, which was proven to be optimal in the maximum a posteriori sense and has polynomial time dependency on the point set sizes. Comparison [30] has been made between this method and standard probabilistic relaxation labeling (PRL) [21] using different forms of point metrics and under different levels of additive noise. The comparison showed that Caetano’s method is more effective than PRL.

The constraint to Caetano’s model is that the mapping must be a total function: every point in the data image must be mapped to one point in the target image, that is, no outliers are allowed. But in many applications such as panoramic mosaics, this constraint may not be applicable; for example, there may be more or fewer outliers existing in the real world. The outliers are the feature points in either image that have no counterparts in the other one, while “signal” points in this paper are defined as feature points which are not outliers. Fig. 1 shows two images to be mapped (Fig. 1a is defined as a data image, while Fig. 1b is defined as a target image), with feature points detected by the Harris corner detector. In these two images, “signal” points are denoted by ∗, and outliers are denoted by □. Theoretically, the point matching between the two images can not be solved using Caetano’s model. The experimental results in Section 4 show that the performance of Caetano’s algorithm worsens when there are more outliers increase in the data image.

In this paper, we discuss the reason that Caetano’s algorithm soon breaks down when there are outliers in the data image, and draw the conclusion that the selection of the separators in the graphical models will greatly influence the result of the mapping. We propose a new algorithm is brought forward to generate graphical models that can ensure an optimal selection of separators. The results demonstrate that the proposed technique outperforms Caetano’s model.