A Bayesian weighting principle for the fundamental matrix estimation☆
Introduction
Matching two images of a single scene is a fundamental problem in computer vision. It is a part of stereo, motion, and object recognition problems which are equivalent in the sense that they all share the same geometrical constraint, the epipolar constraint. Fundamental matrix is a generalization of the essential matrix (Longuet-Higgins, 1981) and it consists of the minimal amount of information between two images to construct the epipolar geometry between them. Importance of the fundamental matrix in computer vision is crucial due to its versatility and applicability to many difficult problems (Csurka et al., 1997).
Estimation of the fundamental matrix has been intensively studied during the last two decades. The best known robust methods are probably the M-estimators and the Least Median of Squares (LMedS), but they still suffer either from outliers or Gaussian noise in data (Xu and Zhang, 1996). In this paper, a new approach is proposed to cope with the disturbing effects of both outliers and Gaussian noise in data. Our experiments with both synthetic and real data show that our method gives better results compared to the previous methods if the affine camera model is assumed. However, when the full perspective model is used it will be shown that the problem becomes much more difficult because it is difficult to approximate the form of the residual distributions precisely.
The novelty of our proposed method is to cope with the major drawback of the all the known robust F-matrix estimation methods: in fact, they (a) either do not use the knowledge that the relevant matching points are normally distributed or (b) violate this assumption when the outliers are discarded. We will show that even a false match, which are usually considered to be equivalent to an outlier, can have a small residual. Therefore it would be impossible to classify the found matching points to relevant and false without an error. Moreover, the classification procedure would violate the normal distribution assumption because the relevant classified matches would not follow the Gaussian distribution anymore.
Section snippets
Background of the F-matrix estimation
With the full perspective projection model the simplest linear solution for the F-matrix may be obtained by seven matched point pairs between two images because it has only seven degrees of freedom. In this case, the solution is necessarily not unique but three different solutions may exist (Xu and Zhang, 1996, Huang and Netravali, 1994).
A unique solution is obtained if eight or more point matches are used, but because the problem is then over-determined an approximate solution must be sought,
Bayesian weighting principle
From the methods presented above, only the robust methods take outliers into account, but they do not work well for at least the following reasons. The M-estimators try to blindly compensate the effect of outliers by replacing the Gaussian distribution assumption by a long tailed distribution. The performance of the M-estimators therefore depends on how well the new distribution corresponds to the actual residual which is, however, unknown a priori. On the other hand, the LMedS method does not
Affine model experiments
When the affine camera model is considered, the estimation principle presented above is easy to implement because the Gaussian distribution assumption for false matches is a good approximation. This follows from the fact that the physical residual of false matches is defined asNow, because the residual ϵf may be considered as a sum of four random variables () the normal distribution assumption is reasonable on the basis of the central limit theorem which states that
Full perspective projection model experiments
When the full perspective projection model is used we should also use a physical criterion such as (2), i.e., minimize the square distance between the matched points and the corresponding epipolar lines. Then, a Gaussian distribution assumption is made to the residual of relevant point pairs. However, for false matches this is unfortunately not a good approximation and the use of the Bayesian weighting principle is therefore more difficult than with the affine camera model.
We first tried to
Conclusions
We have proposed a new approach to the fundamental matrix estimation problem. The well-known approaches such as M-estimators or the LMedS are based on either selecting a new long tailed distribution for the residual or random sampling of points and minimizing the squared median of the resulting residuals. These heuristical approaches, however, do not accurately model the residual distributions and therefore the F-matrix estimates are biased.
Our approach, the Bayesian weighting principle,
References (14)
- et al.
Characterizing the uncertainty of the fundamental matrix
Computer Vision Image Understanding
(1997) - Beardsley, P., Zisserman, A., Murray, D., 1994. Navigation using affine structure from motion. In: Computer...
- Boufama, B., Mohr, R. 1995. Epipole and fundamental matrix estimation using virtual parallax. In: Proceedings of the...
- Hartley, R.I. 1993. Euclidean reconstruction from uncalibrated views. In: Proceedings of the Second Europe–US Workshop...
In defense of the eight-point algorithm
IEEE Trans. Pattern Anal. Machine Intell.
(1997)- et al.
Motion and structure from feature correspondences: a review
Proc. IEEE
(1994) - INRIA Syntim Project. 1999....
Cited by (0)
- ☆
This work is supported by the GETA Graduate School and Academy of Finland, Research Centre for Computational Science and Engineering, project. no. 44897 (Finnish Centre of Excellence Programme 2000–2005).