Robust epipolar geometry estimation using genetic algorithm

doi:10.1016/S0167-8655(98)00032-4

Pattern Recognition Letters

Volume 19, Issue 9, July 1998, Pages 829-838

https://doi.org/10.1016/S0167-8655(98)00032-4 Get rights and content

Abstract

Epipolar geometry is an important constraint to establish the correspondences in stereo vision. The 3×3 fundamental matrix describes the epipolar geometry between two uncalibrated images. In this paper, we formulate the epipolar geometry estimation as a global optimization problem, and then we present a genetic algorithm for parameter searching. Experiments with simulated and real data show that our algorithm performs very well in terms of robustness to outliers, rate of convergence and quality of the final estimation.

Introduction

Matching different views of a single scene remains one of the most difficult problems in computer vision. Two images of the same scene are related by the epipolar geometry constraint, which is the only geometrical constraint available regardless of specific objects. In this way, epipolar geometry plays an important role in stereo vision. Its recent applications cover 3D reconstruction (Faugeras, 1992; Shashua, 1993), camera self-calibration (Maybank and Faugeras, 1992; Ma, 1996), stereo analysis (Hartley and Gupta, 1993; Robert and Hebert, 1993), motion segmentation (Nishimura et al., 1990; Sinclair et al., 1994), image synthesis (Laveau and Faugeras, 1994), etc. Epipolar geometry between two uncalibrated images can be computed by a certain number of corresponding points. A great deal of literature exists on this subject (Longuet-Higgins, 1981; Zhang et al., 1995; Hartley, 1995; Xu and Zhang, 1996). The common methods generally fall into three categories: linear method, nonlinear iterative method, and robust estimation method, such as M-estimator or LMedS.

In most applications, the corresponding points are inevitably corrupted by outliers and noise, such as false matches and badly located points. Hence robustness to outliers is not trivial if algorithms can be utilized in engineering and scientific problems. By robustness, we mean that the performance of algorithm should not be affected significantly by small deviations from the assumed model and it should not deteriorate drastically due to outlier and noise. M-estimator or LMedS is more robust than two other methods since the influence of outliers has been reduced in M-estimator or they are simply discarded in LMedS. However, these two approaches work poorly when outlier proportion is higher than a half.

Robust parameter estimation is usually formulated as a non-convex optimization problem. In the past few years, the optimization technique is primarily based on gradient descent or random sample. Gradient descent can become stuck into local minima easily, which is far away from the true global optimum. Random sample, upon which almost all of highly robust estimator based, can avoid the local minima in some extent. But it is potentially time-consuming particularly when a large number of outliers are involved. Another drawback for random sample lies in the fact that outlier proportion often can not be known in advance. Thus it is difficult for this technique to determine how many subsamples are required to guarantee finding global optimum.

The aim of this paper is how to use genetic algorithm to alleviate the above problems. Genetic algorithm is a global optimization algorithm, which can effectively avoid falling in local minima. Besides, unlike random sample, GA has a good criterion for convergence. Finally, as we will demonstrate later GA is more efficient than random sample in computation efficiency. The rest of this paper is organized as follows. In Section 2, we first introduce the epipolar geometry constraints under full perspective, and then show how to formulate robust parameter estimation into a global optimization problem. Section 3gives a brief review of genetic algorithm. The application of GA in the robust epipolar geometry estimation is presented in Section 4. In Section 5, we give our experiment's results in detail. We conclude our paper in Section 6.

Section snippets

Problem formulation

In this section the epipolar geometry is first introduced. Neither the intrinsic parameters of the images nor the extrinsic parameters are assumed to be known. Then, we will formulate robust epipolar geometry estimation as a global optimization problem.

Genetic algorithm: a review

Genetic algorithm (Holland, 1975; Goldberg, 1989; Michalewicz, 1992) is a stochastic optimization technique that mimics the principles of natural evolution. Its recent applications in computer vision include adaptive image segmentation (Bhanu et al., 1995); edge detection (Bhandarkar et al., 1994); morphological filter design (Harvey and Marshall, 1996), etc.

According to the genetic evolution theory, stronger and fitter individuals have better chances of survival than weaker ones. Offsprings

Genetic algorithm: application in epipolar geometry estimation

The input to our algorithm is corresponding points set $D={y_{i} | i=1,…,N}$ , where parameter vector $y_{i} =(u_{i},v_{i},u′_{i},v′_{i})$ represents corresponding point. The output is the best minimal subset and an inlier subset that is consistent with this minimal subset.

Experimental results

In this section, we present experimental results using genetic algorithm with epipolar geometry estimation under full perspective. To compute the quality of the estimated fundamental matrix $F$ , we use the following evaluation formula: $Q_{F} = ∑^{N_{in}}_{i=1} d(m ̃_{i}, F m ̃ ′_{i}) N_{in},$ where N_in is the number of all the inliers, d is the point-to-line Euclidean distance expressed in pixels.
Note. This evaluation formula is carried out using all matched points, except outliers, and not just the ones that were used to

Conclusion

In this paper, we propose a generic robust parameter estimation method. Its application in epipolar geometry estimation is then reported. We also make the detailed comparison between our methods and other typical existing methods in this specific problem. Comparing with previous approaches, the proposed one has the following main advantages:

•
It is very robust to outliers. In fact, as long as the number of inliers is higher than the minimum number to give a unique solution, our approach can

References (22)

C.A. Ankenbrandt et al.
Scene recognition using genetic algorithms with sematic nets
Pattern Recognition Letters
(1990)
S.M. Bhandarkar et al.
An edge detection technique using genetic algorithm-based optimization
Pattern Recognition
(1994)
N.R. Harvey et al.
The use of genetic algorithms in morphological filter design
Signal Processing: Image Communication
(1996)
Z. Zhang et al.
A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry
Artificial Intelligence
(1995)
Bhanu, B., Lee, S., Ming, J., 1995. Adaptive image segmentation using a genetic algorithm. IEEE Trans. Systems Man...
Faugeras, O.D., 1992. What can be seen in the three dimensions with an uncalibrated stereo rig?. In: Sandini, G. (Ed.),...
Goldberg, D.E., 1989. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading,...
Harris, C., 1987. A combined corner and edge detection. In: Proc. Alvey...
Hartley, R.I., 1995. In defence of the 8-point algorithm. In: Proc. 5th Internat. Conf. on Computer Vision, Boston, MA,...
Hartley, R.I., Gupta, R., 1993. Computing matched-epipolar projections. In: Proc. Conf. on Computer Vision and Pattern...

Holland, J., 1975. Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor,...

Cited by (36)

Novel adaptive genetic algorithm sample consensus
2019, Applied Soft Computing Journal
Citation Excerpt :
One of the first usages of the genetic algorithms in computer vision is presented in the Saito and Mori [23] paper in stereo matching. As the benefits of genetic algorithms were revealed, researchers utilized this technique in other contexts of computer vision as well [24–27]. GASAC [6] was one of the first forms of the evolutionary approaches in the RANSAC algorithm.
Random Sample Consensus (RANSAC) is a successful algorithm in model fitting applications when there are numerous outliers within the dataset. Achieving a proper model is guaranteed through the pure exploration strategy of RANSAC. However, finding the optimum result requires exploitation. Genetic Algorithm Sample Consensus (GASAC) is an evolutionary paradigm which adds the exploitation capability to RANSAC. Although GASAC improves the results of RANSAC, it has a fixed strategy for balancing between exploration and exploitation. In this paper, a new paradigm is proposed based on genetic algorithms using an adaptive strategy. We propose an adaptive genetic operator to select the proper number of high fitness individuals as parents and mutate the rest. This operator can adjust the ratio of exploration vs. exploitation phases according to the amount of outliers. Also, a learning method is proposed for the mutation operator to gradually learn which gene is the best replacement for the mutated gene. This operator guides the exploration phase towards good solution areas and therefore produces better individuals for further exploitation. The proposed method is extensively evaluated in two sets of experiments. In all tests, our method outperformed the other methods in terms of both the number of inliers found and the speed of the algorithm.
Three-dimensional microscope vision system based on micro laser line scanning and adaptive genetic algorithms
2017, Optics Communications
A microscope vision system to retrieve small metallic surface via micro laser line scanning and genetic algorithms is presented. In this technique, a 36 µm laser line is projected on the metallic surface through a laser diode head, which is placed to a small distance away from the target. The micro laser line is captured by a CCD camera, which is attached to the microscope. The surface topography is computed by triangulation by means of the line position and microscope vision parameters. The calibration of the microscope vision system is carried out by an adaptive genetic algorithm based on the line position. In this algorithm, an objective function is constructed from the microscope geometry to determine the microscope vision parameters. Also, the genetic algorithm provides the search space to calculate the microscope vision parameters with high accuracy in fast form. This procedure avoids errors produced by the missing of references and physical measurements, which are employed by the traditional microscope vision systems. The contribution of the proposed system is corroborated by an evaluation via accuracy and speed of the traditional microscope vision systems, which retrieve micro-scale surface topography.
Robust trifocal tensor constraints for structure from motion estimation
2013, Pattern Recognition Letters
Citation Excerpt :
Genetic algorithms (GA) are often used for parameter estimations. Chai and Ma (1998) and Tang et al. (2008) applied GA to estimate the fundamental matrix. Hu et al. (2003) also implemented the trifocal tensor estimation using GA.
It is important to estimate accurate camera parameters in multi-view stereo. In this paper, we use three-view relations, the trifocal tensor, to improve the Bundler, a popular structure from motion (SfM) system, for estimating accurate camera parameters. We propose a novel method: the Robust Orthogonal Particle Swarm Optimization (ROPSO) to estimate a robust and accurate trifocal tensor. In ROPSO, we formulate the trifocal tensor estimation as a global optimization problem and use the particle swarm optimization (PSO) for parameter searching. The orthogonal array is used to select the representative initial particles in PSO for more stable results. In the experiments, we use simulated and real ground truth data for statistical analysis. The experimental results show that the proposed ROPSO can achieve more accurate estimation of the trifocal tensor than the traditional methods and has higher probability to find the optimization solution than the traditional methods. Based on the trifocal tensor estimated by the proposed method, the SfM estimation errors can effectively be reduced. The average reprojection errors are reduced from 21.5 pixels to less than 1 pixel.
Fundamental matrix estimation by multiobjective genetic algorithm with Taguchi's method
2012, Applied Soft Computing Journal
Citation Excerpt :
If the eight pairs of corresponding points are known, we can compute the parameters of the fundamental matrix. With more than eight points, a linear least squares minimization problem is to be solved [3,15,20]. Traditional bucketized algorithms divide the plane into some non-overlapping areas [20], and then select one point from each area.
We propose a multiobjective genetic algorithm to compute the fundamental matrix, which are the foundation of multiview geometry and calibration in many 3D applications such as 3D reconstruction. The proposed method is a modification of the Intelligent Multiobjective Evolutionary Algorithm (IMOEA) [7] coupled with Taguchi's method [14]. Our design focuses are the fitness assignment of multiple objective functions, the diversity preservation, and the addition of an elite set. Moreover, we propose to include an additional random population besides the original initial population in genetic algorithms. In each generation we replace the random population and select only the non-dominated individuals into the elite set. The proposed method can explore more general solution space and can locate better solutions. We validate the proposed methods by demonstrating the effectiveness of the proposed methods to estimate of the fundamental matrices.
Epipolar geometry estimation based on evolutionary agents
2008, Pattern Recognition
This paper presents a novel approach based on the use of evolutionary agents for epipolar geometry estimation. In contrast to conventional nonlinear optimization methods, the proposed technique employs each agent to denote a minimal subset to compute the fundamental matrix, and considers the data set of correspondences as a 1D cellular environment, in which the agents inhabit and evolve. The agents execute some evolutionary behavior, and evolve autonomously in a vast solution space to reach the optimal (or near optima) result. Then three different techniques are proposed in order to improve the searching ability and computational efficiency of the original agents. Subset template enables agents to collaborate more efficiently with each other, and inherit accurate information from the whole agent set. Competitive evolutionary agent (CEA) and finite multiple evolutionary agent (FMEA) apply a better evolutionary strategy or decision rule, and focus on different aspects of the evolutionary process. Experimental results with both synthetic data and real images show that the proposed agent-based approaches perform better than other typical methods in terms of accuracy and speed, and are more robust to noise and outliers.
Quaternion wavelet phase based stereo matching for uncalibrated images
2007, Pattern Recognition Letters
This paper proposes a quaternion wavelet phase based stereo matching (QWPSM) scheme for uncalibrated image pairs. In this scheme, we estimate the disparity by directly establishing correspondences between quaternionic phase structures of two quaternion wavelet filtered (QWF) images. Firstly, linear-phase quaternion wavelet filters (LPQWFs) are constructed from real biorthogonal wavelet bases. Then, quaternion phases are extracted under each scale through quaternion wavelet filtering of the multiscale transformed image pyramids. The disparity estimation is formed as a minimization process of a local energy weighted cost function, and propagated from coarse to fine scales. Costs can adaptively alleviate the negative effects of phase singularities, which are the main causes of mismatches in phase-based stereo matching. Multiscale matching strategy is used to avoid phase wrapping and improve convergence speed. Experimental results are promising in various image pairs.

View all citing articles on Scopus

¹: This work was partially supported by the Chinese National Science Foundation, the Chinese National Key Project for Basic Researchers, and the Chinese National High Technology Program (863).

View full text

Robust epipolar geometry estimation using genetic algorithm1

Abstract

Introduction

Section snippets

Problem formulation

Genetic algorithm: a review

Genetic algorithm: application in epipolar geometry estimation

Experimental results

Conclusion

Pattern Recognition Letters

Pattern Recognition

Signal Processing: Image Communication

Artificial Intelligence

Robust epipolar geometry estimation using genetic algorithm¹