Projective reconstruction of ellipses from multiple images
Introduction
In computer vision, the recovery of 3D scene from a sequence of images is a fundamental problem. Existing research mainly concentrates on 3D reconstruction of point features [1], [2], [3] and line segments [4], [5], [6]. However, an ellipse (a circle is included as a special case), is one of the most common features that have been used for 3D location estimation for the following reasons: (1) objects with circular and elliptical contours are frequently seen in the real world and (2) the image-location accuracy of an ellipse is higher than that of a point or a line segment [7], [8], which is very attractive for applications. For these reasons, ellipses have been used to solve various machine-vision-related problems. For example, they are used to accurately locate a mobile robot's position using circular landmarks [9], [10], to estimate the 3D orientation of objects with circular contours [11], and to recognize 3D pre-marked objects using circular markers [12].
Therefore, the 3D reconstruction of ellipses is of fundamental importance. Recently, the problem of pose estimation, stereo and motion estimation based on ellipses has attracted some attention [13], [14], [15], [16], [17], but there are still much fewer articles dealing with ellipses than those devoted to points and line segments. Although there are a few papers on conics reconstruction (of which ellipses can be considered as a special case), there is little work specifically on ellipse reconstruction. The 3D reconstruction of ellipses is therefore still an open problem and needs further exploring.
In [13], [14], algorithms are proposed to reconstruct the positions and orientations of conics in 3D space, where the cameras are assumed to be calibrated. By deriving two polynomial conditions, Long [15] proposes a unified closed-form mathematical solution for conic reconstruction from two uncalibrated images. The method is simple and stable, but it assumes that the pair of projection matrices is known. Instead of directly recovering the parametric description of a quadratic curve in 3D space as the above mentioned methods, Xie [16] emphasizes on the 3D reconstruction of points belonging to the 3D conics. By imposing a planarity constraint, 3D conics are reconstructed once the 3D points on them are known. The experiments show that the reconstruction precision is much better than direct 3D recovery of the conic parameters. This method assumes that the relative geometry between two perspective views is known, and thus the point correspondences on the curves are known.
Up to now, all the methods we know for 3D conic reconstruction share two limitations. The first one is that they require prior information of the cameras or the geometric relations between images, which is not always available in real applications. The second one is that the reconstructions are based on a pair of images only. It is known that 3D conic reconstruction over two views is ambiguous up to two solutions [13], [14], [15], defined by any one of the plane pair, as shown in Fig. 1.
For higher-order general algebraic curves, Kaminski et al. proposed algorithms to reconstruct general algebraic curves (both for planar [18] and [19] for non-planar cases) from multiple views, assuming known camera matrices. In [20], Berthillson et al. introduce the concept of affine shape, and an algorithm for reconstructing general curves from uncalibrated images is proposed in [21], which relies on aligning the parameterizations of matched curves so that the points traced on each curve become corresponding points. This algorithm works with any number of images taken by uncalibrated cameras, but a final step of bundle adjustment is required to refine the reconstruction, and it has difficulty to reconstruct multiple curves from multiple views with missing curves and/or partial occlusion of curves in some of the views.
To solve the existing problems mentioned above, we propose a point-based ellipse reconstruction method in this paper. The proposed method recovers an ellipse in 3D space by reconstructing 3D points (referred to as representative points) on it, by minimizing the geometric distance from the projections of the N representative points to the corresponding measured ellipses in images.
The proposed approach has the following advantages:
- •
The method reconstructs the 3D ellipses over a sequence of images, which provides more information and help to obtain a unique solution.
- •
No prior knowledge about the cameras or the relative geometry of the images is required.
- •
The reconstruction is realized by iteratively minimizing a cost function which represents a geometric measure of the goodness of reconstruction without the need of further refinement by bundle adjustment.
- •
The iterative optimization algorithm is guaranteed to converge and is robust to noise in images.
- •
The method can recover multiple ellipses simultaneously and it readily handles missing and/or partially occluded ellipses in images.
- •
The method recovers the projection matrices simultaneously with the 3D ellipses. By doing so, our algorithm achieves higher reconstruction accuracies, since the ellipse data also contributes to the estimation.
This paper is organized as follows. In Section 2, we briefly summarize the point-based projective reconstruction method of [2], which forms the basis of our proposed ellipse reconstruction method. In Section 3, the ellipse reconstruction problem is reformulated as a minimization problem. The proposed algorithm is presented in Section 4. Experimental results are given in Section 5 to demonstrate the performance of the proposed method. Concluding remarks are given in Section 6.
Section snippets
Projective reconstruction of 3D points
Suppose we are given m views of t 3D points. The l-th point is projected onto the i-th view and measured as in homogeneous coordinates. Since it is allowed that some points be missing in some views, the available measurements are defined by an index set:The goal of projective point reconstruction is to estimate the t 3D points from the available measurements on the images. Let us denote the l-th
Given data
As shown in Fig. 2, suppose we are given m views of a 3D scene composed of n ellipses and t feature points (not necessarily on the ellipses). It is known that an ellipse in 3D space projects to an ellipse on a 2D image [22]. Since the ellipses may not be visible in all the views, we define an index set for the measured 2D ellipses asThe measured 2D ellipses will be denoted by (the ellipses on the images in Fig. 2) and the 2D points by
Ellipse reconstruction
In this section, we will show how (5) can be reformulated as a multi-linear least-squares minimization problem that can be solved by means of an iterative procedure.
Experimental results
The proposed method is evaluated by both synthetic and real data. The algorithm is implemented using Matlab 7.0 running under Windows XP on a Pentium 4 2.40 GHz machine with 512 MB RAM.
Conclusions
In this paper, a new approach is developed for reconstructing 3D ellipses from images taken by uncalibrated cameras. We reconstruct a 3D ellipse (or a circle) by minimizing the distance of reprojected representative points on the ellipse (or circle) from the measured 2D ellipses (or circles) on different images, except for a reference ellipse where the distance is measured from some pre-defined representative points to their reprojections. By solving a sequence of weighted
Acknowledgments
The work in this paper was supported by the Research Grants Council of Hong Kong Special Administrative Region, China (GRF Projects HKU 712808E and HKU 711208E) and CRCG of the University of Hong Kong.
About the Author—F. MAI received her B.Eng. and M.Eng. degrees in Control Theory and Control Engineering in 2000 and 2003 respectively from Tianjin University, China, and her Ph.D. degree from the University of Hong Kong in 2008. She is now a research associate in the Department of Electrical and Electronic Engineering at the University of Hong Kong. Her research interests include feature extraction, feature matching and 3D reconstruction.
References (25)
- et al.
3-D shape from contour and selective confirmation
Computer Vision, Graphics, and Image Processing
(1983) - P. Sturm, B. Triggs, A factorization based algorithm for multiple-image projective structure and motion, in:...
- et al.
Projective reconstruction from multiple views with minimization of 2D reprojection error
International Journal of Computer Vision
(2006) - et al.
Shape and motion from image streams under orthography: a factorization method
International Journal of Computer Vision
(1992) - R. I. Hartley, Projective reconstruction from line correspondences, in: Proceedings of IEEE Computer Society Conference...
Lines and points in three views and the trifocal tensor
International Journal of Computer Vision
(1997)- et al.
Projective reconstruction from line-correspondences in multiple uncalibrated images
Pattern Recognition
(2006) - et al.
Ranking of pattern for use in automation
IEEE Transactions on Industrial Electronics and Control Instrumentation
(1977) - R.A. Young, Locating industrial parts with subpixel accuracies, in: SPIE Proceedings of Optics, Illumination, Image...
- M.J. Magee, J.K. Aggarwal, Determining the position of a robot using a single calibration object, in: Proceedings of...
Position verification of a mobile robot using standard pattern
IEEE Journal of Robotics and Automation
Cited by (0)
About the Author—F. MAI received her B.Eng. and M.Eng. degrees in Control Theory and Control Engineering in 2000 and 2003 respectively from Tianjin University, China, and her Ph.D. degree from the University of Hong Kong in 2008. She is now a research associate in the Department of Electrical and Electronic Engineering at the University of Hong Kong. Her research interests include feature extraction, feature matching and 3D reconstruction.
About the Author—Y.S. HUNG received his B.Sc. (Eng) degree in Electrical Engineering and B.Sc. degree in Mathematics, both from the University of Hong Kong, and his M.Phil. and Ph.D. degrees from the University of Cambridge. He has worked as a research associate at the University of Cambridge, and as a lecturer at the University of Surrey before he joined the University of Hong Kong in 1989, where he is now a professor and the head of the department. Dr Hung is a recipient of the Best Teaching Awards in 1991 from the Hong Kong University Students’ Union. He is a chartered engineer, a fellow of IEE and a senior member of IEEE. He has authored and co-authored over 100 publications in books, international journals and conferences.
About the Author—G. CHESI received the Laurea degree in Information Engineering from the University of Firenze in 1997 and the Ph.D. in Systems Engineering from the University of Bologna in 2001. He was a visiting scientist at the Department of Engineering of the University of Cambridge (1999–2000) and Department of Information Physics and Computing of the University of Tokyo (2001–2004). He was with the Department of Information Engineering of the University of Siena (2000–2006) and he joined the Department of Electrical and Electronic Engineering of the University of Hong Kong in 2006. Dr. Chesi was the recipient of the Best Student Award of the Faculty of Engineering of the University of Firenze in 1997. He is an Associate Editor of Automatica, an Associate Editor of IEEE Transactions on Automatic Control, and a Guest Editor of the Special Issue on ‘Positive Polynomials in Control’ of IEEE Transactions on Automatic Control. His research interests include computer vision, nonlinear systems, robotics, robust control, and systems biology. He is author of the book “Homogeneous Polynomial Forms for Robustness Analysis of Uncertain Systems” (Springer, 2009).