Self-calibration from turn-table sequences in presence of zoom and focus

https://doi.org/10.1016/j.cviu.2006.01.004Get rights and content

Abstract

This paper proposes a novel method, using constant inter-frame motion, for self-calibration from an image sequence of an object rotating around a single axis with varying camera internal parameters. Our approach makes use of the facts that in many commercial systems rotation angles are often controlled by an electromechanical system, and that the inter-frame essential matrices are invariant if the rotation angles are constant but not necessary known. Therefore, recovering camera internal parameters is possible by making use of the equivalence of essential matrices which relate the unknown calibration matrices to the fundamental matrices computed from the point correspondences. We also describe a linear method that works under restrictive conditions on camera internal parameters, the solution of which can be used as the starting point of the iterative non-linear method with looser constraints. The results are refined by enforcing the global constraints that the projected trajectory of any 3D point should be a conic after compensating for the focusing and zooming effects. Finally, using the bundle adjustment method tailored to the special case, i.e., static camera and constant object rotation, the 3D structure of the object is recovered and the camera parameters are further refined simultaneously. To determine the accuracy and the robustness of the proposed algorithm, we present the results on both synthetic and real sequences.

Introduction

Acquiring 3D models from circular motion sequences, particularly turn-table sequences, has been widely used by computer vision and graphics researchers, e.g., [36], [31], [4], [35], since these methods are simple and robust. Generally, the whole reconstruction procedure includes: first, the determination of camera positions at different viewpoints or, equivalently, the different positions of the rotating device; second, the detection of object boundaries or silhouettes; third, the extraction of a visual hull as the surface model from a volume representation [21]. Fitzgibbon et al. [9] extended the analysis of the circular motion to recover unknown rotation angles from uncalibrated image sequences based on a projective geometry approach and multi-view geometric constraints. [28], [29] recovered the circular motion by using surface profiles. Wong et al. [42] also presented a method for camera calibration using surfaces of revolution, which is related to circular motion since an object placed on a turn-table spans a surface of revolution. Recently, Jiang et al. [19], [18] developed new methods to compute single axis motion by either fitting the conic to the locus of the tracked points in at least five images or computing a plane homography from a minimal of two points in four images. Colombo et al. [6] improved the approach [42] in which the calibration of a natural camera, a pinhole camera with zero skew and unit aspect ratio [22], requires the presence of two different surfaces of revolutions in the same view. In addition, the method [6] relaxes the conditions claimed by Bougnoux [19], that three ellipses are needed to compute the imaged circular points.

However, most of these methods deal with the case in which a static camera with fixed internal parameters views an object rotating on a turn-table (Fig. 1), and utilize the fixed image entities of the circular motion. These fixed image entities (Fig. 3) include two lines: one is the image of the rotation axis ls, a line of fixed points, while the other one, called the horizon line l, is the image of the vanishing line of the horizontal planes, e.g., π1 and π2 (πA and πB in Fig. 1). Unlike the image of the rotation axis, the horizon line is a fixed line, but not a line of fixed points. Under the assumption of the fixed camera internal parameters, the image of the absolute conic is fixed for a rigid motion. Therefore, there are two points, i and j, located at the intersection of the absolute conic with the horizon line, and remaining fixed in all images. Actually, these two fixed points are the images of the two circular points on the horizontal planes, and can be found by the intersections of conic loci of corresponding points since the trajectories of space points are circles in 3D space and intersect in the circular points on the plane at infinity. However, these entities are fixed only when the camera has fixed internal parameters. For example, the projected trajectory of a 3D point is not a conic any more when the camera’s internal parameters are varying (Fig. 2B).

In this paper, we concentrate on the situation where the stationary camera is free to zoom and focus, and assume that in many commercial systems, rotation angles are often controlled by an electromechanical system [36], [31], [35], [26], i.e., they are constant and even known. We show that the inter-frame essential matrices are invariant if the rotation angle is constant but not necessary known and, therefore, recovering camera internal parameters is possible by making use of the equivalence of essential matrices. We also introduce a linear method that works under restrictive conditions on camera internal parameters, such as known camera skew, aspect ratio and principal point, the solution of which can be used as the starting point of the iterative non-linear methods with looser constraints. The results are optimized by enforcing the global constraints that the projected trajectories of 3D points should be conics after compensating the focusing and zooming effects. Finally, using the bundle adjustment method tailored to the special case, i.e., static camera and constant rotation angle, the 3D structure of the object is recovered and the camera parameters are further refined simultaneously.

The rest of the paper is organized as follows. We start with the description of previous work on self-calibration in the next section, and then present the preliminaries of the pinhole camera model and epipolar geometry in Section 3. In Section 4, a practical calibration method, making use of constant inter-frame motion, is developed. A simple linear solution is also given which can be used as an initialization. The method is then validated through the experiments on both computer simulation and real data in Section 5. Finally, Section 6 concludes the paper with perspective of this work.

Section snippets

Related work

Self- (or auto-) calibration is the process of determining internal camera parameters directly from a set of uncalibrated images. This differs from conventional calibration, where the camera internal parameters are determined from the image of a known calibration grid, e.g., [41], [46], [14], or properties of the scene, such as vanishing points of orthogonal directions [5], [22]. In self-calibration the metric properties of the cameras are determined directly from constraints on the internal

Pinhole camera model

A real world camera can be modeled by a pinhole or perspective camera model. A pinhole camera, based on the principle of collinearity, projects a region of R3 lying in front of the camera into a region of the image plane R2. As is well known, a 3D point X = [X Y Z 1]T and its corresponding projection x = [x y 1]T in the image plane are related via a 3 × 4 matrix P asxK[R|t]PX,K=fγfu00λfv0001,where ∼ indicates equality up to multiplication by a non-zero scale factor, R is a 3 × 3 orthonormal rotation

Solving for self-calibration

It is well known that when the projective image measurements alone are used it is only possible to recover the scene up to an unknown projective transformation [7], [11]. Additional scene, motion or calibration constraints are required for a metric or Euclidean reconstruction. We also use the constraints on camera internal parameters similar to previous self-calibration methods. However, the main difference is that constant inter-frame motion is exploited in this paper.

Experimental results

The proposed approach has been tested on both simulated and real image sequences. First, a synthetic image sequence is used to assess the quality of the algorithm under simulated circumstances. Both the amount of noise on the projected image points and on the rotation angles of the objects are varied. Then results are given for real image sequences to demonstrate the usability of this proposed solution.

Conclusion

This paper focuses on the problem of self-calibration from an image sequence of an object rotating around a single axis in presence of varying camera internal parameters. Using the invariance property of the inter-frame essential matrices when the rotation angle is constant, we present a new and simple algorithm for camera calibration. Compared to the existing methods, we effectively utilize the prior information, such as constant rotation angle and circular motion, and design a two-stage

References (47)

  • H. Trivedi

    Can multiple views make up for lack of camera registration?

    Image Vision Comput.

    (1988)
  • L.D. Agapito et al.

    Self-calibration of rotating and zooming cameras

    Int. J. Comput. Vision

    (2001)
  • M. Armstrong, A. Zisserman, R. Hartley, Self-calibration from image triplets, in: Proc. ECCV, 1996, pp....
  • S. Bougnoux. From projective to euclidean space under any practical situation, a criticism of self-calibration, in:...
  • E. Boyer, Object models from contour sequences, in: Proc. ECCV, 1996, pp....
  • B. Caprile et al.

    Using vanishing points for camera calibration

    Int. J. Comput. Vision

    (1990)
  • C. Colombo et al.

    Metric 3D reconstruction and texture acquisition of surfaces of revolution from a single uncalibrated view

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2005)
  • O. Faugeras, What can be seen in three dimensions with an uncalibrated stereo rig?, in: Proc. ECCV, 1992, pp....
  • O. Faugeras, T. Luong, S. Maybank, Camera self-calibration: theory and experiments, in: Proc. of ECCV, 1992, pp....
  • A.W. Fitzgibbon, G. Cross, A. Zisserman, Automatic 3D model construction for turn-table sequences, in: SMILE Wkshp.,...
  • J. Frahm, R. Koch, Camera calibration with known rotation, in: Proc. IEEE ICCV, 2003, pp....
  • R.I. Hartley, Estimation of relative camera positions for uncalibrated cameras, in: Proc. ECCV, 1992, pp....
  • R.I. Hartley

    Self-calibration of stationary cameras

    Int. J. Comput. Vision

    (1997)
  • R.I. Hartley et al.

    Multiple View Geometry in Computer Vision

    (2004)
  • J. Heikkila

    Geometric camera calibration using circular control points

    IEEE Trans, Pattern Anal. Mach. Intell.

    (2000)
  • A. Heyden, K. Astrom, Euclidean reconstruction from image sequences with varying and unknown focal length and principal...
  • A. Heyden, K. Astrom, Flexible calibration: minimal cases for auto-calibration, in: Proc. IEEE ICCV, 1999, pp....
  • T.S. Huang et al.

    Some properties of the e matrix in twoview motion estimation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1989)
  • G. Jiang et al.

    Circular motion geometry using minimal data

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2004)
  • G. Jiang et al.

    Single axis geometry by fitting conics

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2003)
  • F. Kahl et al.

    Critical motions for auto-calibration when some intrinsic parameters can vary

    J. Math. Imaging Vis.

    (2000)
  • K. Kutulakos, S. Seitz, A theory of shape by space carving, in: Proc. IEEE CVPR, 1999, pp....
  • D. Liebowitz, A. Zisserman, Combining scene and auto-calibration constraints, in: Proc. IEEE ICCV, 1999, pp....
  • Cited by (0)

    View full text