Camera auto-calibration using a sequence of 2D images with small rotations

https://doi.org/10.1016/j.patrec.2004.02.011Get rights and content

Abstract

In this study, we describe an auto-calibration algorithm with fixed but unknown camera parameters. We have modified Triggs' algorithm to incorporate known aspect ratio and skew values to make it applicable for small rotation around a single axis. The algorithm despite being a quadratic one is easy to solve. We have applied the algorithm to some artificial objects with known size and dimensions for evaluation purposes. In addition, the accuracy of the algorithm has been verified using synthetic data. The described method is particularly suitable for three dimensional human head modeling.

Introduction

There is a significant body of work on three-dimensional (3D) analysis of images in recent years. Many application areas such as computer animation, medical imaging and teleconferencing require 3D information of the environment that can be simulated on our machines. The first step in 3D analysis is 3D modeling of objects. This step, in general has been proven to be a difficult task to accomplish. The reason for the difficulty is the need for very sensitive and reliable measurements that can either be obtained by using complicated measuring devices to find the depth information or by developing algorithms to extract this information from two-dimensional (2D) images. In the latter case, in order to deal with the problem, researchers have either limited the objects to be modeled to a small and known class of entities (Fua, 2000; Lengagne et al., 1998; Zhang, 2001) or imposed restrictions on the parameters of the problem (Triggs, 1997; Zhang et al., 1998). In general, it is possible to incorporate the information about the scene or camera into the formulation of the problem. The information about the camera includes its physical properties (intrinsic parameters) and its position and orientation (extrinsic properties). Between two images, the position and orientation of the camera may change. The change is given by a translation and rotation. Position and orientation of camera may change in three axes of the coordinate system. These changes altogether let us to find the unknown camera parameters under the assumption that all or at least some of the camera parameters remain constant. However, if the camera rotation is not about at least two independent axes, it is not possible to find some of the camera intrinsic parameters, such as the aspect ratio of scene and camera. In addition, small translation between two images results in small baseline which causes a poor estimate of 3D structure of the objects in the scene.

Determining intrinsic camera parameters without any assumption about the outside world is called camera auto-calibration. Faugeras et al. established the mathematical basis of camera auto-calibration and 3D reconstruction based on auto-calibration (Faugeras, 1992, Faugeras, 1995; Faugeras and Robert, 1994). When a sequence of images is captured, camera motion can be of either general motion or restricted motion. However, in both of the types, there exist some specific motions known as critical motions, where no unique (degenerate) solution can be found for camera parameters. In the case of general motion, there is no assumption on the type of the movements made by the camera. Among the auto-calibration algorithms for general camera motion, there are methods based on absolute quadric (Triggs, 1997; Heyden and Äström, 1996), Kruppa equations (Faugeras et al., 1992), essential matrix properties (Hartley, 1992), and modulus constraints (Pollefeys et al., 1996). In the second type, camera movements are restricted to some specific cases. In this type, we may mention the algorithms based on planar movement of the camera (Triggs, 1998, Armstrong et al., 1996, Fusiello, 2000, Zhang, 2001), and stationary rotating camera with no translation (Agapito et al., 2001, Hartley and Zisserman, 2000). Discussions on critical motions in auto-calibration are given by Sturm (1997), Kahl and Triggs (1999), Triggs (1997) and Torr et al. (1999). Many studies are also devoted to the application of auto-calibration to specific applications. One of these application areas is 3D modeling of human head which is considered using both calibrated and uncalibrated data (Fua, 2000, Lengagne et al., 1998, Mulayim et al., 2003, Zhang, 2001, Zhang et al., 1998, Yilmaz et al., 2003).

In this study, we describe an auto-calibration algorithm with fixed but unknown camera parameters. The algorithm is initially proposed by Triggs (1997) and we have modified Triggs' algorithm to incorporate known aspect ratio and skew values to make it applicable for small rotation around a single axis and for small translation (Hassanpour and Atalay, 2002a, Hassanpour and Atalay, 2002b). This type of rotation and translation is quite common when the images are taken with a handheld camera by a person standing in (almost) a fixed place that happens frequently in 3D modeling of human heads.

The organization of the paper is as follows. Section 2 is an introduction to the background of auto-calibration and 3D reconstruction, and Section 3 describes the modified auto-calibration algorithm. We present the experimental results in Section 4 and then conclude the paper.

Section snippets

Auto-calibration and 3D reconstruction

A simple model of a camera may be defined by a pinhole camera. Considering both camera position and orientation and its physical features, the mapping from world coordinates to image plane coordinates is given by Eq. (1).P=K[R|t]where t and R indicate the translation and rotation of the camera coordinate center with respect to world coordinate system, respectively and K is camera intrinsic matrix. Eq. (1) combines the intrinsic and extrinsic parameters of a projective camera and P is called

Triggs' modified method

To find the homography converting a projective reconstruction to a metric one, we need intrinsic camera parameters. Intrinsic camera parameters are related to the dual of absolute conic Ω via Eq. (3).Ω=KKTΩ and π are encoded in a concise form using a degenerate dual quadric which is called the absolute dual quadric Q. The relationship between Ω and Q is given by Eq. (4).Ω=PQPTRemark that π is the null vector of Q which shows how it encodes Ω and π. Eq. (4) also gives the

Experimental results

The method described in the previous section is implemented using MATLAB. We have imposed the restrictions as equality constraints and used quadratic sequential programming to solve them. The results of three different experiments carried out using this method are presented in this section. A fourth experiment is explained without numerical results.

The method requires at least three images. However, the reliability and stability of the results improve as the number of images increases. In the

Conclusion

An auto-calibration method for a sequence of images with small rotations based on a modified form of Trigg's auto-calibration method is described and experimental results are demonstrated. When auto-calibration is employed, some image sequences generate non-unique (degenerate) solutions. One of these cases is when the camera makes a rotation around a single axis. This case happens frequently during the acquisition of a human head (face) in which the images are taken by rotating the head or the

References (34)

  • A. Fusiello

    Uncalibrated Euclidean reconstruction: a review

    Image Vision Comput.

    (2000)
  • L. Agapito et al.

    Self calibration of rotating and zooming cameras

    Int. J. Comput. Vision

    (2001)
  • Armstrong, M., Zisserman, A., Hartley, R., 1996. Self-calibration from image triplets. European Conference on Computer...
  • Faugeras, O., 1992. What can be seen in three dimensions with an uncalibrated stereo rig. European Conference on...
  • Faugeras, O., Luong, Q., Maybank, S., 1992. Camera self-calibration: Theory and experiments. European Conference on...
  • Faugeras, O., Robert, L., 1994. What can two images tell us about a third one? European Conference on Computer Vision....
  • O. Faugeras

    Stratification of 3-D vision: projective, affine, and metric representations

    J. Opt. Soc. Am. A

    (1995)
  • Fletcher, R., 1980. Practical Methods of Optimization, vol. 1, Unconstrained Optimization, and vol. 2, Constrained...
  • P. Fua

    Regularized bundle-adjustment to model heads from image sequences without calibration data

    Int. J. Comput. Vision

    (2000)
  • Harris, C., Stephens, M., 1988. A combined corner and edge detector. In: Fourth Alvey Vision Conference. pp....
  • R. Hartley et al.

    Multiple View Geometry in Computer Vision

    (2000)
  • Hartley, R., 1995. In defense of the 8-point algorithm. International Conference on Computer Vision. pp....
  • Hartley, R.I., 1992. Calibration of Cameras Using the Essential Matrix. DARPA Image Understanding Workshop. pp....
  • Hassanpour, R., Atalay, V., 2002a. Head Modeling with Camera Auto-calibration and Deformation. In: 7th International...
  • Hassanpour, R., Atalay, V., 2002b. Camera Auto-Calibration using a Sequence of 2D Images with Small Rotations and...
  • Heyden, A., Äström, K., 1996. Euclidean Reconstruction from Constant Intrinsic Parameters. In: 13th International...
  • Kahl, F., Triggs, B., 1999. Critical motions in euclidean structure from motion. In: International Conference on...
  • Cited by (15)

    • Online self-calibration for mobile vision based on laser imaging and computer algorithms

      2011, Optics and Lasers in Engineering
      Citation Excerpt :

      Another self-recalibration based on a homography projected on a reference plane reports an error of 1.41% [12]. A camera auto-calibration based on calibrated points in a reference plane reports an error of 2.0% [24]. A self-calibration method based on the coplanarities of a laser line on two reference planes reports an error of 1.8% [16,18].

    • Human Joints Auto-Calibration Method Based on SuperGlue

      2023, Chinese Control Conference, CCC
    • Software and sensor issues for autonomous systems based on machine learning solutions

      2020, 2020 IEEE International Workshop on Metrology for AeroSpace, MetroAeroSpace 2020 - Proceedings
    • Autonomous Star Camera Calibration and Spacecraft Attitude Determination

      2015, Journal of Intelligent and Robotic Systems: Theory and Applications
    • Efficient planar selfcalibration of a stationary camera with variable intrinsic parameters

      2013, Journal of Theoretical and Applied Information Technology
    View all citing articles on Scopus
    View full text