Elsevier

Pattern Recognition Letters

Volume 127, 1 November 2019, Pages 174-180
Pattern Recognition Letters

Simultaneous robot-world and hand-eye calibration by the alternative linear programming

https://doi.org/10.1016/j.patrec.2018.08.023Get rights and content

Highlights

  • The hand-eye calibration is reconsidered as minimizing two kinematic loops’ errors.

  • The alternative linear programming is introduced to solve the calibration.

  • The semi-convex constraint is also proposed for the construction of cost function.

  • Two formulations of calibration are presented: homogeneous matrix, dual quaternion.

Abstract

A typical hand-eye robot system has two related kinematic loops, which proper functioning requires the accurate determination of robot-world (RW) and hand-eye (HY) transformations in them. Insofar as HY calibration can resolve only the issues related to HY transformation, this study is focused on the joint treatment of the RW&HY transformations, which process is referred to as simultaneous RW&HY calibration. An alternative linear programming is adopted for the construction of semi-convex objective functions via two algorithms. These involve the homogeneous matrix and dual quaternion parameterizations, respectively. Their feasibility was further tested using simulated and real experimental datasets and compared to the results obtained via two available methods (NL and LMI). The results obtained strongly suggest that the homogeneous matrix parametrization (ALP1) had better performance than the dual quaternion one (ALP2). Meanwhile, both of them yielded the good optimal calibration solutions instead of local minima ones. Therefore, both formulations provide new insights into the behavior and complexity of the simultaneous RW&HY calibration.

Introduction

Practical applications of image-guided systems, such as in robot assisted surgeries and autonomously guided vehicles, require mounting of the sensors (a camera, a laser scanner, or an ultrasound probe) on the end-effector of a robotic manipulators. Such a typical “hand-eye” (HY) system is depicted in Fig. 1. The proper functioning of these sensors relies on the calibration of the two relative transformations, namely (i) transformation X between the sensor and end-effector and (ii) transformation Z from the robot base to the global coordinate system (world). As shown in Fig. 1, X and Z can be used to deduce two kinematic loop equations, the first of which has the following form:AX=XB,where A is the motion matrix of the sensor, and B is the motion matrix of end-effector. Furthermore, A can also be used to express the relative motion between two adjacent sensor poses Ai and Ai+1 (i=1,2,,m)A=Ai+11Ai.

Similarly, B can also be used to express the relative motion between two adjacent robot poses Bi and Bi+1 (i=1,2,,m)B=Bi+11Bi.

The second equation has the following formAiX=ZBi,and features the number of poses Ai,Bi (i=1,2,,m). Based on the above two mentioned kinematic loop equations (1), (2), two different methods of calibration for HY system are proposed. The first one, which is referred to as hand-eye calibration, is realized by solving Eq. (1). The second one, which is commonly referred to as the simultaneous robot-world and hand-eye calibration, requires solving Eq. (2).

There are multiple methods for solving the HY calibration equation [16] which yield different linear solutions. Authors Tsai and Lenz [20] have put forward an efficient linear algorithm to solve the HY equation and pointed out that in order to do so, which required at least two rotation-containing motions with non-parallel rotation axes. A new HY calibration algorithm that used dual quaternions, which could simultaneously determine the rotation and translation by following the singular value decomposition (SVD) approach, was developed in [3]. A similar method that was based on screw motion was further adopted in [24], while a probabilistic method for HY calibration, which could be efficiently applied in case of unknown dependencies between the sensor and robot motions, was proposed in [11]. The optimization of possible random noise effect on HY calibrations has also been recommended by multiple researchers as an efficient method of minimizing errors. Thus, Zhuang and Shiu [26] implemented the non-linear optimization method for both the rotational and translational parts, whereas a parameter similar to the Frobenius norms was applied as the cost function in the calibration method. A similar optimization strategy was also presented in [5]. The application of canonical co-ordinates in the HY calibration made it possible to simplify the characterization of parameters in optimization in [14]. The non-linear optimization was proposed in [9] with the help of the rotational parameters formulated in quaternions. Further progress in this direction was made in the study [17], where a new metric of rigid transformations in a 3-dimensional Euclidean space, denoted SE(3), which possessed the ability of automatic optimal weighting, was adopted for the non-linear optimization in HY calibration. A recent publication Wu and Ren [21] described the HY equation conversion into a linear form based on the Kronecker product, which yielded a closed-form solution for the Gauss–Newton iteration. In spite of the excellent results obtained in the above studies, the optimization process elucidated in most of them was trapped in local minima, unless the appropriate initial values were preset. Therefore, a convex optimization algorithm was proposed in [6], which application to the above problem in [23] furnished a globally applicable minimization-based method that required no input of the initial values. Recently, the global optimal HY calibration method, which directly used the camera measurements and applied the branch-and-bound approach to minimize an objective function based on the epipolar constraint, was elaborated in [7].

Authors Zhuang et al. [25] proposed a method for solving the discrepancies of simultaneous RW&HY calibration, which applied the quaternion algebra to the derivation of explicit linear solutions for X and Z. The study [4] introduced an alternative process of RW&HY transformation, which encompassed the application of quaternion algebra and positive quadratic error function, furnished the non-linear constrained minimization of the latter, and yielded simultaneous solutions for rotations and translations. Other researchers [8] expressed the discrepancies in simultaneous RW&HY calibration as multivariate polynomial optimization problems over semi-algebraic sets and solved them using the method of convex linear matrix inequality (LMI) relaxations. In work [19], the rotation components of X and Z were parameterized using the Euler angles, and the solution was derived using the Levenberg-Marquardt iterative approach. The probabilistic method [10] was also found to be instrumental in finding solutions for X and Z without a priori knowledge of the position data correspondences. The number of unknown matrices in kinematic loop Eq. (2) is higher than that in Eq. (1), which makes the HY calibration more laborious than the simultaneous RW&HY one. However, it is not mandatory to compute the robot and sensor relative motions before solving Eq. (1). In other words, simultaneous RW&HY calibration has less data-generation errors than HY one. Hence, it is expedient to conduct extensive research on finding a solution for Eq. (1).

This study involves proposition and subsequent investigation of a novel iterative method that can be employed for ensuring the simultaneous RW&HY calibration via Eq. (2). The proposed method serves to solve the rotational and translational parts simultaneously, requires no initial estimates, and provides the optimal solutions in L1-norm. This can be achieved by formulating the simultaneous RW&HY calibration problem as a semi-convex optimization problem [15] and solving it with the help of the alternative linear programming (ALP) method. The latter applies the least absolute deviation L1-norm, which minimizes the sum of the absolute differences between the target and estimated values, and is found to be more robust than the least square-based L2-norm, which minimizes the sum of the square of the differences between the target and estimated values.

The remainder of this article is organized as follows. In Section 2, we review the definition of a semi-convex function and the corresponding alternating iterative method (AIM). In Section 3, two simultaneous RW&HY calibration optimization problems are formulated for homogenous and quaternion spaces, respectively. The implementation details and performance of the proposed solution in both synthetic and real data calibration scenarios are presented in Section 4.

Section snippets

Semi-convex function and alternating iterative method

Let C and D be convex subsets of Rk and Rl, respectively. The function f(θ, φ) is treated as a semi-convex function of (θ, φ), only if [15]:

  • f(θ, φ) is defined on C × D;

  • For any given θ ∈ C, f(θ, •) is strictly convex on D, and for any given φ ∈ D, f(•, φ) is strictly convex on C.

In fact, there exist many appropriate semi-convex functions that can be used as objective functions for the optimization.

Consider the semi-convex optimization problemminθC,φDf(θ,φ),where f(θ, φ) is defined on C × D,

Simultaneous robot-world and hand-eye calibration

In this section, two formulations of the simultaneous RW&HY calibration problem are presented. Firstly, the calibration problem is reduced to the following minimization problem in the homogeneous space SE(3)={[Rt0T1]|RSO(3),tR3}:minX,ZSE(3)i=1mAiXZBi1where the matrix L1 norm is defined as 1=vec()1, and the symbol vec(•) is the vectorization operator of the matrix. The second formulation uses the dual quaternion parametrization to minimize the vector L1-normminqˇX,qˇZR8i=1maˇiqˇX

Experiments

The experiments with simulated and real data were conducted to validate the proposed calibration methods in this section. For the real data experiment, we used a MOTOMAN HP3 serial 6-DOF manipulator with two PointGrey flea2 CCD cameras rigidly attached to its end-effector. We simulated the same setup in the synthetic experiment for better cohesion of simulated and real test results.

Algorithms 1 and 2 were implemented in the CVX Matlab-based modeling system for convex optimization [6] that made

Conclusions

The results obtained in this study demonstrate the feasibility of the ALP-based optimization for solving the simultaneous RW&HY calibration problems. Two alternative options for simultaneous RW&HY calibration parametrization forms, namely (i) homogeneous matrix (ALP1), and (ii) dual quaternion (ALP2) are analyzed in detail. The experimental results obtained strongly suggest that the homogeneous matrix parametrization (ALP1) had better performance than the dual quaternion one (ALP2). Meanwhile,

Conflict of interest

The author declares that he has no conflicts of interest.

Acknowledgments

The author would like to thank Prof. Yuncai Liu of Shanghai Jiao Tong University for his technical supports. This work has been supported by the Specialized Research Fund for the Doctoral Program of Higher Education of China (No.20130131120036), the Promotive Research Fund for Excellent Young and Middle-aged Scientists of Shandong Province (No.BS2013DX027), and the National Natural Science Foundation of China (No.81401543).

References (26)

  • W. Blaschke

    Kinematik und Quaternionen

    (1960)
  • J.Y. Bouguet, A release of a camera calibration toolbox for matlab,...
  • K. Daniilidis

    Hand-eye calibration using dual quaternions

    Int. J. Rob. Res.

    (1999)
  • F. Dornaika et al.

    Simultaneous robot-world and hand-eye calibration

    IEEE Trans. Robot. Autom.

    (1998)
  • I. Fassi et al.

    Hand to sensor calibration: a geometrical interpretation of the matrix equation ax=xb

    J. Robot. Syst.

    (2005)
  • M. Grant et al.

    Graph implementations for nonsmooth convex programs

    Recent Advances in Learning and Control-Lecture Notes in Control and Information Sciences

    (2008)
  • J. Heller et al.

    Globally optimal hand-eye calibration using branch-and-bound

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2016)
  • J. Heller et al.

    Hand-eye and robot-world calibration by global polynomial optimization

    Proc. IEEE Inter. Conf. Robot. Autom.

    (2014)
  • R. Horaud et al.

    Hand-eye calibration

    Int. J. Rob. Res.

    (1995)
  • H. Li et al.

    Simultaneous hand-eye and robot-world calibration by solving the ax=yb problem without correspondence

    IEEE Rob. Autom. Lett.

    (2016)
  • Q. Ma et al.

    New probabilistic approaches to the ax=xb hand-eye calibration without correspondence

    (2016)
  • J.M. McCarthy

    Introduction to Theoretical Kinematics

    (1990)
  • H. Neudecker

    A note on kronecker matrix product and matrix equation systems

    SIAM J. Appl. Math.

    (1969)
  • Cited by (35)

    • Robot-world and hand–eye calibration based on motion tensor with applications in uncalibrated robot

      2022, Measurement: Journal of the International Measurement Confederation
    • A novel singular-free solution based on principle of transfer theory for the hand-eye calibration problem

      2022, Mechanism and Machine Theory
      Citation Excerpt :

      According to the quaternion and Kronecker product, Zhao [13] proposed an objective function under infinite norm metric criterion and obtained the optimal solution of hand-eye calibration equation by a second-order cone optimization method without initial value. Then the global optimal method proposed by Zhao [14] based on Alternative linear programming (ALP) can also generate highly accurate results and does not require initial estimates. In addition, based on the polar constraint theory, Heller et al. [15] proposed an optimization index under infinite norm and transformed the hand-eye calibration problem into a nonlinear optimization problem.

    • Simultaneous robot–world and hand–eye calibration based on a pair of dual equations

      2021, Measurement: Journal of the International Measurement Confederation
    View all citing articles on Scopus
    View full text