Simultaneous robot-world and hand-eye calibration by the alternative linear programming
Introduction
Practical applications of image-guided systems, such as in robot assisted surgeries and autonomously guided vehicles, require mounting of the sensors (a camera, a laser scanner, or an ultrasound probe) on the end-effector of a robotic manipulators. Such a typical “hand-eye” (HY) system is depicted in Fig. 1. The proper functioning of these sensors relies on the calibration of the two relative transformations, namely (i) transformation between the sensor and end-effector and (ii) transformation from the robot base to the global coordinate system (world). As shown in Fig. 1, and can be used to deduce two kinematic loop equations, the first of which has the following form:where is the motion matrix of the sensor, and is the motion matrix of end-effector. Furthermore, can also be used to express the relative motion between two adjacent sensor poses and ()
Similarly, can also be used to express the relative motion between two adjacent robot poses and ()
The second equation has the following formand features the number of poses (). Based on the above two mentioned kinematic loop equations (1), (2), two different methods of calibration for HY system are proposed. The first one, which is referred to as hand-eye calibration, is realized by solving Eq. (1). The second one, which is commonly referred to as the simultaneous robot-world and hand-eye calibration, requires solving Eq. (2).
There are multiple methods for solving the HY calibration equation [16] which yield different linear solutions. Authors Tsai and Lenz [20] have put forward an efficient linear algorithm to solve the HY equation and pointed out that in order to do so, which required at least two rotation-containing motions with non-parallel rotation axes. A new HY calibration algorithm that used dual quaternions, which could simultaneously determine the rotation and translation by following the singular value decomposition (SVD) approach, was developed in [3]. A similar method that was based on screw motion was further adopted in [24], while a probabilistic method for HY calibration, which could be efficiently applied in case of unknown dependencies between the sensor and robot motions, was proposed in [11]. The optimization of possible random noise effect on HY calibrations has also been recommended by multiple researchers as an efficient method of minimizing errors. Thus, Zhuang and Shiu [26] implemented the non-linear optimization method for both the rotational and translational parts, whereas a parameter similar to the Frobenius norms was applied as the cost function in the calibration method. A similar optimization strategy was also presented in [5]. The application of canonical co-ordinates in the HY calibration made it possible to simplify the characterization of parameters in optimization in [14]. The non-linear optimization was proposed in [9] with the help of the rotational parameters formulated in quaternions. Further progress in this direction was made in the study [17], where a new metric of rigid transformations in a 3-dimensional Euclidean space, denoted SE(3), which possessed the ability of automatic optimal weighting, was adopted for the non-linear optimization in HY calibration. A recent publication Wu and Ren [21] described the HY equation conversion into a linear form based on the Kronecker product, which yielded a closed-form solution for the Gauss–Newton iteration. In spite of the excellent results obtained in the above studies, the optimization process elucidated in most of them was trapped in local minima, unless the appropriate initial values were preset. Therefore, a convex optimization algorithm was proposed in [6], which application to the above problem in [23] furnished a globally applicable minimization-based method that required no input of the initial values. Recently, the global optimal HY calibration method, which directly used the camera measurements and applied the branch-and-bound approach to minimize an objective function based on the epipolar constraint, was elaborated in [7].
Authors Zhuang et al. [25] proposed a method for solving the discrepancies of simultaneous RW&HY calibration, which applied the quaternion algebra to the derivation of explicit linear solutions for and . The study [4] introduced an alternative process of RW&HY transformation, which encompassed the application of quaternion algebra and positive quadratic error function, furnished the non-linear constrained minimization of the latter, and yielded simultaneous solutions for rotations and translations. Other researchers [8] expressed the discrepancies in simultaneous RW&HY calibration as multivariate polynomial optimization problems over semi-algebraic sets and solved them using the method of convex linear matrix inequality (LMI) relaxations. In work [19], the rotation components of and were parameterized using the Euler angles, and the solution was derived using the Levenberg-Marquardt iterative approach. The probabilistic method [10] was also found to be instrumental in finding solutions for and without a priori knowledge of the position data correspondences. The number of unknown matrices in kinematic loop Eq. (2) is higher than that in Eq. (1), which makes the HY calibration more laborious than the simultaneous RW&HY one. However, it is not mandatory to compute the robot and sensor relative motions before solving Eq. (1). In other words, simultaneous RW&HY calibration has less data-generation errors than HY one. Hence, it is expedient to conduct extensive research on finding a solution for Eq. (1).
This study involves proposition and subsequent investigation of a novel iterative method that can be employed for ensuring the simultaneous RW&HY calibration via Eq. (2). The proposed method serves to solve the rotational and translational parts simultaneously, requires no initial estimates, and provides the optimal solutions in L1-norm. This can be achieved by formulating the simultaneous RW&HY calibration problem as a semi-convex optimization problem [15] and solving it with the help of the alternative linear programming (ALP) method. The latter applies the least absolute deviation L1-norm, which minimizes the sum of the absolute differences between the target and estimated values, and is found to be more robust than the least square-based L2-norm, which minimizes the sum of the square of the differences between the target and estimated values.
The remainder of this article is organized as follows. In Section 2, we review the definition of a semi-convex function and the corresponding alternating iterative method (AIM). In Section 3, two simultaneous RW&HY calibration optimization problems are formulated for homogenous and quaternion spaces, respectively. The implementation details and performance of the proposed solution in both synthetic and real data calibration scenarios are presented in Section 4.
Section snippets
Semi-convex function and alternating iterative method
Let C and D be convex subsets of and respectively. The function f(θ, φ) is treated as a semi-convex function of (θ, φ), only if [15]:
- •
f(θ, φ) is defined on C × D;
- •
For any given θ ∈ C, f(θ, •) is strictly convex on D, and for any given φ ∈ D, f(•, φ) is strictly convex on C.
In fact, there exist many appropriate semi-convex functions that can be used as objective functions for the optimization.
Consider the semi-convex optimization problemwhere f(θ, φ) is defined on C × D,
Simultaneous robot-world and hand-eye calibration
In this section, two formulations of the simultaneous RW&HY calibration problem are presented. Firstly, the calibration problem is reduced to the following minimization problem in the homogeneous space :where the matrix L1 norm is defined as and the symbol vec(•) is the vectorization operator of the matrix. The second formulation uses the dual quaternion parametrization to minimize the vector L1-norm
Experiments
The experiments with simulated and real data were conducted to validate the proposed calibration methods in this section. For the real data experiment, we used a MOTOMAN HP3 serial 6-DOF manipulator with two PointGrey flea2 CCD cameras rigidly attached to its end-effector. We simulated the same setup in the synthetic experiment for better cohesion of simulated and real test results.
Algorithms 1 and 2 were implemented in the CVX Matlab-based modeling system for convex optimization [6] that made
Conclusions
The results obtained in this study demonstrate the feasibility of the ALP-based optimization for solving the simultaneous RW&HY calibration problems. Two alternative options for simultaneous RW&HY calibration parametrization forms, namely (i) homogeneous matrix (ALP1), and (ii) dual quaternion (ALP2) are analyzed in detail. The experimental results obtained strongly suggest that the homogeneous matrix parametrization (ALP1) had better performance than the dual quaternion one (ALP2). Meanwhile,
Conflict of interest
The author declares that he has no conflicts of interest.
Acknowledgments
The author would like to thank Prof. Yuncai Liu of Shanghai Jiao Tong University for his technical supports. This work has been supported by the Specialized Research Fund for the Doctoral Program of Higher Education of China (No.20130131120036), the Promotive Research Fund for Excellent Young and Middle-aged Scientists of Shandong Province (No.BS2013DX027), and the National Natural Science Foundation of China (No.81401543).
References (26)
Kinematik und Quaternionen
(1960)- J.Y. Bouguet, A release of a camera calibration toolbox for matlab,...
Hand-eye calibration using dual quaternions
Int. J. Rob. Res.
(1999)- et al.
Simultaneous robot-world and hand-eye calibration
IEEE Trans. Robot. Autom.
(1998) - et al.
Hand to sensor calibration: a geometrical interpretation of the matrix equation ax=xb
J. Robot. Syst.
(2005) - et al.
Graph implementations for nonsmooth convex programs
Recent Advances in Learning and Control-Lecture Notes in Control and Information Sciences
(2008) - et al.
Globally optimal hand-eye calibration using branch-and-bound
IEEE Trans. Pattern Anal. Mach. Intell.
(2016) - et al.
Hand-eye and robot-world calibration by global polynomial optimization
Proc. IEEE Inter. Conf. Robot. Autom.
(2014) - et al.
Hand-eye calibration
Int. J. Rob. Res.
(1995) - et al.
Simultaneous hand-eye and robot-world calibration by solving the ax=yb problem without correspondence
IEEE Rob. Autom. Lett.
(2016)
New probabilistic approaches to the ax=xb hand-eye calibration without correspondence
Introduction to Theoretical Kinematics
A note on kronecker matrix product and matrix equation systems
SIAM J. Appl. Math.
Cited by (35)
A two-step solution for robot-world calibration made intelligible by implementing Chasles’ motion decomposition in Ad(SE(3))
2024, Mechanism and Machine TheoryRobot-world and hand–eye calibration based on motion tensor with applications in uncalibrated robot
2022, Measurement: Journal of the International Measurement ConfederationA novel singular-free solution based on principle of transfer theory for the hand-eye calibration problem
2022, Mechanism and Machine TheoryCitation Excerpt :According to the quaternion and Kronecker product, Zhao [13] proposed an objective function under infinite norm metric criterion and obtained the optimal solution of hand-eye calibration equation by a second-order cone optimization method without initial value. Then the global optimal method proposed by Zhao [14] based on Alternative linear programming (ALP) can also generate highly accurate results and does not require initial estimates. In addition, based on the polar constraint theory, Heller et al. [15] proposed an optimization index under infinite norm and transformed the hand-eye calibration problem into a nonlinear optimization problem.
Simultaneous robot–world and hand–eye calibration based on a pair of dual equations
2021, Measurement: Journal of the International Measurement Confederation