Abstract
This paper is concerned with the problem of estimating the relative orientation between an inertial measurement unit (IMU) and a camera. Unlike most existing IMU-camera calibrations, the main challenge in this paper is that the information output from the IMU is incomplete. For example, only two tilt information can be read from the gravity sensor of a smart phone. Despite incomplete inertial information, there are strong restrictions between the IMU and camera coordinate systems. This paper addresses the incomplete information based IMU-camera calibration problem by exploiting the intrinsic restrictions among the coordinate transformations. First, the IMU transformation between two poses is formulated with the unknown IMU information. Then the defective IMU information is restored using the complementary visual information. Finally, the Levenberg-Marquardt (LM) algorithm is applied to estimate the optimal calibration result in noisy environments. Experiments on both synthetic and real data show the validity and robustness of our algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bajd, T., Mihelj, M., Munih, M., 2013. Introduction to Robotics. Springer, the Netherlands. [doi:10.1007/978-94-007-6101-8]
Chen, H.H., 1991. A screw motion approach to uniqueness analysis of head-eye geometry. Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, p.145–151. [doi:10.1109/CVPR.1991.139677]
Chow, J.C.K., Kamel, M., 1991. Finding the position and orientation of a sensor on a robot manipulator using quaternions. Int. J. Robot. Res., 10(3):240–254. [doi:10.1177/027836499101000305]
Fassi, I., Legnani, G., 2005. Hand to sensor calibration: a geometrical interpretation of the matrix equation AX=XB. J. Robot. Syst., 22(9):497–506. [doi:10.1002/rob.20082]
Gu, Z.P., 2011. A Study on Monocular Simultaneous Localization and Mapping. PhD Thesis, Chinese Academy of Sciences, Beijing, China (in Chinese).
Gu, Z.P., Dong, Q.L., 2012. Monocular SLAM based on partial IMU information. J. Comput. Aid. Des. Comput. Graph., 24(2):155–160 (in Chinese).
Hartley, R., Zisserman, A., 2003. Multiple View Geometry in Computer Vision. Cambridge University Press, NY, USA.
Heller, J., Havlena, M., Pajdla, T., 2012. A branch-and-bound algorithm for globally optimal hand-eye calibration. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, p.1608–1615. [doi:10.1109/CVPR.2012.6247853]
Horaud, R., Dornaika, F., 1995. Hand-eye calibration. Int. J. Robot. Res., 14(3):195–210. [doi:10.1177/027836499501400301]
Lobo, J., Dias, J., 2007. Relative pose calibration between visual and inertial sensors. Int. J. Robot. Res., 26(6):561–575. [doi:10.1177/0278364907079276]
Lu, Y., Chou, J.C.K., 1995. Eight-space quaternion approach for robotic hand-eye calibration. Proc. IEEE Int. Conf. on Systems, Man and Cybernetics, p.3316–3321. [doi:10.1109/ICSMC.1995.538297]
Ma, S., Zhang, Z., 1998. Computer Vision. Science Press, Beijing, China (in Chinese).
Marquardt, D.W., 1963. An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math., 11(2):431–441. [doi:10.1137/0111030]
Mirzaei, F.M., Roumeliotis, S.I., 2008. A Kalman filter-based algorithm for IMU-camera calibration: observability analysis and performance evaluation. IEEE Trans. Robot., 24(5):1143–1156. [doi:10.1109/TRO.2008.2004486]
Nützi, G., Weiss, S., Scaramuzza, D., et al., 2011. Fusion of IMU and vision for absolute scale estimation in monocular SLAM. J. Intell. Robot. Syst., 61(1–4):287–299. [doi:10.1007/s10846-010-9490-z]
Park, F.C., Martin, B.J., 1994. Robot sensor calibration: solving AX=XB on the Euclidean group. IEEE Trans. Robot. Autom., 10(5):717–721. [doi:10.1109/70.326576]
Seo, Y., Choi, Y.J., Lee, S.W., 2009. A branch-and-bound algorithm for globally optimal calibration of a camera-and-rotation-sensor system. Proc. 12th Int. Conf. on Computer Vision, p.1173–1178. [doi:10.1109/ICCV.2009.5459343]
Shiu, Y.C., Ahmad, S., 1989. Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form AX=XB. IEEE Trans. Robot. Autom., 5(1):16–29. [doi:10.1109/70.88014]
Strobl, K.H., Hirzinger, G., 2006. Optimal hand-eye calibration. Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, p.4647–4653. [doi:10.1109/IROS.2006.282250]
Tsai, R.Y., Lenz, R.K., 1989. A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Trans Robot. Autom., 5(3):345–358. [doi:10.1109/70.34770]
Vicentini, F., Pedrocchi, N., Malosio, M., et al., 2011. High-accuracy hand-eye calibration from motion on manifolds. Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, p.3327–3334. [doi:10.1109/IROS.2011.6094587]
Wang, Y., Ma, X., Wang, Y., et al., 2012. Location optimization of multiple distribution centers under fuzzy environment. J. Zhejiang Univ.-Sci. A (Appl. Phys. & Eng.), 13(10):782–798. [doi:10.1631/jzus.A1200137]
Zhang, Z., 2000. A flexible new technique for camera calibration. IEEE Trans. Patt. Anal. Mach. Intell., 22(11): 1330–1334. [doi:10.1109/34.888718]
Zhao, Z., 2011. Hand-eye calibration using convex optimization. Proc. IEEE Int. Conf. on Robotics and Automation, p.2947–2952. [doi:10.1109/ICRA.2011.5979569]
Zhao, Z., Liu, Y., 2009. A hand-eye calibration algorithm based on screw motions. Robotica, 27(2):217–223. [doi:10.1017/S0263574708004608]
Author information
Authors and Affiliations
Corresponding author
Additional information
Project supported by the National Natural Science Foundation of China (Nos. 61340046, 60875050, and 60675025), the National High-Tech R&D Program (863) of China (No. 2006AA04Z247), the Science and Technology Innovation Commission of Shenzhen Municipality (Nos. JCYJ20120614152234873, CXC20110421001 0A, JCYJ20130331144631730, and JCYJ20130331144716089), and the Specialized Research Fund for the Doctoral Programme of Higher Education of China (No. 20130001110011)
Rights and permissions
About this article
Cite this article
Liu, H., Zhou, Yl. & Gu, Zp. Inertial measurement unit-camera calibration based on incomplete inertial sensor information. J. Zhejiang Univ. - Sci. C 15, 999–1008 (2014). https://doi.org/10.1631/jzus.C1400038
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.C1400038