Abstract
Real-time and robust tracking of 3D objects based on a 3D model with multiple cameras is still an unsolved problem albeit relevant in many practical and industrial applications. Major problems are caused by appearance changes of the object. We present a template-based tracking algorithm for piecewise planar objects. It is robust against changes in the appearance of the object (occlusion, illumination variation, specularities). The version we propose supports multiple cameras. The method consists in minimizing the error between the observed images of the object and the warped images of the planes. We use the mutual information as registration function combined with an inverse composition approach for reducing the computational costs and get a near-real-time algorithm. We discuss different hypotheses that can be made for the optimization algorithm.






Similar content being viewed by others
References
Baker, S., Matthews, I.: Lucas-kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(3), 221–255 (2004). doi:10.1023/B:VISI.0000011205.11775.fd
Benhimane, S., Malis, E.: Integration of euclidean constraints in template based visual tracking of piecewise-planar scenes. In: Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, pp. 1218–1223 (2006). doi:10.1109/IROS.2006.281859
Benhimane, S., Malis, E.: Homography-based 2d visual tracking and servoing. Int. J. Rob. Res. 26(7), 661–676 (2007). doi:10.1177/0278364907080252
Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, Hoboken (2012)
Dame, A., Marchand, E.: Mutual information-based visual servoing. IEEE Trans. Robot. 27(5), 958–969 (2011). doi:10.1109/TRO.2011.2147090
Dame, A., Marchand, E.: Second-order optimization of mutual information for real-time image registration. IEEE Trans. Image Process. 21(9), 4190–4203 (2012). doi:10.1109/TIP.2012.2199124
Delabarre, B., Marchand, E.: Camera localization using mutual information-based multiplane tracking. In: Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, pp. 1620–1625 (2013). doi:10.1109/IROS.2013.6696566
Dowson, N., Bowden, R.: A unifying framework for mutual information methods for use in non-linear optimisation. In: A. Leonardis, H. Bischof, A. Pinz (eds.) Computer Vision ECCV 2006, Lecture Notes in Computer Science, vol. 3951, pp. 365–378. Springer, Berlin Heidelberg (2006). doi:10.1007/11744023_29
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Fraissinet-Tachet, M., Kuijper, A., Schmitt, M.: Mutual information-based piecewise planar object tracking. Master’s thesis, Fraunhofer IGD, Technische Universitaet Darmstadt (2014)
Jiang, N., Cui, Z., Tan, P.: A global linear method for camera pose registration. In: IEEE International Conference on Computer Vision, ICCV 2013, Sydney, 1–8 Dec 2013, pp. 481–488 (2013). doi:10.1109/ICCV.2013.66
Kneip, L., Li, H.: Efficient computation of relative pose for multi-camera systems. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, 23–28 June 2014, pp. 446–453 (2014). doi:10.1109/CVPR.2014.64
Kuijper, A.: Mutual information aspects of scale space images. Pattern Recogn. 37(12), 2361–2373 (2004). doi:10.1016/j.patcog.2004.04.014
Lee, G.H., Pollefeys, M., Fraundorfer, F.: Relative pose estimation for a multi-camera system with known vertical direction. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, 23–28 June 2014, pp. 540–547 (2014). doi:10.1109/CVPR.2014.76
Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision. IJCAI 81, 674–679 (1981)
Malis, E., Vargas, M.: Deeper understanding of the homography decomposition for vision-based control. Research Report RR-6303, INRIA (2007). http://hal.inria.fr/inria-00174036
Metz, C.E.: Basic principles of ROC analysis. Semin. Nucl. Med. 8(4), 283–298 (1978)
Panin, G., Knoll, A.: Mutual information-based 3d object tracking. Int. J. Comput. Vision 78(1), 107–118 (2008). doi:10.1007/s11263-007-0083-7
Pluim, J.P.W., Maintz, J., Viergever, M.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22(8), 986–1004 (2003). doi:10.1109/TMI.2003.815867
Prisacariu, V.A., Kähler, O., Murray, D.W., Reid, I.D.: Simultaneous 3d tracking and reconstruction on a mobile phone. In: IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2013, Adelaide, 1–4 Oct 2013, pp. 89–98 (2013). doi:10.1109/ISMAR.2013.6671768
Rios-Cabrera, R., Tuytelaars, T.: Discriminatively trained templates for 3d object detection: A real time scalable approach. In: IEEE International Conference on Computer Vision, ICCV 2013, Sydney, 1–8 Dec 2013, pp. 2048–2055 (2013). doi:10.1109/ICCV.2013.256
Rios-Cabrera, R., Tuytelaars, T., Gool, L.J.V.: Efficient multi-camera detection, tracking, and identification using a shared set of haar-features. In: The 24th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011, Colorado Springs, 20–25 June 2011, pp. 65–71 (2011). doi:10.1109/CVPR.2011.5995735
Roulston, M.S.: Estimating the errors on measured entropy and mutual information. Physica D: Nonlinear Phenomena 125(34), 285–294 (1999). doi:10.1016/S0167-2789(98)00269-3 http://www.sciencedirect.com/science/article/pii/S0167278998002693
Thevenaz, P., Unser, M.: Optimization of mutual information for multiresolution image registration. IEEE Trans. Image Process. 9(12), 2083–2099 (2000). doi:10.1109/83.887976
Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3d tracking using online and offline information. IEEE Trans. Pattern Anal. Mach. Intell. 26(10), 1385–1391 (2004)
Vigueras-Gomez, J.F., Sclaroff, S.: Fast vision-based scene modeling for augmented reality in unprepared man-made environments. J. Ambient Intell. Smart Environ. 5(5), 525–537 (2013). http://dl.acm.org/citation.cfm?id=2594708.2594716
Viola, P., Wells, W.: Alignment by maximization of mutual information. In: Computer Vision, 1995. Proceedings., Fifth International Conference on, pp. 16–23 (1995). doi:10.1109/ICCV.1995.466930
Wagner, D., Reitmayr, G., Mulloni, A., Méndez, E., Diaz, S.: Mobile augmented reality - tracking, mapping and rendering. In: IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2014, Munich, 10–12 Sept 2014, p. 383 (2014). doi:10.1109/ISMAR.2014.6948500
Wu, Y., Lim, J., Yang, M.: Online object tracking: a benchmark. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, 23–28 June 2013, pp. 2411–2418 (2013). doi:10.1109/CVPR.2013.312
Acknowledgments
This work has been developed in the Software Campus project CAD-VISION. CAD-VISION (Reference No: 01IS12053) is partly funded by the German ministry of education and research (BMBF) within the research programme ICT 2020.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fraissinet-Tachet, M., Schmitt, M., Wen, Z. et al. Multi-Camera Piecewise Planar Object Tracking with Mutual Information. J Math Imaging Vis 56, 591–602 (2016). https://doi.org/10.1007/s10851-016-0660-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10851-016-0660-x