Skip to main content
SpringerLink
Log in

Multi-Camera Piecewise Planar Object Tracking with Mutual Information

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Real-time and robust tracking of 3D objects based on a 3D model with multiple cameras is still an unsolved problem albeit relevant in many practical and industrial applications. Major problems are caused by appearance changes of the object. We present a template-based tracking algorithm for piecewise planar objects. It is robust against changes in the appearance of the object (occlusion, illumination variation, specularities). The version we propose supports multiple cameras. The method consists in minimizing the error between the observed images of the object and the warped images of the planes. We use the mutual information as registration function combined with an inverse composition approach for reducing the computational costs and get a near-real-time algorithm. We discuss different hypotheses that can be made for the optimization algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Baker, S., Matthews, I.: Lucas-kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(3), 221–255 (2004). doi:10.1023/B:VISI.0000011205.11775.fd

    Article  Google Scholar 

  2. Benhimane, S., Malis, E.: Integration of euclidean constraints in template based visual tracking of piecewise-planar scenes. In: Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, pp. 1218–1223 (2006). doi:10.1109/IROS.2006.281859

  3. Benhimane, S., Malis, E.: Homography-based 2d visual tracking and servoing. Int. J. Rob. Res. 26(7), 661–676 (2007). doi:10.1177/0278364907080252

    Article  Google Scholar 

  4. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, Hoboken (2012)

    MATH  Google Scholar 

  5. Dame, A., Marchand, E.: Mutual information-based visual servoing. IEEE Trans. Robot. 27(5), 958–969 (2011). doi:10.1109/TRO.2011.2147090

    Article  Google Scholar 

  6. Dame, A., Marchand, E.: Second-order optimization of mutual information for real-time image registration. IEEE Trans. Image Process. 21(9), 4190–4203 (2012). doi:10.1109/TIP.2012.2199124

    Article  MathSciNet  Google Scholar 

  7. Delabarre, B., Marchand, E.: Camera localization using mutual information-based multiplane tracking. In: Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, pp. 1620–1625 (2013). doi:10.1109/IROS.2013.6696566

  8. Dowson, N., Bowden, R.: A unifying framework for mutual information methods for use in non-linear optimisation. In: A. Leonardis, H. Bischof, A. Pinz (eds.) Computer Vision ECCV 2006, Lecture Notes in Computer Science, vol. 3951, pp. 365–378. Springer, Berlin Heidelberg (2006). doi:10.1007/11744023_29

  9. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  10. Fraissinet-Tachet, M., Kuijper, A., Schmitt, M.: Mutual information-based piecewise planar object tracking. Master’s thesis, Fraunhofer IGD, Technische Universitaet Darmstadt (2014)

  11. Jiang, N., Cui, Z., Tan, P.: A global linear method for camera pose registration. In: IEEE International Conference on Computer Vision, ICCV 2013, Sydney, 1–8 Dec 2013, pp. 481–488 (2013). doi:10.1109/ICCV.2013.66

  12. Kneip, L., Li, H.: Efficient computation of relative pose for multi-camera systems. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, 23–28 June 2014, pp. 446–453 (2014). doi:10.1109/CVPR.2014.64

  13. Kuijper, A.: Mutual information aspects of scale space images. Pattern Recogn. 37(12), 2361–2373 (2004). doi:10.1016/j.patcog.2004.04.014

    Article  MATH  Google Scholar 

  14. Lee, G.H., Pollefeys, M., Fraundorfer, F.: Relative pose estimation for a multi-camera system with known vertical direction. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, 23–28 June 2014, pp. 540–547 (2014). doi:10.1109/CVPR.2014.76

  15. Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision. IJCAI 81, 674–679 (1981)

    Google Scholar 

  16. Malis, E., Vargas, M.: Deeper understanding of the homography decomposition for vision-based control. Research Report RR-6303, INRIA (2007). http://hal.inria.fr/inria-00174036

  17. Metz, C.E.: Basic principles of ROC analysis. Semin. Nucl. Med. 8(4), 283–298 (1978)

    Article  MathSciNet  Google Scholar 

  18. Panin, G., Knoll, A.: Mutual information-based 3d object tracking. Int. J. Comput. Vision 78(1), 107–118 (2008). doi:10.1007/s11263-007-0083-7

    Article  Google Scholar 

  19. Pluim, J.P.W., Maintz, J., Viergever, M.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22(8), 986–1004 (2003). doi:10.1109/TMI.2003.815867

    Article  MATH  Google Scholar 

  20. Prisacariu, V.A., Kähler, O., Murray, D.W., Reid, I.D.: Simultaneous 3d tracking and reconstruction on a mobile phone. In: IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2013, Adelaide, 1–4 Oct 2013, pp. 89–98 (2013). doi:10.1109/ISMAR.2013.6671768

  21. Rios-Cabrera, R., Tuytelaars, T.: Discriminatively trained templates for 3d object detection: A real time scalable approach. In: IEEE International Conference on Computer Vision, ICCV 2013, Sydney, 1–8 Dec 2013, pp. 2048–2055 (2013). doi:10.1109/ICCV.2013.256

  22. Rios-Cabrera, R., Tuytelaars, T., Gool, L.J.V.: Efficient multi-camera detection, tracking, and identification using a shared set of haar-features. In: The 24th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011, Colorado Springs, 20–25 June 2011, pp. 65–71 (2011). doi:10.1109/CVPR.2011.5995735

  23. Roulston, M.S.: Estimating the errors on measured entropy and mutual information. Physica D: Nonlinear Phenomena 125(34), 285–294 (1999). doi:10.1016/S0167-2789(98)00269-3 http://www.sciencedirect.com/science/article/pii/S0167278998002693

  24. Thevenaz, P., Unser, M.: Optimization of mutual information for multiresolution image registration. IEEE Trans. Image Process. 9(12), 2083–2099 (2000). doi:10.1109/83.887976

    Article  MATH  Google Scholar 

  25. Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3d tracking using online and offline information. IEEE Trans. Pattern Anal. Mach. Intell. 26(10), 1385–1391 (2004)

    Article  Google Scholar 

  26. Vigueras-Gomez, J.F., Sclaroff, S.: Fast vision-based scene modeling for augmented reality in unprepared man-made environments. J. Ambient Intell. Smart Environ. 5(5), 525–537 (2013). http://dl.acm.org/citation.cfm?id=2594708.2594716

  27. Viola, P., Wells, W.: Alignment by maximization of mutual information. In: Computer Vision, 1995. Proceedings., Fifth International Conference on, pp. 16–23 (1995). doi:10.1109/ICCV.1995.466930

  28. Wagner, D., Reitmayr, G., Mulloni, A., Méndez, E., Diaz, S.: Mobile augmented reality - tracking, mapping and rendering. In: IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2014, Munich, 10–12 Sept 2014, p. 383 (2014). doi:10.1109/ISMAR.2014.6948500

  29. Wu, Y., Lim, J., Yang, M.: Online object tracking: a benchmark. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, 23–28 June 2013, pp. 2411–2418 (2013). doi:10.1109/CVPR.2013.312

Download references

Acknowledgments

This work has been developed in the Software Campus project CAD-VISION. CAD-VISION (Reference No: 01IS12053) is partly funded by the German ministry of education and research (BMBF) within the research programme ICT 2020.

Author information

Authors and Affiliations

  1. Fraunhofer IGD, Fraunhoferstrae 5, 64283, Darmstadt, Germany

    Matthieu Fraissinet-Tachet, Michael Schmitt & Arjan Kuijper

  2. Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, 130033, China

    Zhuoman Wen

  3. University of the Chinese Academy of Sciences, Beijing, 100049, China

    Zhuoman Wen

Authors
  1. Matthieu Fraissinet-Tachet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Schmitt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhuoman Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arjan Kuijper
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthieu Fraissinet-Tachet.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fraissinet-Tachet, M., Schmitt, M., Wen, Z. et al. Multi-Camera Piecewise Planar Object Tracking with Mutual Information. J Math Imaging Vis 56, 591–602 (2016). https://doi.org/10.1007/s10851-016-0660-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-016-0660-x

Keywords

Search

Navigation