Skip to main content
SpringerLink
Log in

Tracking 3-D motion from straight lines with trifocal tensors

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

We present a novel approach to track the position and orientation of a stereo camera using line features in the images. The method combines the strengths of trifocal tensors and Bayesian filtering. The trifocal tensor provides a geometric constraint to lock line features among every three frames. It eliminates the explicit reconstruction of the scene even if the 3-D scene structure is not known. Such a trifocal constraint thus makes the algorithm fast and robust. The twist motion model is applied to further improve its computation efficiency. Another major contribution is that our approach can obtain the 3-D camera motion using as little as 2 line correspondences instead of 13 in the traditional approaches. This makes the approach attractive for realistic applications. The performance of the proposed method has been evaluated using both synthetic and real data with encouraging results. Our algorithm is able to estimate 3-D camera motion in real scenarios accurately having little drifting from an image sequence longer than a 1,000 frames.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Aldoma, A., Marton, Z.C., Tombari, F., Wohlkinger, W., Potthast, C., Zeisl, B., Rusu, R.B., Gedikli, S., Vincze, M.: Tutorial: point cloud library: three-dimensional object recognition and 6 dof pose estimation. IEEE Robot. Autom. Mag. 19(3), 80–91 (2012)

    Article  Google Scholar 

  2. Andreff, N., Espiau, B., Horaud, R.: Visual servoing from lines. Int. J. Robot. Res. 21(8), 679–699 (2002)

    Article  Google Scholar 

  3. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 679–698 (1986)

    Article  Google Scholar 

  4. Chandraker, M., Lim, J., Kriegman, D.: Moving in stereo: efficient structure and motion using lines. In: Proceedings of the 12th International Conference on Computer Vision 2009, pp. 1741–1748 (2009)

  5. Chen, H.H.: Pose determination from line-to-plane correspondences: existence condition and closed-form solutions. In: Proceedings of the Third International Conference on Computer Vision 1990, pp. 374–378 (1990)

  6. Chiuso, A., Favaro, P., Jin, H., Soatto, S.: Structure from motion causally integrated over time. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 523–535 (2002)

    Article  Google Scholar 

  7. Christy, S., Horaud, R.: Iterative pose computation from line correspondences. Comput. Vis. Image Underst. 73(1), 137–144 (1999)

    Article  MATH  Google Scholar 

  8. Elqursh, A., Elgammal, A.: Line-based relative pose estimation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2011, pp. 3049–3056 (2011)

  9. Hartley, R.I.: Lines and points in three views and the trifocal tensor. Int. J. Comput. Vis. 22(2), 125–140 (1997)

    Article  Google Scholar 

  10. Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Image Understanding Workshop, pp. 121–130 (1981)

  11. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  12. Bouguet, J.Y.: Camera calibration toolbox for matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/ (2010)

  13. Lippiello, V., Siciliano, B., Villani, L.: Position and orientation estimation based on Kalman filtering of stereo images. In: Proceedings of the IEEE International Conference on Control Applications, 2001, pp. 702–707 (2001)

  14. Lowe, D.G.: Fitting parameterized three-dimensional models to images. IEEE Trans. Pattern Anal. Mach. Intell. 13(5), 441–450 (1991)

    Article  MathSciNet  Google Scholar 

  15. Qin, L.-J., Zhu, F.: A new method for pose estimation from line correspondences. Acta Automatica Sinica 34(2), 130–134 (2008)

    Article  MathSciNet  Google Scholar 

  16. Rhee, S.-M., Lee, Y.-B., Kim, J.D., Rhee, T.: Pose estimation of a depth camera using plane features. In: IEEE International Conference Consumer Electronics 2013, pp. 133–134 (2013)

  17. Schmid, C. Zisserman, A.N.: Automatic line matching across views. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1997, pp. 666–671 (1997)

  18. Schmid, C., Zisserman, A.: The geometry and matching of lines and curves over multiple views. Int. J. Comput. Vis. 40(3), 199–233 (2000)

    Article  MATH  Google Scholar 

  19. Taylor, C.J., Kriegman, D.J.: Structure and motion from line segments in multiple images. IEEE Trans. Pattern Anal. Mach. Intell. 17(11), 1021–1032 (1995)

    Article  Google Scholar 

  20. Wang, Z., Wu, F., Hu, Z.: MSLD: a robust descriptor for line matching. Pattern Recogn. 42(5), 941–953 (2009)

    Article  Google Scholar 

  21. Werner, T., Pajdla, T.: Oriented matching constraints. In: Proceedings of British Machine Vision Conference, pp. 441–450 (2001)

  22. Ye, M., Wang, X., Yang, R., Ren, L., Pollefeys, M.: Accurate 3d pose estimation from a single depth image. In: Proceedings of International Conference on Computer Vision, pp. 731–738 (2011)

  23. Yu, Y.K., Wong, K.H., Chang, M.M.Y., Or, S.H.: Recursive camera-motion estimation with the trifocal tensor. IEEE Trans. Syst. Man Cybern. B Cybern. 36(5), 1081–1090 (2006)

    Article  Google Scholar 

  24. Yu, Y.K., Wong, K.H., Chang, M.M.Y.: Pose estimation for augmented reality applications using genetic algorithm. IEEE Trans. Syst. Man Cybern. B Cybern. 35(6), 1295–1301 (2005)

    Article  Google Scholar 

  25. Yu, Y.K., Wong, K.H., Or, S.H., Chang, M.M.Y.: Robust 3-d motion tracking from stereo images: a model-less method. IEEE Trans. Instrum. Meas. 57(3), 622–630 (2008)

  26. Zhang, Z., Shan, Y.: Incremental motion estimation through modified bundle adjustment. In: Proceedings of International Conference on Image Processing, vol. 2, p. 343 (2003)

  27. Li, Z., Sastry, S.S., Murray, R.: A mathematical introduction to robotic manipulation. CRC Press (1994)

  28. Comport, A.I., Malis, E., Rives, P.: Accurate quadrifocal tracking for robust 3d visual odometry. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 40–45 (2007)

  29. Weng, J., Huang, T.S., Ahuja, N.: Motion and structure from line correspondences; closed-form solution, uniqueness, and optimization. IEEE Trans. Pattern Anal. Mach. Intell. 14(3), 318–336 (1992)

    Article  Google Scholar 

  30. Yu, Y.K., Wong, K.H., Or, S.H., Junzhou, C.: Controlling virtual cameras based on a robust model-free pose acquisition technique. IEEE Trans Multimed 11(1), 184–190 (2009)

    Article  Google Scholar 

  31. Yu, Y.K., Wong, K.H., Chang, M.M.Y.: Merging artificial objects with marker-less video sequences based on the interacting multiple model method. IEEE Trans Multimed 8(3), 521–528 (2006)

    Article  Google Scholar 

  32. Yu, Y.K., Wong, K.H., Chang, M.Y.Y.: A fast recursive 3D model reconstruction algorithm for multimedia applications. In: Proceeding of the International Conference on Pattern Recognition, vol. 2, pp. 241–244 (2004)

Download references

Author information

Authors and Affiliations

  1. Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong, China

    Kai Ki Lee & Michael Ming Yuen Chang

  2. Hong Kong, China

    Ying Kin Yu

  3. Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China

    Kin Hong Wong

Authors
  1. Kai Ki Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Kin Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kin Hong Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Ming Yuen Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai Ki Lee.

Additional information

Communicated by Q. Tian.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, K.K., Yu, Y.K., Wong, K.H. et al. Tracking 3-D motion from straight lines with trifocal tensors. Multimedia Systems 22, 181–195 (2016). https://doi.org/10.1007/s00530-014-0439-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-014-0439-0

Keywords

Search

Navigation