Skip to main content
SpringerLink
Log in

An Analytically Stable Structure and Motion Observer Based on Monocular Vision

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper presents a novel nonlinear sliding-mode differentiator-based complete-order observer for structure and motion identification with a calibrated monocular camera. In comparison with earlier work that requires prior knowledge of either the Euclidean geometry of the observed object or the linear acceleration of the camera and is restricted to establishing stability and convergence from image-plane measurements of a single tracked feature, the proposed scheme assumes partial velocity state feedback to asymptotically identify the true-scale Euclidean coordinates of numerous observed object features and the unknown motion parameters. The dynamics of the motion parameters are assumed to be described by a model with unknown parameters that incorporates a bounded uncertainty, and a Lyapunov analysis is provided to prove that the observer yields exponentially convergent estimates that converge to a uniform ultimate bound under a generic persistency of excitation condition. Numerical and experimental results are obtained that demonstrate the robust performance of the current scheme in the presence of model error and measurement noise.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Chen, X., Kano, H.: A new state observer for perspective systems. IEEE Trans. Autom. Control. 47(4), 658–663 (2002). doi:10.1109/9.995045

    Article  MathSciNet  Google Scholar 

  2. Karagiannis, D., Astolfi, A.: A new solution to the problem of range identication in perspective vision systems. IEEE Trans. Autom. Control. 50(12), 2074–2077 (2005). doi:10.1109/TAC.2005.860269

    Article  Google Scholar 

  3. Karagiannis, D., Carnevale, D., Astolfi, A.: Invariant manifold based reduced-order observer design for nonlinear systems. IEEE Trans. Autom. Control. 53(11), 2602–2614 (2008)

    Article  MathSciNet  Google Scholar 

  4. Jankovic, M., Ghosh, B.K.: Visually guided ranging from observations of points, lines and curves via an identier based nonlinear observer. Syst. Control Lett. 25, 63–73 (1995)

    Article  MATH  Google Scholar 

  5. Dixon, W.E., Fang, Y., Dawson, D.M., Flynn, T.J.: Range identication for perspective vision systems. IEEE Trans. Autom. Control. 48(12), 2232–2238 (2003)

    Article  Google Scholar 

  6. Luca, A. D., Oriolo, G., Giordano, P. R.: On-line estimation of feature depth for image-based visual servoing schemes In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp. 2823–2828, Roma (2007)

    Google Scholar 

  7. Ma, L., Chen, Y., Moore, K.L.: Range identication for perspective dynamic system with single homogeneous observation In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp. 5207–5211, New Orleans (2004)

    Google Scholar 

  8. Dani, A.P., Fischer, N.R., Kan, Z., Dixon, W.E.: Globally exponentially stable observer for vision-based range estimation. Mechatronics 22(4), 381–389 (2012). doi:10.1016/j.mechatronics.2011.10.001

    Article  Google Scholar 

  9. Grave, I., Tang, Y.: A new observer for perspective vision systems under noisy measurements. IEEE Trans. Autom. Control. 60(2), 503–508 (2015)

    Article  MathSciNet  Google Scholar 

  10. Tsai, R.Y., Huang, T.S.: Estimating three-dimensional motion parameters of a rigid planar patch. IEEE Trans. Acoust. Speech Signal Process. 29(6), 1147–1152 (1981)

    Article  Google Scholar 

  11. Conroy, J., Humbert, S.J.: Structure from motion in computationally constrained systems In: Proceedings of SPIE 8725. doi:10.1117/12.2015296 (2013)

    Google Scholar 

  12. Morbidi, F., Prattichizzo, D.: Range estimation from a moving camera: an immersion and invariance approach In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp. 2810–2815, Kobe (2009)

    Google Scholar 

  13. Dahl, O., Wang, Y., Lynch, A.F., Heyden, A.: Observer forms for perspective systems. Automatica 46(11), 1829–1834 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Nath, N., Braganza, D., Dawson, D.M., Burg, T.: Range identification for perspective vision systems: a position based approach. Int. J. Robot. Autom., 206. doi:10.2316/Journal.206.2011.2.206-3409

  15. Keshavan, J., Humbert, J.S.: An optical flow-based solution to the problem of range identification in perspective vision systems. J. Intell. Robot Syst. (2016). doi:10.1007/s10846-016-0404-6

  16. Broida, T.J., Chellappa, R.: Estimation of object motion parameters from noisy images. IEEE Trans. Pattern Anal. Mach. Intell. 8(1), 90–99 (1986)

    Article  Google Scholar 

  17. Gennery, D.B.: Tracking known three-dimensional objects In: Proceedings of AAAI 2nd National Conference on Artificial Intelligence, pp. 13–17 Pittsburgh (1982)

    Google Scholar 

  18. Gennery, D.B.: Visual tracking of known three-dimensional objects. Int. J. Comp. Vision 7(3), 243–270 (1992)

    Article  Google Scholar 

  19. Dahl, O., Nyberg, F., Heyden, A.: Nonlinear and adaptive observers for perspective dynamic systems In: Proceedings of American Control Conference (ACC), pp. 1966–1971, New York (2007)

    Google Scholar 

  20. Soatto, S., Frezza, R., Perona, P.: Motion Estimation via Dynamic Vision. IEEE Trans. Autom. Control. 41(41), 393–412 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  21. Dani, A., Fischer, N., Dixon, W.E.: Single camera structure motion. IEEE Trans. Autom. Control. 57(1), 241–246 (2012)

    Article  MathSciNet  Google Scholar 

  22. Chen, X., Kano, H.: State observer for a class of nonlinear systems and its application to machine vision. IEEE Trans. Autom. Control. 49(11), 2085–2091 (2004)

    Article  MathSciNet  Google Scholar 

  23. Dahl, O., Heyden, A.: Dynamic structure from motion based on nonlinear adaptive observers In: Proceedings of International Conference on Pattern Recognition, pp 1–4 (2008)

    Google Scholar 

  24. Chitrakaran, V.K., Dawson, D.M., Dixon, W.E., Chen, J.: Identification of a moving object’s velocity with a fixed camera. Automatica 41(3), 553–562 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  25. Sturm, P., Triggs, B.: A factorization based algorithm for multi-image projective structure and motion. Lect. Notes Comput. Sci. 1065, 709–720 (1996)

    Article  Google Scholar 

  26. Ramalingam, S., Lodha, S., Sturm, P.: A generic structure-from-motion framework. Comput. Vis. Image Understand. 103(3), 218–228 (2006)

    Article  Google Scholar 

  27. Oliensis, J.: A critique of structure-from-motion algorithms. Comput. Vis. Image Underst. 80, 172–214 (2000)

    Article  MATH  Google Scholar 

  28. Oliensis, J., Hartley, R.: Iterative extensions of the strum/triggs algorithm: Convergence and nonconvergence. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2217–2233 (2007)

    Article  Google Scholar 

  29. Kahl, F., Hartley, R.: Multiple-view geometry under the \(\mathcal {L}_{\infty -norm}\). IEEE Trans. Pattern Anal. Mach. Intell. 30(9), 1603–1617 (2008)

    Article  Google Scholar 

  30. Azarbayejani, A., Pentland, A.P.: Recursive estimation of motion, structure, and focal length. IEEE Trans. Pattern Anal. Mach. Intell. 17(6), 562–575 (1995). doi:10.1109/34.387503

    Article  Google Scholar 

  31. Matthies, L., Kanade, T., Szeliski, R.: Kalman filter-based algorithm for estimating depth from image sequence. Int. J. Comput. Vision 3, 209–236 (1989)

    Article  Google Scholar 

  32. Sridhar, B., Soursa, R., Hussein, B.: Passive range estimation from rotorcraft low-altitude flight. Inter. J. Mach. Vision App. 6, 10–24 (1993)

    Article  Google Scholar 

  33. Chiuso, A., Favaro, P., Jin, H., Soatto, S.: Structure from motion causally integrated over time. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 523–535 (2002)

    Article  Google Scholar 

  34. Durrant-Whyte, H., Bailey, T.: Simultaneous localization and mapping (slam): Part II. IEEE Robot. Autom. Mag. 13(3), 108–117 (2006)

    Article  Google Scholar 

  35. Davison, A.J., Reid, I.D., Molton, N.D., Stasse, O.: Monoslam: Real-time single camera slam. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)

    Article  Google Scholar 

  36. Ayache, N., Faugeras, O.: Maintaining representations of the environment of a mobile robot. IEEE Trans. Robot. Autom. 6(5), 804–819 (1989)

    Article  Google Scholar 

  37. Dickmanns, E.D., Graefe, V.: Dynamic monocular machine vision. Mach. Vision App. 1, 223–240 (1988)

    Article  Google Scholar 

  38. Faugeras, O.D., Ayache, N., Faverjon, B.: Building visual maps by combining noisy stereo measurements In: Proceedings of IEEE Conference on Robotics and Automation San Francisco (1986)

    Google Scholar 

  39. Young, G.S., Chellappa, R.: 3D motion estimation using a sequence of noisy stereo images: Models, estimation and uniqueness. IEEE Trans. Pattern Anal. Mach. Intell. 12(8), 735–759 (1990)

    Article  Google Scholar 

  40. Reif, K., Sonnermann, F., Unbehauen, R.: An EKF-based nonlinear observer with a prescribed degree of stability. Automatica 34, 1119–1123 (1998)

    Article  MATH  Google Scholar 

  41. Abdursul, R., Inaba, H., Ghosh, B.: Nonlinear observers for perspective time-invariant linear systems. Automatica 40(3), 481–490 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  42. Ghosh, B., Loucks, E.: A realization theory for perspective systems with applications to parameter estimation problems in machine vision. IEEE Trans. Autom. Control. 41(12), 1706–1722 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  43. Ghosh, B., Loucks, E.: A perspective theory for motion and shape estimation in machine vision. SIAM J. Control Optim. 33(5), 1530–1559 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  44. Ma, L., Cao, C., Hovakimayan, N., Woolsey, C., Dixon, W.E.: Fast estimation for range identification in the presence of unknown motion parameters. IMA J. Appl. Math. 75(2), 165–189 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  45. Levant, A.: Robust exact differentiation via sliding mode technique. Automatica 34(3), 379–384 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  46. Suzuki, S., Furuta, K., Shiratori, S.: Adaptive impact shot control by pendulum like juggling system. Int. J. Japan Soc. Mech. Eng. 46(3), 973–981 (2003)

    Google Scholar 

  47. Smaoui, M., Brun, X., Thomasset, D.: A robust differentiator-controller design for an electropneumatic system In: Proceedings of IEEE Conference on Decision and Control, pp. 4385–4390, Seville (2005)

    Google Scholar 

  48. Sidhom, L., Smaoui, M., Thomasset, D., Brun, X., Bideaux, E.: Adaptive higher order sliding modes for two-dimensional derivative estimation. IFAC World Congress, pp. 3063–3071 Mliano (2011)

  49. Soatto, S., Perona, P.: Reducing structure from motion: a general framework for dynamic vision. Part 1: Modeling. IEEE Trans. Pattern Anal. Mach. Intell. 20(9), 933–942 (1996)

    Article  Google Scholar 

  50. Sastry, S., Bodson, M.: Adaptive control: stability, convergence, and robustness Upper Saddle River (NJ): Prentice Hall (1989)

  51. Khalil, H.K.: Nonlinear systems. 3rd ed Prentice Hall (2002)

  52. Bouguet, J.: Camera calibration toolbox for MATLAB http://www.vision.caltech.edu/bouguetj (2007) [retrieved 15 (2014)

  53. Lucas, B. D., Kanade, T.: An iterative image registration technique with an application to stereo vision In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)

    Google Scholar 

  54. Shi, J., Tomasi, C.: Good features to track In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 593–600, Seattle (1994)

    Google Scholar 

  55. Birchfield, S.: KLT: an implementation of the Kanade-Lucas-Tomasi feature tracker http://www.ces.clemson.edu/~stb/klt/ [retrieved 15 March 2015] (2007)

Download references

Author information

Authors and Affiliations

  1. Autonomous Vehicles Laboratory, University of Maryland, College Park, MD, 20742, USA

    Jishnu Keshavan & J. Sean Humbert

Authors
  1. Jishnu Keshavan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Sean Humbert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jishnu Keshavan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Keshavan, J., Humbert, J.S. An Analytically Stable Structure and Motion Observer Based on Monocular Vision. J Intell Robot Syst 86, 495–510 (2017). https://doi.org/10.1007/s10846-017-0470-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-017-0470-4

Keywords

Search

Navigation