Skip to main content
SpringerLink
Log in

Unmanned Aerial Vehicle Localization Based on Monocular Vision and Online Mosaicking

A New Mapping Framework

  • Published:
Journal of Intelligent and Robotic Systems Aims and scope Submit manuscript

Abstract

This paper presents a new approach for vision-based UAV localization, using mosaics as environment representations. Inter-image motions are used to estimate the motion of the UAV. Online mosaicking is applied to reduce the impact of the accumulative errors in UAV position estimation. A new method to build an stochastic mosaic given the image-to-image homographies is detailed. The mosaic consists of a network of inter-image relations, and is used to create a consistent view of the environment of the UAV and hence, to detect the drift in position estimation by using the mosaic as a resource. The technique could be called simultaneous localization and mosaicking. This technique is specially suitable for monitoring and surveillance tasks in which the UAV will repeatedly cover the same area. The paper also shows experimental results with real UAVs where the benefits of the proposed method are evident.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Amidi, O., Kanade, T., Fujita, K.: A visual odometer for autonomous helicopter fight. In: Proceedings of the Fifth International Conference on Intelligent Autonomous Systems (IAS-5), Sapporo, June 1998

  2. Byrne, J., Cosgrove, M., Mehra, R.: Stereo based obstacle detection for an unmanned air vehicle. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, May, pp. 2830–2835. IEEE, Piscataway (2006)

    Chapter  Google Scholar 

  3. Caballero, F., Ferruz, J., Ollero, A.: Image motion compensation in aerial image sequences (in Spanish). In: XXV Jornadas de Automtica (2004, September)

  4. Caballero, F., Merino, L., Ferruz, J., Ollero, A.: A visual odometer without 3D reconstruction for aerial vehicles. Applications to building inspection. In: Proceedings of the International Conference on Robotics and Automation, April, pp. 4684–4689. IEEE, Piscataway (2005)

    Google Scholar 

  5. Caballero, F., Merino, L., Ferruz, J., Ollero, A.: Improving vision-based planar motion estimation for unmanned aerial vehicles through online mosaicing. In: Proceedings of the International Conference on Robotics and Automation, May, pp. 2860–2865. IEEE, Piscataway (2006)

    Google Scholar 

  6. Corke, P., Sikka, P., Roberts, J.M.: Height estimation for an autonomous helicopter. In: Proceedings of ISER, pp. 101–110 (2000)

  7. Del-Cerro, J., Barrientos, A., Campoy, P., García, P.: An autonomous helicopter guided by computer vision for inspection of overhead power cable. In: Proceedings of the Workshop WS6 Aerial Robotics, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems - IROS 2002, pp. 69–78. IEEE, Piscataway (2002)

    Google Scholar 

  8. Demonceaux, C., Vasseur, P., Pegard, C.: Omnidirectional vision on UAV for attitude computation. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, May, pp. 2842–2847. IEEE, Piscataway (2006)

    Chapter  Google Scholar 

  9. Dickmanns, E.D., Schell, F.R.: Autonomous landing of airplanes using dynamic machine vision. In: Proc. of the IEEE Workshop Applications of Computer Vision, December, pp. 172–179. IEEE, Piscataway (1992)

    Chapter  Google Scholar 

  10. Ferruz, J., Ollero, A.: Real-time feature matching in image sequences for non-structured environments. applications to vehicle guidance. J. Intell. Robot. Syst. 28, 85–123 (2000)

    Google Scholar 

  11. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  12. Fukao, T., Fujitani, K., Kanade, T.: Image-based tracking control of a blimp. In: Proc. 42nd IEEE Conference on Decision and Control, vol. 5, pp. 5414–5419. IEEE, Piscataway (2003)

    Google Scholar 

  13. Garcia, R., Batlle, J., Cuf, X., Amat, J.: Positioning an underwater vehicle through image mosaicking. In: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA, vol. 3, pp. 2779–2784, Seoul, Rep. of Korea. IEEE, Piscataway (2001)

    Google Scholar 

  14. Garcia, R., Puig, J., Ridao, P., Cufi, X.: Augmented state kalman filtering for auv navigation. In: Proceedings of the International Conference on Robotics and Automation, pp. 4010–4015, Washington, DC, May 2002

  15. Garcia-Pardo, P.J., Sukhatme, G.S., Montgomery, J.F.: Towards vision-based safe landing for an autonomous helicopter. Robot. Auton. Syst. 38(1), 19–29 (2001)

    Article  Google Scholar 

  16. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)

    MATH  Google Scholar 

  17. Hrabar, S., Sukhatme, G.S.: Omnidirectional vision for an autonomous helicopter. In: Proceedings of the International Conference on Robotics and Automation, vol. 1, pp. 558–563. IEEE, Piscataway (2003)

    Google Scholar 

  18. Hrabar, S., Sukhatme, G.S., Corke, P., Usher, K., Roberts, J.: Combined optic-flow and stereo-based navigation of urban canyons for a UAV. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3309–3316. IEEE, Piscataway (2005)

    Google Scholar 

  19. Hygounenc, E., Jung, I.-K., Soueres, P., Lacroix, S.: The autonomous blimp project of LAAS-CNRS: achievements in flight control and terrain mapping. Int. J. Rob. Res. 23(4–5), 473–511 (2004)

    Article  Google Scholar 

  20. Julier, S., Uhlmann, J.: A new extension of the Kalman filter to nonlinear systems. In: Proceedings of the 11th Int. Symp. on Aerospace/Defence Sensing, Simulation and Controls, Orlando, April 1997

  21. Kim, J., Sukkarieh, S.: Autonomous airborne navigation in unknown terrain environments. IEEE Trans. Aerosp. Electron. Syst. 40(3), 1031–1045 (2004)

    Article  Google Scholar 

  22. Lacroix, S., Jung, I.K., Soueres, P., Hygounenc, E., Berry, J.P.: The autonomous blimp project of LAAS/CNRS—current status and research challenges. In: Proceeding of the International Conference on Intelligent Robots and Systems, IROS, Workshop WS6 Aerial Robotics, pp. 35–42. IEEE/RSJ, Piscataway (2002)

    Google Scholar 

  23. Lemaire, T., Lacroix, S., Solà, J.: A practical 3D bearing only SLAM algorithm. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2449–2454. IEEE/RSJ, Piscataway (2005)

    Google Scholar 

  24. Ling, L., Ridley, M., Kim, J.-H., Nettleton, E., Sukkarieh, S.: Six DoF decentralised SLAM. In: Proceedings of the Australasian Conference on Robotics and Automation, Taipei, September 2003

  25. Mahalanobis, P.C.: On the generalized distance in statistics. In: Proceedings of the National Institute of Science of India, vol. 12, pp. 49–55 (1936)

  26. Mahony, R., Hamel, T.: Image-based visual servo control of aerial robotic systems using linear image features. IEEE Trans. Robot. 21(2), 227–239 (2005)

    Article  Google Scholar 

  27. Mejías, L., Saripalli, S., Campoy, P., Sukhatme, G.S.: Visual servoing of an autonomous helicopter in urban areas using feature tracking. J. Field Robot. 23(3–4), 185–199 (2006)

    Article  Google Scholar 

  28. Ollero, A., Ferruz, J., Caballero, F., Hurtado, S., Merino, L.: Motion compensation and object detection for autonomous helicopter visual navigation in the COMETS system. In: Proceedings of the International Conference on Robotics and Automation, ICRA, pp. 19–24. IEEE, Piscataway (2004)

    Google Scholar 

  29. Ollero, A., Merino, L.: Control and perception techniques for aerial robotics. Ann. Rev. Control 28, 167–178 (2004)

    Article  Google Scholar 

  30. Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. Wiley-IEEE, New York (1987)

    Book  MATH  Google Scholar 

  31. Saripalli, S., Montgomery, J.F., Sukhatme, G.S.: Visually guided landing of an unmanned aerial vehicle. IEEE Trans. Robot. Autom. 19(3), 371–380 (2003)

    Article  Google Scholar 

  32. Saripalli, S., Sukhatme, G.S.: Landing on a mobile target using an autonomous helicopter. In: Proceedings of the International Conference on Field and Service Robotics, July, FSR (2003)

  33. Shakernia, O., Vidal, R., Sharp, C., Ma, Y., Sastry, S.: Multiple view motion estimation and control for landing an aerial vehicle. In: Proceedings of the International Conference on Robotics and Automation, ICRA, May, vol. 3, pp. 2793–2798. IEEE, Piscataway (2002)

    Google Scholar 

  34. Tardos, J., Neira, J., Newman, P., Leonard, J.: Robust mapping and localization in indoor environments using sonar data. Int. J. Rob. Res. 24(4), 311–330 (2002)

    Article  Google Scholar 

  35. Triggs, B.: Autocalibration from planar scenes. In: Proceedings of the 5th European Conference on Computer Vision, ECCV, vol. 1, pp. 89–105. Springer, London (1998)

    Google Scholar 

  36. Vidal, R., Sastry, S., Kim, J., Shakernia, O., Shim, D.: The Berkeley aerial robot project (BEAR). In: Proceeding of the International Conference on Intelligent Robots and Systems, IROS, pp. 1–10. IEEE/RSJ, Piscataway (2002)

    Google Scholar 

  37. Welch, G., Bishop, G.: An introduction to the Kalman Filter. Technical Report 95-041, University of North Carolina at Chapel Hill (1995)

  38. Wu, A.D., Johnson, E.N., Proctor, A.A.: Vision-aided inertial navigation for flight control. In: Proc. of AIAA Guidance, Navigation, and Control Conference and Exhibit (2005)

  39. Yakimenko, O.A., Kaminer, I.I., Lentz, W.J., Ghyzel, P.A.: Unmanned aircraft navigation for shipboard landing using infrared vision. IEEE Trans. Aerosp. Electron. Syst. 38(4), 1181–1200 (2002)

    Article  Google Scholar 

  40. Zhang, H., Ostrowski, J.P.: Visual servoing with dynamics: control of an unmanned blimp. In: Proceedings of the International Conference on Robotics and Automation, vol. 1, pp. 618–623 (1999)

  41. Zhang, Z.: Parameters estimation techniques. a tutorial with application to conic fitting. Technical report, INRIA, France (1995, October)

  42. Zhang, Z., Hintz, K.J.: Evolving neural networks for video attitude and hight sensor. In: Proc. of the SPIE International Symposium on Aerospace/Defense Sensing and Control, April, vol. 2484, pp. 383–393 (1995)

Download references

Author information

Authors and Affiliations

  1. Robotics, Vision and Control Group, University of Seville, Avd. de los Descubrimientos s/n, 41092, Sevilla, Spain

    Fernando Caballero, Joaquín Ferruz & Aníbal Ollero

  2. Pablo de Olavide University, Crta. Utrera km1, 41013, Sevilla, Spain

    Luis Merino

  3. Centro Avanzado de Tecnologías Aeroespaciales, Aéropolis, N-IV, km. 529 C/ Wilbur y Orville Wright, 41309, La Rinconada, Sevilla, Spain

    Aníbal Ollero

Authors
  1. Fernando Caballero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luis Merino
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joaquín Ferruz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aníbal Ollero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando Caballero.

Additional information

This work was partially supported by the AWARE project (IST-2006-33579) funded by the European Commission, and the Spanish projects SADCON (2005/TEP-375, Junta de Andaluca) and ROBAUCO (Profit Programme of the Spanish Ministry for Science and Technology).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Caballero, F., Merino, L., Ferruz, J. et al. Unmanned Aerial Vehicle Localization Based on Monocular Vision and Online Mosaicking . J Intell Robot Syst 55, 323–343 (2009). https://doi.org/10.1007/s10846-008-9305-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-008-9305-7

Keywords

Search

Navigation