Skip to main content
SpringerLink
Log in

Optical-Aided Aircraft Navigation using Decoupled Visual SLAM with Range Sensor Augmentation

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper presents an optical-aided navigation method for automatic flights where satellite navigation might be disturbed. The proposed solution follows common approaches where satellite position updates are replaced with measurements from environment sensors such as a camera, lidar or radar as required. The alternative positioning is determined by a localization and mapping (SLAM) algorithm that handles 2D feature inputs from monocular camera images as well as 3D inputs from camera images that are augmented by range measurements. The method requires neither known landmarks nor a globally flat terrain. Beside the visual SLAM algorithm, the paper describes how to generate 3D feature inputs from lidar and radar sources and how to benefit from both monocular triangulation and 3D features. Regarding state estimation, the approach decouples visual SLAM from the filter updates. This allows software and hardware separation, i.e. visual SLAM computations on powerful hardware while the main filter can be installed on real-time hardware with possible lower capabilities. The localization quality in case of satellite dropouts is tested with data sets from manned and unmanned flights with different sensors while keeping all parameters constant. The tests show the applicability of this method in flat and hilly terrain and with different path lengths from few hundred meters to many kilometers. The relative navigation achieves an accumulation error of 1–6 % of distance traveled depending on the flight scenario. In addition to the flights, the paper discusses flight profile limitations when optical navigation methods are used.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abeywardena, D., Wang, Z., Kodagoda, S., Dissanayake, G.: Visual-Inertial Fusion for Quadrotor Micro Air Vehicles with Improved Scale Observability. In: IEEE International Conference on Robotics and Automation, pp. 3133–3138 (2013)

  2. Amidi, O., Kanade, T., Fujita, K.: A visual odometer for autonomous helicopter flight. Journal of Robotics and Autonomous Systems 28, 185–193 (1999)

    Article  Google Scholar 

  3. Andert, F., Ammann, N., Püschel, J., Dittrich, J.: On the Safe Navigation Problem for Unmanned Aircraft: Visual Odometry and Alignment Optimizations for UAV Positioning. In: International Conference on Unmanned Aircraft Systems (ICUAS), pp. 734–743 (2014)

  4. Andert, F., Lorenz, S., Mejias, L., Bratanov, D.: Radar-Aided Optical Navigation for Long and Large-Scale Flights over Unknown and Non-Flat Terrain. In: International Conference on Unmanned Aircraft Systems (ICUAS), pp. 465–474 (2016)

  5. Andert, F., Mejias, L.: Improving Monocular SLAM with Altimeter Hints for Fixed-Wing Aircraft Navigation and Emergency Landing. In: International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1008–1016 (2015)

  6. Bryson, M., Sukkarieh, S.: Building a robust implementation of bearing-only inertial slam for a uav. J. Field Rob. 24(1-2), 113–143 (2007)

    Article  Google Scholar 

  7. Bryson, M., Sukkarieh, S.: Inertial Sensor-Based Simultaneous Localization and Mapping for UAVs, chap. 21, pp. 401–431. Handbook of Unmanned Aerial Vehicles Springer (2015)

  8. Chowdhary, G., Johnson, E.N., Magree, D., Wu, A., Shein, A.: Gps-denied indoor and outdoor monocular vision-aided navigation and control of unmanned aircraft. J. Field Rob. 30(3), 415–438 (2013)

    Article  Google Scholar 

  9. Dauer, J., Goormann, L., Torens, C.: Steps towards scalable and modularized flight software for unmanned aircraft systems International Journal of Advanced Robotic Systems 11(81) (2014)

  10. DJI: Inspire 1 User Manual, v2.0, Chapter: Vision Positioning System. dl.djicdn.com/downloads/inspire_1/en/Inspire_1_User_Manual_en_v2.0_1218.pdf (2015)

  11. Dusha, D., Mejias, L.: Error analysis and attitude observability of a monocular gps/visual odometry integrated navigation filter. Int. J. Robot. Res. 31(6), 714–737 (2012)

    Article  Google Scholar 

  12. Engel, J., Sturm, J., Cremers, D.: Scale-aware navigation of a low-cost quadrocopter with a monocular camera. Robot. Auton. Syst. 62(11), 1646–1656 (2014)

    Article  Google Scholar 

  13. European Aviation Safety Authority: Concept for operations of drones. www.easa.europa.eu/easa-and-you/civil-drones-rpas (2015)

  14. Fosbury, A.M., Crassidis, J.: Optimal trajectory determination for increased relative navigation observability of air vehicles. J. Guid. Control. Dyn. 31(4), 824–834 (2008)

    Article  Google Scholar 

  15. Fraundorfer, F., Scaramuzza, D.: Visual odometry – part II: matching, robustness, optimization, and applications. IEEE Robot. Autom. Mag. 19(2), 78–90 (2012)

    Article  Google Scholar 

  16. Johnson, E.N., Schrage, D.P.: The Georgia Tech Unmanned Aerial Research Vehicle: Gtmax. In: AIAA Guidance, Navigation and Control Conference (2003)

  17. Kendoul, F.: Survey of advances in guidance, navigation, and control of unmanned rotorcraft systems. J. Field Rob. 29(2), 315–378 (2012)

    Article  Google Scholar 

  18. Kendoul, F., Nonami, K., Fantoni, I., Lozano, R.: An adaptive vision-based autopilot for mini flying machines guidance, navigation and control. Auton. Robot. 27(3), 165–188 (2009)

    Article  Google Scholar 

  19. Klein, G., Murray, D.: Parallel Tracking and Mapping for Small Ar Workspaces. In: International Symposium on Mixed and Augmented Reality (2007)

  20. Krause, S., Evert, R.: Remission Based Improvement of Extrinsic Parameter Calibration of Camera and Laser Scanner. In: 12Th International Conference on Control, Automation, Robotics Andamp; Vision, pp. 829–834 (2012)

  21. Kwon, Y.H.: Direct linear transform method http://www.kwon3d.com/theory/dlt/dlt.html (1998)

  22. Lu, F., Hartley, R.: A Fast Optimal Algorithm for L 2 Triangulation. In: Asian Conf. on Computer Vision (2007)

  23. Lucas, B.D., Kanade, T.: An Iterative Image Registration Technique with an Application to Stereo Vision. In: International Joint Conference on Artificial Intelligence, Pp. 674–679 (1981)

  24. Martínez, C., Mondragón, I.F., Olivares-Méndez, M.A., Campoy, P.: On-board and ground visual pose estimation techniques for uav control. J. Intell. Robot. Syst. 61(1), 301–320 (2011)

    Article  Google Scholar 

  25. Mouragnon, E., Lhullier, M., Dhorne, M., Deyekser, F., Seyd, P.: Generic and real-time structure from motion using local bundle adjustment. Image Vis. Comput. 27, 1178–1193 (2009)

    Article  Google Scholar 

  26. Scaramuzza, D., Fraundorfer, F.: Visual odometry – part I: The first 30 years and fundamentals. IEEE Robot. Autom. Mag. 18(4), 80–92 (2011)

    Article  Google Scholar 

  27. Scaramuzza, D., et al.: Vision-controlled micro flying robots: from system design to autonomous navigation and mapping in GPS-denied environments. IEEE Robot. Autom. Mag. 21(3), 26–40 (2014)

    Article  Google Scholar 

  28. Schmid, K., Ruess, F., Suppa, M., Burschka, D.: State Estimation for Highly Dynamic Flying Systems Using Keyframe Odometry with Varying Time Delays. In: IEEE International Conference on Intelligent Robots and Systems, pp. 2997–3004 (2012)

  29. Schmid, K., Tomić, T., Ruess, F., Hirschmüller, H., Suppa, M.: Stereo Vision Based Indoor/Outdoor Navigation for Flying Robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3955–3962 (2013)

  30. Shen, S., Mulgaonkar, Y., Michael, N., Kumar, V.: Vision-Based State Estimation and Trajectory Control Towards High-Speed Flight with a Quadrotor. In: Robotics: Science and Systems (2013)

  31. Strobl, K.H., Hirzinger, G.: More accurate pinhole camera calibration with imperfect planar target.. In: IEEE International Conference on Computer Vision (ICCV 2011), 1st IEEE Workshop on Challenges and Opportunities in Robot Perception, pp. 1068–1075. www.robotic.dlr.de/callab (2011)

  32. Sünderhauf, N., Konolige, K., Lacroix, S., Protzel, P.: Visual Odometry Using Sparse Bundle Adjustment on an Autonomous Outdoor Vehicle. In: Tagungsband Autonome Mobile Systeme, pp. 157–163 (2005)

  33. The Law Library of Congress, Global Legal Research Center: Regulation of drones. www.loc.gov/law/help/regulation-of-drones (2016)

  34. Verveld, M.J., Chu, Q.P., de Wagter, C., Mulder, J.A. : Optic flow based state estimation for an indoor micro air vehicle. In: AIAA Guidance, Navigation, and Control Conference (2010)

  35. Wang, J., Garratt, M., Lambert, A., Wang, J.J., Han, S., Sinclair, D.: Integration of gps/ins/vision sensors to navigate unmanned aerial vehicles. The International Archives of the Photogrammetry Remote Sensing and Spatial Information Sciences 37(963-970) (2008)

  36. Warren, M., Upcroft, B.: High Altitude Stereo Visual Odometry. In: Robotics: Science and Systems (2013)

  37. Weiss, S., Scaramuzza, D., Siegwart, R.: Monocular-slam-based navigation for autonomous micro helicopters in gps-denied environments. J. Field Rob. 28(6), 854–874 (2011)

    Article  Google Scholar 

  38. Williams, W., Harris, M.: The Challenge of Flight-Testing Unmanned Air Vehicles. In: Systems Engineering, Test and Evaluation Conference (2002)

  39. Wu, A.D., Johnson, E.: Autonomous Flight in Gps-Denied Environments Using Monocular Vision and Inertial Sensors. In: AIAA Infotech@Aerospace Conference (2010)

  40. Zhang, J., Singh, S.: INS Assisted Monocular Visual Odometry for Aerial Vehicles. In: 9Th International Conference on Field and Service Robotics (FSR), Pp. 183–197 (2013)

  41. Zhang, J., Singh, S.: Visual-Lidar Odometry and Mapping: Low-Drift, Robust, and Fast. In: IEEE International Conference on Robotics and Automation, Pp. 2174–2181 (2015)

Download references

Author information

Authors and Affiliations

  1. German Aerospace Center (DLR), Institute of Flight Systems, Unmanned Aircraft, Braunschweig, Germany

    Franz Andert, Nikolaus Ammann, Stefan Krause & Sven Lorenz

  2. Australian Research Centre for Aerospace Automation, Queensland University of Technology, Brisbane, Australia

    Dmitry Bratanov & Luis Mejias

Authors
  1. Franz Andert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nikolaus Ammann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefan Krause
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sven Lorenz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dmitry Bratanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Luis Mejias
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Franz Andert.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Andert, F., Ammann, N., Krause, S. et al. Optical-Aided Aircraft Navigation using Decoupled Visual SLAM with Range Sensor Augmentation. J Intell Robot Syst 88, 547–565 (2017). https://doi.org/10.1007/s10846-016-0457-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-016-0457-6

Keywords

Search

Navigation