Skip to main content
SpringerLink
Log in

Appearance-Based Visual-Teach-And-Repeat Navigation Technique for Micro Aerial Vehicle

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The objective of this paper is to develop a vision-based navigation technique for micro aerial vehicles, quadrotor type, to operate in GPS-denied environment. The navigation method has been developed while using appearance-based Visual-Teach-and-Repeat (VT&R) technique. In a teaching phase, a quadrotor is manually navigated along a desired route to collect a set of reference images. In a repeating phase, the quadrotor is able to autonomously follow the desired route using these reference images. Self-localization is developed to determine the current segment of the desired route by a number of Speeded-Up Robust Features (SURF), matched between the current image and the reference images. In this paper, three methods of self-localization are proposed and compared. After performing self-localization, the quadrotor computes appearance-based motion control commands (desired yaw and height) for the next movement in order to keep track of the desired route. This computation is developed on Funnel Lane theory, which was originally proposed in Chen and Birchfield (IEEE Trans. Robot. 25(3), 749–754 (11)) for 2D navigation of a ground vehicle. The paper extends this theory to 3D navigation of a quadrotor.The proposed self-localization methods are tested with several image databases. Finally, an online experiment of proposed VT&R technique is demonstrated using Ar.Drone quadrotor model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Parrot AR.Drone ver 2.0 quadrotor, http://ardrone2.parrot.com

  2. Robot Operating System (ROS), http://www.ros.org

  3. Bachrach, A., Prentice, S.: RANGE: Robust autonomous navigation in GPS-denied environments. Journal of Field Robotics 28(5), 644–666 (2011)

    Article  Google Scholar 

  4. Barfoot, T.D., Stenning, B., Furgale, P., McManus, C.: Exploiting Reusable Paths in Mobile Robotics: Benefits and Challenges for Long-term Autonomy. In: Proceedings of Conference on Computer and Robot Vision, pp. 388–395 (2012)

  5. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-Up Robust Features (SURF). Comp. Vision Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  6. Becerral, H.M., Courbon, J., Mezouar, Y., Sagues, C.: Wheeled mobile robots navigation from a visual memory using wide field of view cameras. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5693–5699 (2010)

  7. Bradski, G., Kaehler, A.: Learning OpenCV: Computer vision with the openCV library. O’Reilly Media (2008)

  8. Calonder, M., Lepetit, V., Ozuysal, M., Trzcinski, T., Strecha, C., Fua, P.: BRIEF: Computing a Local Binary Descriptor very Fast. IEEE Transactions on Pattern Analysis and Machine Intelligence (2011)

  9. Fischler, M.A., Bolles, R.C.: Random Sample Consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  10. Chen, Z., Birchfield, S.T.: Qualitative vision-based mobile robot navigation. In: Proceedings of IEEE International Conference on Robotics and Automation, May, pp. 2686–2692 (2006)

  11. Chen, Z., Birchfield, S.T.: Qualitative vision-based path following. IEEE Trans. Robot. 25(3), 749–754 (2009)

    Article  Google Scholar 

  12. Churchill, D., Vardy, A.: An orientation invariant visual homing algorithm. J. Intell. Robot. Syst. (2013)

  13. Courbon, J., Mezouar, Y.: Visual navigation of a quadrotor aerial vehicle. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5315–5320 (2009)

  14. Courbon, J., Mezouar, Y., Martinet, P.: Visual navigation of a quadrotor Aerial Vehicle. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5315–5320 (2009)

  15. Dame, A., Marchand, E.: A new information theoretic approach for appearance-based navigation of non-holonomic vehicle. In: Proceedings of IEEE International Conference on Robotics and Automation, pp 2459–2464 (2011)

  16. De Silva, O., Mann, G.K.I., Gosine, R.G.: Development of a relative localization scheme for ground-aerial multi-robot systems. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 870–875 (2012)

  17. Eckert, J., German, R., Dressler, F.: An indoor localization framework for four-rotor flying robots using low-power sensor nodes. IEEE Trans. Instrum. Meas. 60(2), 336–344 (2011)

    Article  Google Scholar 

  18. Eckert, J., German, R., Dressler, F.: Self-Organized localization in indoor environments using the ALF framework. International Journal of Communication Networks and Distributed Systems 10(2), 102–122 (2013)

    Article  Google Scholar 

  19. Erhard, S., Wenzel, K.E., Zell, A.: Flyphone: Visual Self-Localisation Using a Mobile Phone as Onboard Image Processor on a Quadrocopter. J. Intell. Robot. Syst. 57(1–4), 451–465 (2009)

    Google Scholar 

  20. Forster, C., Pizzoli, M., Scaramuzza, D.: Air-ground localization and map augmentation using monocular dense reconstruction. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3971–3978 (2013)

  21. Fraundorfer, F., Heng, L.: Vision-based autonomous mapping and exploration using a quadrotor MAV. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4557–4564 (2012)

  22. Fu, C., Olivares-Mendez, M.A., Suarez-Fernandez, R., Campoy, P.: Monocular Visual-Inertial SLAM-Based Collision Avoidance Strategy for Fail-Safe UAV Using Fuzzy Logic Controllers. J. Intell. Robot. Syst. 73(1–4), 513–533 (2013)

    Google Scholar 

  23. Furgale, P., Barfoot, T.: Stereo mapping and localization for long-range path following on rough terrain. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 4410–4416 (2010)

  24. Furgale, P., Barfoot, T.D.: Visual teach and repeat for long range rover autonomy. Journal of Field Robotics 27(2006), 534–560 (2010)

    Article  Google Scholar 

  25. Hochdorfer, S., Schlegel, C.: 6 DoF SLAM using a ToF camera: The challenge of a continuously growing number of landmarks. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3981–3986 (2010)

  26. Krajnik, T., De Cristóforis, P., Faigl, J., Szücsová, H., Nitsche, M., Preucil, L., Mejail, M.: Image features for long-term mobile robot autonomy. IEEE Conference on Robotics and Automation, Workshop on Long-Term Autonomy (2013)

  27. Kushleyev, A., Mellinger, D., Powers, C., Kumar, V.: Towards a swarm of agile micro quadrotors. Auton. Robot. 35(4), 287–300 (2013)

    Article  Google Scholar 

  28. Lee, G.H., Achtelik, M., Fraundorfer, F., Pollefeys, M., Siegwart, R.: A benchmarking tool for MAV visual pose estimation. In: Proceedings of 11th International Conference on Control Automation Robotics & Vision, vol. 1, pp. 1541–1546 (2010)

  29. Liu, W., Zheng, N., Xue, J., Zhang, X., Yu, Z.: Visual appearance-based unmanned vehicle sequential localization. Int. J. Adv. Robot. Syst. 10 (2013)

  30. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

  31. Lupashin, S., Hehn, M., Mueller, M.W., Schoellig, A.P., Sherback, M., DAndrea, R.: A platform for aerial robotics research and demonstration: The Flying Machine Arena. Mechatronics 24(1), 41–54 (2014)

    Article  Google Scholar 

  32. Majdik, A.L., Albers-Schoenberg, Y., Scaramuzza, D.: MAV urban localization from Google street view data. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3979–3986 (2013)

  33. McManus, C., Furgale, P., Stenning, B., Barfoot, T.D.: Visual Teach and Repeat using appearance-based lidar. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 389–396 (2012)

  34. Mellinger, D., Kumar, V.: Minimum snap trajectory generation and control for quadrotors.. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 2520–2525 (2011)

  35. Michael, N., Mellinger, D., Lindsey, Q., Kumar, V.: The GRASP Multiple Micro-UAV Testbed. IEEE Robotics Automation Magazine 17, 56–65 (2010)

    Article  Google Scholar 

  36. Michael, N., Shen, S., Mohta, K., Mulgaonkar, Y., Kumar, V., Nagatani, K., Okada, Y., Kiribayashi, S., Otake, K., Yoshida, K., Ohno, K., Takeuchi, E., Tadokoro, S.: Collaborative mapping of an earthquake-damaged building via ground and aerial robots. Journal of Field Robotics 29 (5), 832–841 (2012)

    Article  Google Scholar 

  37. Mori, T., Scherer, S.: First results in detecting and avoiding frontal obstacles from a monocular camera for micro unmanned aerial vehicles. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 1750–1757 (2013)

  38. Munoz, L.E., Castillo, P., Garcia, P.: Observer-control scheme for autonomous navigation: Flight tests validation in a quadrotor vehicle. In: Proceedings of International Conference on Unmanned Aircraft Systems, pp 795–804 (2013)

  39. Murillo, A.: Surf features for efficient robot localization with omnidirectional images. In: Proceedings of IEEE International Conference on Robotics and Automation, April, pp. 10–14 (2007)

  40. Nguyen, T., Mann, G.K., Gosine, R.G.: Vision-based qualitative path-following control of quadrotor aerial vehicle. In: Proceedings of International Conference on Unmanned Aircraft Systems (2014)

  41. Nguyen, T., Mann, G.K., Gosine, R.G.: Vision-Based Qualitative Path-Following Control of Quadrotor Aerial Vehicle with Speeded-Up Robust Features. In: Proceedings of Canadian Conference on Computer and Robot Vision, pp. 321–327 (2014)

  42. Ostafew, C.J., Schoellig, A.P., Barfoot, T.D.: Visual teach and repeat, repeat, repeat: Iterative Learning Control to improve mobile robot path tracking in challenging outdoor environments. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 176–181 (2013)

  43. Pfrunder, A., Schoellig, A.P., Barfoot, T.D.: A Proof-Of-Concept Demonstration of Visual Teach and Repeat on a Quadrocopter Using an Altitude Sensor and a Monocular Camera. In: Proceedings of Canadian Conference on Computer and Robot Vision (2014)

  44. Pronobis, A., Caputo, B.: COLD: The CoSy Localization Database. Int. J. Robot. Res. 28(5), 588–594 (2009)

    Article  Google Scholar 

  45. Sebastian Thrun Wolfram Burgard, D.F.: Probabilistic Robotics. MIT Press (2005)

  46. Shen, S., Michael, N., Kumar, V.: Autonomous multi-floor indoor navigation with a computationally constrained MAV. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 20–25 (2011)

  47. Smith, M., Baldwin, I., Churchill, W., Paul, R., Newman, P.: The New College Vision and Laser Data Set. Int. J. Robot. Res. 28(5), 595–599 (2009)

    Article  Google Scholar 

  48. Vaca-Castano, G., Zamir, A.R., Shah, M.: City scale geo-spatial trajectory estimation of a moving camera. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1186–1193 (2012)

  49. Valgren, C., Lilienthal, A.J.: SIFT, SURF & seasons: Appearance-based long-term localization in outdoor environments. Robot. Auton. Syst. 58(2), 149–156 (2010)

    Article  Google Scholar 

  50. Vandrish, P., Vardy, A., King, P.: Towards AUV Route Following Using Qualitative Navigation. In: Proceedings of IEEE Canadian Conference on Computer and Robot Vision, pp. 425–432 (2012)

  51. Vardy, A.: Using feature scale change for robot localization along a route. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4830–4835 (2010)

  52. Zhang, A.M., Kleeman, L.: Robust Appearance Based Visual Route Following for Navigation in Large-scale Outdoor Environments. Int. J. Robot. Res. 28(3), 331–356 (2009)

    Article  Google Scholar 

  53. Zhou, Q., Zhang, Y.: Dead reckoning and Kalman filter design for trajectory tracking of a quadrotor UAV. In: Proceedings of IEEE/ASME International Conference on Mechatronics and Embedded Systems, pp. 119–124 (2010)

Download references

Author information

Authors and Affiliations

  1. Intelligent Systems Lab, Faculty of Engineering and Applied Science, Memorial University of Newfoundland, A1B 3X5, St. John’s, NL, Canada

    Trung Nguyen, George K. I. Mann, Raymond G. Gosine & Andrew Vardy

Authors
  1. Trung Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George K. I. Mann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Raymond G. Gosine
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrew Vardy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Trung Nguyen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, T., Mann, G.K.I., Gosine, R.G. et al. Appearance-Based Visual-Teach-And-Repeat Navigation Technique for Micro Aerial Vehicle. J Intell Robot Syst 84, 217–240 (2016). https://doi.org/10.1007/s10846-015-0320-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-015-0320-1

Keywords

Search

Navigation