Skip to main content
SpringerLink
Log in
Book cover

Robotics Research pp 235–252Cite as

Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera

  • Chapter
  • First Online:

Part of the book series: Springer Tracts in Advanced Robotics ((STAR,volume 100))

Abstract

RGB-D cameras provide both a color image and per-pixel depth estimates. The richness of their data and the recent development of low-cost sensors have combined to present an attractive opportunity for mobile robotics research. In this paper, we describe a system for visual odometry and mapping using an RGB-D camera, and its application to autonomous flight. By leveraging results from recent state-of-the-art algorithms and hardware, our system enables 3D flight in cluttered environments using only onboard sensor data. All computation and sensing required for local position control are performed onboard the vehicle, reducing the dependence on unreliable wireless links. We evaluate the effectiveness of our system for stabilizing and controlling a quadrotor micro air vehicle, demonstrate its use for constructing detailed 3D maps of an indoor environment, and discuss its limitations.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. M. Achtelik, A. Bachrach, R. He, S. Prentice, N. Roy, Stereo vision and laser odometry for autonomous helicopters in GPS-denied indoor environments, in Proceedings of the SPIE Unmanned Systems Technology XI, vol. 7332, Orlando, F, 2009

    Google Scholar 

  2. S. Ahrens, D. Levine, G. Andrews, J.P. How, Vision-based guidance and control of a hovering vehicle in unknown, GPS-denied environments, in IEEE International Conference on Robotics and Automation, 2009, pp. 2643–2648

    Google Scholar 

  3. A. Bachrach, R. He, N. Roy, Autonomous flight in unknown indoor environments. Int. J. Micro Air Veh. 1(4), 217–228 (2009)

    Google Scholar 

  4. S. Benhimane, E. Malis. Improving vision-based control using efficient second-order minimization techniques, in IEEE International Conference on Robotics and Automation, 2004

    Google Scholar 

  5. M. Blösch, S. Weiss, D. Scaramuzza, R. Siegwart, Vision based MAV navigation in unknown and unstructured environments, in IEEE International Conference on Robotics and Automation, 2010, pp. 21–28

    Google Scholar 

  6. G. Buskey, J. Roberts, P. Corke, G. Wyeth. Helicopter automation using a low-cost sensing system. Comput. Control Eng. J. 15(2), 8–9 (2004)

    Google Scholar 

  7. M. Calonder, V. Lepetit, and P. Fua. Keypoint signatures for fast learning and recognition, in European Conference on Computer Vision (2008), pp. 58–71

    Google Scholar 

  8. K. Celik, Soon J. Chung, and A. Somani. Mono-vision corner SLAM for indoor navigation, in IEEE International Conference on Electro/Information Technology (2008), pp. 343–348

    Google Scholar 

  9. G. Grisetti, S. Grzonka, C. Stachniss, P. Pfaff, and W. Burgard. Estimation of accurate maxi- mum likelihood maps in 3D, in IEEE International Conference on Intelligent Robots and Systems (2007)

    Google Scholar 

  10. G. Grisetti, C. Stachniss, S. Grzonka, and W. Burgard. A tree parameterization for efficiently computing maximum likelihood maps using gradient descent, in Proceedings of Robotics: Science and Systems (2007)

    Google Scholar 

  11. S. Grzonka, G. Grisetti, W. Burgard, Towards a navigation system for autonomous indoor flying, in IEEE International Conference on Robotics and Automation (Kobe, Japan, 2009)

    Google Scholar 

  12. C. Harris, M. Stephens, A combined corner and edge detector, in Alvey Vision Conference (1988), pp. 147–151

    Google Scholar 

  13. R. He, S. Prentice, N. Roy, Planning in information space for a quadrotor helicopter in a GPS-denied environment, in IEEE International Conference on Robotics and Automation (Los Angeles, CA, 2008), pp. 1814–1820

    Google Scholar 

  14. P. Henry, M. Krainin, E. Herbst, X. Ren, D. Fox, RGB-D mapping: using depth cameras for dense 3d modeling of indoor environments, in International Symposium on Experimental Robotics (2010)

    Google Scholar 

  15. H. Hirschmuller, P.R. Innocent, J.M. Garibaldi, Fast, unconstrained camera motion estimation from stereo without tracking and robust statistics, in Proceedings of International Conference on Control, Automation, Robotics and Vision, vol. 2, 2002, pp. 1099–1104

    Google Scholar 

  16. B.K.P. Horn, Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. 4(4), 629–642 (1987)

    Article  Google Scholar 

  17. A. Howard, Real-time stereo visual odometry for autonomous ground vehicles, in IEEE International Conference on Intelligent Robots and Systems (2008)

    Google Scholar 

  18. A.E. Johnson, S.B. Goldberg, Y. Cheng, L.H. Matthies, Robust and efficient stereo feature tracking for visual odometry, in IEEE International Conference on Robotics and Automation (Pasadena, CA, 2008)

    Google Scholar 

  19. M. Kaess, A. Ranganathan, F. Dellaert, iSAM: incremental smoothing and mapping. IEEE Trans. Robot. (TRO) 24(6), 1365–1378 (2008)

    Google Scholar 

  20. J. Kelly, G.S. Sukhatme, An experimental study of aerial stereo visual odometry, in Proceedings of Symposium on Intelligent Autonomous Vehicles (Toulouse, France, 2007)

    Google Scholar 

  21. K. Konolige, Sparse bundle adjustment, in Proceedings of the British Machine Vision Conference (BMVC) (2010)

    Google Scholar 

  22. K. Konolige, M. Agrawal, J. Sola, Large-scale visual odometry for rough terrain, in International Symposium on Robotics Research (Hiroshima, Japan, 2007)

    Google Scholar 

  23. C. Mei, G. Sibley, M. Cummins, P. Newman, I. Reid, A constant time efficient stereo SLAM system, in British Machine Vision Conference (2009)

    Google Scholar 

  24. L. Meier, P. Tanskanen, F. Fraundorfer, M. Pollefeys, Pixhawk: a system for autonomous flight using onboard computer vision, in IEEE International Conference on Robotics and Automation (2011)

    Google Scholar 

  25. H. Moravec, Obstacle avoidance and navigation in the real world by a seeing robot rover. PhD thesis, Stanford University (1980)

    Google Scholar 

  26. P. Newman, G. Sibley, M. Smith, M. Cummins, A. Harrison, C. Mei, I. Posner, R. Shade, D. Schröter, L. Murphy, W. Churchill, D. Cole, I. Reid, Navigating, recognising and describing urban spaces with vision and laser. Int. J. Robot. Res. 28(11–12) (2009)

    Google Scholar 

  27. D. Nistér, Preemptive RANSAC for live structure and motion estimation. Mach. Vis. Appl. 16, 321–329 (2005)

    Google Scholar 

  28. D. Nistér, O. Naroditsky, J. Bergen, Visual odometry, in Computer Vision and Pattern Recognition (Washington, D.C., 2004), pp. 652–659

    Google Scholar 

  29. D. Nistér, H. Stewenius, Scalable recognition with a vocabulary tree, in Computer Vision and Pattern Recognition (2006)

    Google Scholar 

  30. E. Olson, J. Leonard, S. Teller, Fast iterative optimization of pose graphs with poor initial estimates, in IEEE International Conference on Robotics and Automation (2006), pp. 2262–2269

    Google Scholar 

  31. M. Pollefeys, D. Nister, J.-M. Frahm, A. Akbarzadeh, P. Mordohai, B. Clipp, C. Engels, D. Gallup, S.-J. Kim, P. Merrell, C. Salmi, S. Sinha, B. Talton, L. Wang, Q. Yang, H. Stewenius, R. Yang, G. Welch, H. Towles, Detailed real-time urban 3D reconstruction from video. Int. J. Comput. Vis. 72(2), 143–167 (2008)

    Google Scholar 

  32. PrimeSense. http://www.primesense.com

  33. E. Rosten, T. Drummond, Machine learning for high-speed corner detection, in European Conference on Computer Vision (2006)

    Google Scholar 

  34. S. Shen, N. Michael, V. Kumar, Autonomous multi-floor indoor navigation with a computationally constrained MAV, in IEEE International Conference on Robotics and Automation (Shanghai, China, 2011)

    Google Scholar 

  35. N. Snavely, S. Seitz, R. Szeliski, Photo tourism: exploring photo collections in 3D, in ACM Transactions on Graphics (Proceedings of SIGGRAPH) (2006)

    Google Scholar 

  36. B. Steder, G. Grisetti, C. Stachniss, W. Burgard, Visual SLAM for flying vehicles. IEEE Trans. Robot. 24(5), 1088–1093 (2008)

    Google Scholar 

  37. B. Triggs, P. McLauchlan, R. Hartley, A. Fitzgibbon, Bundle adjustment—a modern synthesis. Vis. Algor.: Theory Pract. 153–177 (2000)

    Google Scholar 

Download references

Acknowledgements

This research was supported by the Office of Naval Research under MURI N00014-07-1-0749, Science of Autonomy program N00014-09-1-0641 and the Army Research Office under the MAST CTA. D.M. acknowledges travel support from P. Universidad Cato´lica’s School of Engineering. P.H. and D.F. are supported by ONR MURI grant number N00014-09-1- 1052, and by the NSF under contract number IIS-0812671, as well as collaborative participation in the Robotics Consortium sponsored by the U.S Army Research Laboratory under Agreement W911NF-10-2-0016.

Author information

Authors and Affiliations

  1. Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA

    Albert S. Huang, Abraham Bachrach & Nicholas Roy

  2. Department of Computer Science and Engineering, University of Washington, Seattle, WA, USA

    Peter Henry, Michael Krainin & Dieter Fox

  3. Department of Computer Science, Pontificia Universidad Católica de Chile, Santiago, Chile

    Daniel Maturana

Authors
  1. Albert S. Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abraham Bachrach
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Henry
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Krainin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daniel Maturana
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dieter Fox
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nicholas Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Albert S. Huang .

Editor information

Editors and Affiliations

  1. Robotics and Intelligent Machines, Georgia Institute of Technology College of Computing, Atlanta, Georgia, USA

    Henrik I. Christensen

  2. Department of Computer Science, Stanford University, Stanford, California, USA

    Oussama Khatib

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Huang, A.S. et al. (2017). Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera. In: Christensen, H., Khatib, O. (eds) Robotics Research . Springer Tracts in Advanced Robotics, vol 100. Springer, Cham. https://doi.org/10.1007/978-3-319-29363-9_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-29363-9_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-29362-2

  • Online ISBN: 978-3-319-29363-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation