Skip to main content
SpringerLink
Log in

Onboard plane-wise 3D mapping using super-pixels and stereo vision for autonomous flight of a hexacopter

  • Original Research
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

Micro Aerial Vehicles (MAVs) have become very popular with a wide range of applications in cinema, surveillance, search and rescue, among many others. For these applications, an experienced pilot could operate the vehicle while keeping it in the line-of-sight. However, if the MAV flies beyond, the MAV may be highly compromised leading to potential crashes due to undetected obstacles. Hence, an assistance system to aid with obstacle detection and avoidance is highly desirable. This problem becomes more evident for autonomous navigation of MAVs. Nevertheless, autonomous navigation involves intensive computational power that rests on additional hardware to that of the flight controller. In addition, MAVs have an inherent limitation in their payload, restricting the size and weight of the onboard hardware and consequently the computational processing and storage capabilities. Motivated by the limitations in hardware, we propose a method to build an high-level map with enough details to make it useful for autonomous navigation, detecting surfaces that can be modeled as planes, thus enabling real-time operation while running on low budget hardware. Our proposed method carries out the following stages: (a) super-pixel segmentation, (b) planes extraction, (c) real-time mapping and (d) RRT navigation. We tested the proposed method in a custom-made hexacopter capable of carrying a stereoscopic camera and a light computer. We show that our MAV navigates in an unknown environment and successfully senses and avoids potential obstacles in its path.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35

Similar content being viewed by others

References

  1. Dalamagkidis K (2015) Classification of UAVs. Springer, Dordrecht, pp 83–91

    Google Scholar 

  2. Dawadee A, Chahl J, Nandagopal DN (2016) An algorithm for autonomous aerial navigation using landmarks. J Aerosp Eng 29:45–72

  3. Vetrella AR, Fasano G (2016) Cooperative UAV navigation under nominal GPS coverage and in GPS-challenging environments. In: 2016 IEEE 2nd international forum on research and technologies for society and industry leveraging a better tomorrow (RTSI), IEEE, pp 1–5

  4. Skulstad R, Syversen CL, Merz M, Sokolova N, Fossen TI Johansen TA (2015) Net recovery of UAV with single-frequency RTK GPS. In: 2015 IEEE aerospace conference, IEEE, pp 1–10

  5. Aditya A (2012) Implementation of a 4d fast slam including volumetric sum of the uav. In: Proceedings of the international conference on sensing technology, ICST, pp 78–83

  6. Wu J, Fei W, Li Q (2013) An integrated measure and location method based on airborne 2d laser scanning sensor for UAV’s power line inspection. In: Proceedings-2013 5th conference on measuring technology and mechatronics automation, ICMTMA 2013, pp 213–217

  7. Li R, Liu J, Zhang L, Hang Y (2014) LIDAR/MEMS IMU integrated navigation (SLAM) method for a small UAV in indoor environments. In: 2014 DGON inertial sensors and systems (ISS), IEEE, pp 1–15

  8. Wang F, Wang K, Lai S, Phang SK, Chen BM, Lee TH (2014) An efficient UAV navigation solution for confined but partially known indoor environments. In: IEEE International conference on control and automation, ICCA, pp 1351–1356

  9. Braga JRG, Velho HFDC, Shiguemori EH (2015) Estimation of UAV position using LiDAR images for autonomous navigation over the ocean. In: 2015 9th international conference on sensing technology (ICST), IEEE, vol 2016-March, pp 811–816

  10. Opromolla R, Fasano G, Rufino G, Grassi M, Savvaris A (2016) LIDAR-inertial integration for UAV localization and mapping in complex environments. In: 2016 International conference on unmanned aircraft systems, ICUAS 2016, pp 649–656

  11. Masuko K, Takahashi I, Ogawa S, Wu M-H, Oosedo A, Matsumoto T, Go K, Sugai F, Konno A, Uchiyama M (2010) Autonomous takeoff and landing of an unmanned aerial vehicle. 2010 IEEE/SICE International symposium on system integration, pp 248–253

  12. Yao Z, Bin X, Qiang Y, Yang L, Fu W (2012) Autonomous control system for the quadrotor unmanned aerial vehicle. In: Proceedings of the 31st Chinese control conference, pp 4862–4867

  13. Gageik N, Strohmeier M, Montenegro S (2013) An autonomous UAV with an optical flow sensor for positioning and navigation. Int J Adv Robot Syst 10:341

    Article  Google Scholar 

  14. More V, Kumar H, Kaingade S, Gaidhani P, Gupta N (2015) Visual odometry using optic flow for Unmanned Aerial Vehicles. In: 2015 international conference on cognitive computing and information processing(CCIP), IEEE, pp 1–6

  15. Jesus F, Ventura R (2012) Combining monocular and stereo vision in 6D-SLAM for the localization of a tracked wheel robot. In: 2012 IEEE International symposium on safety, security, and rescue robotics (SSRR), IEEE, pp 1–6

  16. Mattar E, Al-Mutib K, Alsulaiman M, Ramdane H, Emaduddin M (2014) A survey : Intelligent based mobile robots stereo vision maps synthesis and learning methodologies. In: 5th International conference on intelligent systems, modelling and simulation, pp 94–98

  17. Moratuwage D, Wang D, Rao A, Senarathne N, Wang H (2014) Rfs collaborative multivehicle slam: slam in dynamic high-clutter environments. IEEE Robot Autom Mag 21(2):53–59

    Article  Google Scholar 

  18. Stasse O, Davison A, Sellaouti R, Yokoi K (2006) Real-time 3D SLAM for humanoid robot considering pattern generator information. In: 2006 IEEE/RSJ international conference on intelligent robots and systems, IEEE, pp 348–355

  19. Ahn S, Yoon S, Hyung S, Kwak N, Roh KS (2012) On-board odometry estimation for 3D vision-based SLAM of humanoid robot. In: 2012 IEEE/RSJ International conference on intelligent robots and systems, IEEE, pp 4006–4012

  20. Meng L, de Silva CW, Zhang J (2014) 3D visual SLAM for an assistive robot in indoor environments using RGB-D cameras. In: 2014 9th International conference on computer science & education, IEEE, pp 32–37

  21. Fallon MF, Folkesson J, McClelland H, Leonard JJ (2013) Relocating underwater features autonomously using sonar-based slam. IEEE J Ocean Eng 38:500–513

    Article  Google Scholar 

  22. Hidalgo F, Braunl T (2015) Review of underwater SLAM techniques. In: 2015 6th International conference on automation, robotics and applications (ICARA), IEEE, pp 306–311

  23. Thamrin NM, Arshad NHM, Adnan R, Sam R, Razak NA, Misnan MF, Mahmud SF (2012) Simultaneous localization and mapping based real-time inter-row tree tracking technique for unmanned aerial vehicle. In: 2012 IEEE international conference on control system, computing and engineering, IEEE, pp 322–327

  24. Bryson M, Sukkarieh S (2008) Observability analysis and active control for airborne SLAM. IEEE Trans Aerosp Electron Syst 44:261–280

    Article  Google Scholar 

  25. Sanchez-Rodriguez J-P, Aceves-Lopez A (2018) A survey on stereo vision-based autonomous navigation for multi-rotor muavs. Robotica 36(8):12251243

    Article  Google Scholar 

  26. Schauwecker K, Zell A (2014) On-board dual-stereo-vision for the navigation of an autonomous mav. J Intell Robot Syst 74(1–2):1–16

    Article  Google Scholar 

  27. Schmid K, Lutz P, Tomić T, Mair E, Hirschmüller H (2014) Autonomous vision-based micro air vehicle for indoor and outdoor navigation. J Field Robot 31:537–570

    Article  Google Scholar 

  28. Cook Z, Zhao Lin, Lee J, Yim W (2015) Unmanned aerial system for first responders. In: 2015 12th international conference on ubiquitous robots and ambient intelligence (URAI), IEEE, pp 306–310

  29. Kim B, Um TT, Suh C, Park FC (2016) Tangent bundle RRT: a randomized algorithm for constrained motion planning. Robotica 34:202–225

    Article  Google Scholar 

  30. Truong XT, Yoong VN, Ngo TD (2015) RGB-D and laser data fusion-based human detection and tracking for socially aware robot navigation framework. In: 2015 IEEE international conference on robotics and biomimetics, IEEE-ROBIO

  31. Klemm S, Oberlander J, Hermann A, Roennau A, Schamm T, Zollner J, Dillmann R (2015) RRT*-connect: faster, asymptotically optimal motion planning. In: 2015 IEEE International conference on robotics and biomimetics (Robio), (Zhuhai, Peoples R China), IEEE, pp 1670–1677

  32. Chi W, Meng M (2017) Risk-RRT*: A robot motion planning algorithm for the human robot coexisting environment. In: 2017 18th International conference on advanced robotics (ICAR), (Hong Kong), IEEE, pp 583–588

  33. Yoon S, Lee D, Jung J, Shim DH (2018) Spline-based RRT*using piecewise continuous collision-checking algorithm for car-like vehicles. J Intell Robot Syst Theory Appl

  34. Zhou K, Yu L, Long Z, Mo S (2017) Local path planning of driverless car navigation based on jump point search method under urban environment. Future Internet 9(3):51

    Article  Google Scholar 

  35. Moses E, Anitha G (2017) Goal directed approach to autonomous motion planning for unmanned vehicles. Defence Sci J 67:45–49

    Article  Google Scholar 

  36. Jayasree K, Jayasree P, Vivek A (2017) Dynamic target tracking using a four wheeled mobile robot with optimal path planning technique. In: Proceedings of 2017 IEEE International conference on circuit ,power and computing technologies (ICCPCT), (Kollam, India), Baselios Mathews II Coll Engn; IEEE

  37. Alves Neto A, Macharet D, Chaimowicz L, Campos M (2013) Path planning with multiple rapidly-exploring random trees for teams of robots. In: 2013 16th International conference on advanced robotics, ICAR

  38. Esrafilian O, Taghirad HD (2017) Autonomous flight and obstacle avoidance of a quadrotor by monocular SLAM. In: 4th RSI International conference on robotics and mechatronics, ICRoM

  39. Tomic T, Schmid K, Lutz P, Domel A, Kassecker M, Mair E, Grixa I, Ruess F, Suppa M, Burschka D (2012) Toward a fully autonomous uav: Research platform for indoor and outdoor urban search and rescue. IEEE Robot Autom Mag 19(3):46–56

    Article  Google Scholar 

  40. Nex F, Remondino F (2014) UAV for 3D mapping applications: a review. Appl Geom 6(1):1–15

    Article  Google Scholar 

  41. Remondino F, Barazzetti L, Nex F, Scaioni M, Sarazzi D (2012) UAV Photogrammetry for mapping and 3D modeling current status and future perspectives. ISPRS-International archives of the photogrammetry, remote sensing and spatial information sciences

  42. Newcombe RA, Lovegrove SJ, Davison AJ (2011) DTAM: Dense tracking and mapping in real-time. In: Proceedings of the IEEE international conference on computer vision

  43. Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: An efficient alternative to SIFT or SURF. Proceedings of the IEEE international conference on computer vision, pp 2564–2571

  44. Bodis-Szomoru A, Riemenschneider H, Van Gool L (jun 2015) Superpixel meshes for fast edge-preserving surface reconstruction. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), IEEE, vol 07-12, pp 2011–2020

  45. Martinez CC, Carranza JM, Mayol-Cuevas W, Estrada MO (2017) Enhancing 3D mapping via real-time superpixel-based segmentation. In: Adjunct proceedings of the 2016 IEEE international symposium on mixed and augmented reality, ISMAR-Adjunct, pp 90–95

  46. Tong H, Liu S, Liu N, Barnes N (2010) A novel object-oriented stereo matching on multi-scale superpixels for low-resolution depth mapping. In: 2010 Annual international conference of the IEEE engineering in medicine and biology society, EMBC’10, pp 5046–5049

  47. Mičušík B, Košecká J (2010) Multi-view superpixel stereo in urban environments. Int J Comput Vis 89(1):106–119

    Article  Google Scholar 

  48. Olufs S, Vincze M (may 2011) Robust single view room structure segmentation in Manhattan-like environments from stereo vision. In: 2011 IEEE International conference on robotics and automation, IEEE, pp 5315–5322

  49. Teixeira L, Chli M (2017) Real-time local 3D reconstruction for aerial inspection using superpixel expansion. Proceedings-IEEE International conference on robotics and automation, pp 4560–4567

  50. Ren CY, Prisacariu VA, Reid ID (2015) gSLICr: SLIC superpixels at over 250Hz. arXiv:1509.04232, pp 1–6

  51. Sanchez Rodriguez JP (2018) Navegacion autonoma de un dron hexacoptero con vision estereoscopica para mapeo y evasion de obstaculos en tiempo real. RITEC

  52. Sanchez-Rodriguez J-P (2018) Drone-navigation-super-pixels (github)

  53. Mur-Artal JMM, Montiel Raul DTJ (2015) ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans Robot 31(5):1147–1163

    Article  Google Scholar 

  54. Poppinga J (2010) Towards autonomous navigation for robots with 3D sensors. PhD thesis, Jacobs University

Download references

Author information

Authors and Affiliations

  1. ITESM-CEM, Av Lago de Guadalupe KM 3.5, Margarita Maza de Juárez, 52926, Cd López Mateos, Méx., México

    Jose-Pablo Sanchez-Rodriguez, Alejandro Aceves-Lopez & Gabriel Flores-Wysocka

  2. INAOE, Luis Enrique Erro 1, Sta María Tonanzintla, 72840, Puebla, Pue., México

    Jose Martinez-Carranza

Authors
  1. Jose-Pablo Sanchez-Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alejandro Aceves-Lopez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jose Martinez-Carranza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gabriel Flores-Wysocka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jose-Pablo Sanchez-Rodriguez.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sanchez-Rodriguez, JP., Aceves-Lopez, A., Martinez-Carranza, J. et al. Onboard plane-wise 3D mapping using super-pixels and stereo vision for autonomous flight of a hexacopter. Intel Serv Robotics 13, 273–287 (2020). https://doi.org/10.1007/s11370-020-00312-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-020-00312-z

Keywords

Search

Navigation