Skip to main content
SpringerLink
Log in

Vision-based landing site evaluation and informed optimal trajectory generation toward autonomous rooftop landing

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Autonomous landing is an essential function for micro air vehicles (MAVs) for many scenarios. We pursue an active perception strategy that enables MAVs with limited onboard sensing and processing capabilities to concurrently assess feasible rooftop landing sites with a vision-based perception system while generating trajectories that balance continued landing site assessment and the requirement to provide visual monitoring of an interest point. The contributions of the work are twofold: (1) a perception system that employs a dense motion stereo approach that determines the 3D model of the captured scene without the need of geo-referenced images, scene geometry constraints, or external navigation aids; and (2) an online trajectory generation approach that balances the need to concurrently explore available rooftop vantages of an interest point while ensuring confidence in the landing site suitability by considering the impact of landing site uncertainty as assessed by the perception system. Simulation and experimental evaluation of the performance of the perception and trajectory generation methodologies are analyzed independently and jointly in order to establish the efficacy and robustness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. The vector \(e_3\) is a unit vector along the world z-axis and the hat operator ( \(\hat{}\) ) is defined by the relation \(\hat{a}b = a \times b\) for \(a, b \in \mathbb {R}^3\). The vehicle mass m and inertia J are assumed to be known.

References

  • Achtelik, M. W., Weiss, S., Lynen, S., Chli, M., & Siegwart, R. (2012). Vision-based MAV navigation: Implementation challenges towards a usable system in real-life scenarios. In Proceedings of robotics: science and systems, Sydney, Australia, workshop on integration of perception with control and navigation for resource-limited, highly dynamic, autonomous systems.

  • Bosch, S., Lacroix, S., & Caballero, F. (2006). Autonomous detection of safe landing areas for an UAV from monocular images. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 5522–5527).

  • Brockers, R., Susca, S., Zhu, D., & Matthies, L. (2012). Fully self-contained vision-aided navigation and landing of a micro air vehicle independent from external sensor inputs. Proceedings of SPIE (Vol. 8387, pp. 83870Q-1–83870Q-10).

  • Cheng, Y. (2010). Real-time surface slope estimation by homography alignment for spacecraft safe landing. In Proceedings of the IEEE international conference on robotics and automation (pp. 2280–2286).

  • Choi, H. L., & How, J. P. (2010). Continuous trajectory planning of mobile sensors for informative forecasting. Automatica, 46(8), 1266–1275.

    Article  MATH  MathSciNet  Google Scholar 

  • Desaraju, V. R., Michael, N., Humenberger, M., Brockers, R., Weiss, S., & Matthies, L. (2014). Vision-based landing site evaluation and trajectory generation toward rooftop landing. In Proceedings of robotics: science and systems. Berkeley, CA.

  • Di, K., & Li, R. (2004). CAHVOR camera model and its photogrammetric conversion for planetary applications. Journal of Geophysical Research, 109, E04004.

    Article  Google Scholar 

  • Eng, P. C. S. (2011). Path planning, guidance, and control for a UAV forced landing. PhD thesis, Queensland University of Technology, Queensland, Australia.

  • Engel, J., Sturm, J., & Cremers, D. (2013). Semi-dense visual odometry for a monocular camera. In 2013 IEEE international conference on computer vision (ICCV), IEEE (pp. 1449–1456).

  • Forster, C., Pizzoli, M., & Scaramuzza, D. (2014). SVO: Fast semi-direct monocular visual odometry. In Proceedings of IEEE international conference on robotics and automation.

  • Frew, E.W., & Rock, S.M. (2003). Trajectory generation for constant velocity target motion estimation using monocular vision. In Proceedings of IEEE international conference on robotics and automation (pp. 3479–3484). Taipei, Taiwan.

  • Hirschmueller, H., Innocent, P., & Garibaldi, J. (2002). Real-time correlation-based stereo vision with reduced border errors. International Journal of Computer Vision, 47, 229–246.

    Article  MATH  Google Scholar 

  • Johnson, A., Montgomery, J., & Matthies, L. (2005). Vision guided landing of an autonomous helicopter in hazardous terrain. In Proceedings of IEEE international conference on robotics and automation (pp. 3966–3971).

  • Klein, G., & Murray, D. (2007). Parallel tracking and mapping for small AR workspaces. In Proceedings of the international symposium on mixed and augmented reality. Nara, Japan.

  • Le Ny, J., & Pappas, G. J. (2009). On trajectory optimization for active sensing in gaussian process models. In Proceedings of the IEEE conference on decision and control. Shanghai, China.

  • Levine, D., Luders, B., & How, J. P. (2013). Information-theoretic motion planning for constrained sensor networks. Journal of Aerospace Information Systems, 10(10), 476–496.

    Article  Google Scholar 

  • Lizarraga, M. (2004). Autonomous landing system for a UAV. Master’s thesis, Naval Postgraduate School, Monterey, CA.

  • Lourenço, H. R., Martin, O., & Stützle, T. (2001). A beginner’s introduction to iterated local search. In Proceedings of the meta-heuristics international conference (pp. 1–6). Porto, Portugal.

  • Low, K. H., Chen, J., Dolan, J. M., Chien, S., & Thompson, D. R. (2012). Decentralized active robotic exploration and mapping for probabilistic field classification in environmental sensing. In International conference on autonomous agents and multiagent systems.

  • Luders, B., Sugel, I., & How, J. P. (2013). Robust trajectory planning for autonomous parafoils under wind uncertainty. In AIAA Infotech@Aerospace conference. Boston, MA.

  • Mahony, R., Kumar, V., & Corke, P. (2012). Multirotor aerial vehicles: Modeling, estimation, and control of quadrotor. IEEE Robotics and Automation Magazine, 19(3), 20–32.

    Article  Google Scholar 

  • Meingast, M., Geyer, C., & Sastry, S. (2004). Vision based terrain recovery for landing unmanned aerial vehicles. In Proceedings of the IEEE conference on decision and control (Vol. 2, p. 1670–1675).

  • Mejias, L., Fitzgerald, D. L., Eng, P. C., & Xi, L. (2009). Forced landing technologies for unmanned aerial vehicles: Towards safer operations. In L. Thanh Mung (Ed.), Aerial vehicles (pp. 415–442). Kirchengasse: In-Tech.

    Google Scholar 

  • Meliou, A., Krause, A., Guestrin, C., & Hellerstein, J. M. (2007). Nonmyopic informative path planning in spatio-temporal models. AAAI (pp. 602–607).

  • Montgomery, J., Johnson, A., Roumeliotis, S., & Matthies, L. (2006). The jet propulsion laboratory autonomous helicopter testbed: A platform for planetary exploration technology research and development. Journal of Field Robotics, 23(3–4), 245–267.

    Article  Google Scholar 

  • Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011). DTAM: Dense tracking and mapping in real-time. In 2011 IEEE international conference on computer vision (ICCV), IEEE (pp. 2320–2327).

  • Nguyen, J., Lawrance, N., Fitch, R., & Sukkarieh, S. (2013). Energy-constrained motion planning for information gathering with autonomous aerial soaring. In Proceedings of the IEEE international conference on robotics and automation (pp. 3825–3831). Karlsruhe, Germany.

  • Pizzoli, M., Forster. C., & Scaramuzza, D. (2014). Remode: Probabilistic, monocular dense reconstruction in real time. In Proceedings of the IEEE international conference on robotics and automation.

  • Ponda, S. S. (2008). Trajectory optimization for target localization using small unmanned aerial vehicles. Master’s thesis, MIT, Cambridge.

  • Prior, S. D., Shen, S., Erbil, M., Brazinskas, M., & Mielniczek, W. (2013). HALO the winning entry to the DARPA UAVForge challenge 2012. In Design, user experience, and usability (pp. 179–188). Berlin: Springer.

  • Rasmussen, C. E., & Ghahramani, Z. (2001). Infinite mixtures of Gaussian process experts. In Advances in neural information processing systems (pp. 881–888). Vancouver, Canada.

  • Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. Cambridge: MIT Press.

    MATH  Google Scholar 

  • Saripalli, S., & Sukhatme, G.S. (2007). Landing a helicopter on a moving target. In Proceedings of the IEEE international conference on robotics and automation (pp. 2030–2035).

  • Scherer, S., Chamberlain, L., & Singh, S. (2012). Autonomous landing at unprepared sites by a full-scale helicopter. Robotics and Autonomous Systems, 60(12), 1545–1562.

    Article  Google Scholar 

  • Seitz, S., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In 2006 IEEE computer society conference on computer vision and pattern recognition (pp. 519–528).

  • Shen, S., Mulgaonkar, Y., Michael, N., & Kumar, V. (2013). Vision-based state estimation and trajectory control towards high-speed flight with a quadrotor. In Proceedings of robotics: science and systems. Berlin, Germany.

  • Singh, S., & Padhi, R. (2009). Automatic path planning and control design for autonomous landing of UAVs using dynamic inversion. In Proceedings of the American control conference (pp. 2409–2414). St. Louis, MO.

  • Snelson, E., & Ghahramani, Z. (2005). Sparse Gaussian processes using pseudo-inputs. In Advances in neural information processing systems (pp. 1257–1264). Vancouver, Canada.

  • Stühmer, J., Gumhold, S., & Cremers, D. (2010). Real-time dense geometry from a handheld camera. In Proceedings of the 32nd DAGM conference on pattern recognition (pp. 11–20).

  • Templeton, T., Shim, D. H., Geyer, C., & Sastry, S. S. (2007). Autonomous vision-based landing and terrain mapping using an MPC-controlled unmanned rotorcraft. In Proceedings of the IEEE international conference on robotics and automation (pp. 1349–1356).

  • Theodore, C., Rowley, D., Hubbard, D., Ansar, A., Matthies, L., Goldberg, S., & Whalley, M. (2006). Flight trials of a rotorcraft unmanned aerial vehicle landing autonomously at unprepared sites. In Forum of the American Helicopter Society. Phoenix, AZ.

  • Tokekar, P., Vander Hook, J., Mulla, D., & Isler, V. (2013). Sensor planning for a symbiotic UAV and UGV system for precision agriculture. Tech. Rep. 13-010, University of Minnesota, Minneapolis, MN.

  • Weiss, S., Achtelik, M. W., Chli, M., & Siegwart, R. (2012). Versatile distributed pose estimation and sensor self-calibration for an autonomous MAV. In Proceedings of the IEEE international conference on robotics and automation (pp. 31–38).

  • Weiss, S., Achtelik, M. W., Lynen, S., Achtelik, M. C., Kneip, L., Chli, M., et al. (2013). Monocular vision for long-term micro aerial vehicle state estimation: A compendium. Journal of Field Robotics, 30(5), 803–831.

    Article  Google Scholar 

Download references

Acknowledgments

We gratefully acknowledge the support of ARL Grant W911NF-08-2-0004.

Author information

Authors and Affiliations

  1. The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Vishnu R. Desaraju & Nathan Michael

  2. Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA

    Roland Brockers, Stephan Weiss, Jeremy Nash & Larry Matthies

  3. AIT Austrian Institute of Technology, Vienna, Austria

    Martin Humenberger

Authors
  1. Vishnu R. Desaraju
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nathan Michael
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martin Humenberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roland Brockers
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stephan Weiss
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jeremy Nash
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Larry Matthies
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vishnu R. Desaraju.

Additional information

This is one of several papers published in Autonomous Robots comprising the “Special Issue on Robotics Science and Systems”.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Desaraju, V.R., Michael, N., Humenberger, M. et al. Vision-based landing site evaluation and informed optimal trajectory generation toward autonomous rooftop landing. Auton Robot 39, 445–463 (2015). https://doi.org/10.1007/s10514-015-9456-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-015-9456-x

Keywords

Search

Navigation