Skip to main content

AeroBotSim: A High-Photo-Fidelity Simulator for Heterogeneous Aerial Systems Under Physical Interaction

  • Conference paper
  • First Online:
Cognitive Systems and Information Processing (ICCSIP 2022)

Abstract

In the field of aerial manipulation, heterogeneous aerial systems with complex configuration become more and more popular, as they can overcome problems of traditional aircrafts in aerial manipulation. However, recent photo-realistic aircraft simulators do not support the accurate contact and collision behavior simulation of aircraft or the dynamics simulation of heterogeneous aerial systems. Besides, high-fidelity images are required in many machine learning-based perception and action algorithms. Therefore, we develop a simulator to provide a solution to aerial manipulation robots training, algorithm tests, and display: AeroBotSim. By using modular design, we decouple rendering engine and physics engine to obtain high-frequency states data while retrieving high-fidelity images. Also, we synchronize the contact information in rendering engine and the physics engine, and design interfaces to custom contact behavior for operation, separation, and recombination simulation. In this paper, we present our framework design and dynamic models of aircrafts under physical interaction. Then, we validate our simulator framework in three aspects: baseline controller in ROS, vision-based algorithm, and contact simulation respectively.

Supported by the National Natural Science Foundation of China under Grant 62173037, National Key R. D. Program of China, and State Key Laboratory of Robotics and Systems (HIT).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://youtu.be/7Ju_dlErOJU.

  2. 2.

    https://youtu.be/JhFNrjos9v8.

References

  1. Pan, S.J., Yang,Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010). https://doi.org/10.1109/TKDE.2009.191

  2. Bagnell, J.A.D.: An Invitation to Imitation. Tech. rep., Carnegie Mellon University (Mar 2015)

    Google Scholar 

  3. Kober, J., Peters, J.: Reinforcement learning in robotics: a survey. In: Wiering, M., van Otterlo, M. (eds.) Reinforcement Learning: State-of-the-Art, pp. 579–610. Springer, Berlin Heidelberg, Berlin, Heidelberg (2012)

    Chapter  Google Scholar 

  4. Hwangbo, J., Lee, J., Hutter, M.: Per-contact iteration method for solving contact dynamics. IEEE Robot. Autom. Lett. 3(2), 895–902 (2018). https://doi.org/10.1109/LRA.2018.2792536

  5. Guerra, W., Tal, E., Murali, V., Ryou, G., Karaman, S.: FlightGoggles: photo-realistic sensor simulation for perception-driven robotics using photogrammetry and virtual reality. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6941–6948 (2019)

    Google Scholar 

  6. Foster, C., Pizzoli, M., Scaramuzza, D.: SVO: Fast semi-direct monocular visual odometry. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 15–22 (2014)

    Google Scholar 

  7. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: Orb-slam: a versatile and accurate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015). https://doi.org/10.1109/TRO.2015.2463671

  8. Zhang, W., Ott, L., Tognon, M., Siegwart, R.: Learning variable impedance control for aerial sliding on uneven heterogeneous surfaces by proprioceptive and tactile sensing. arXiv e-prints p. arXiv:2206.14122 (2022)

  9. Ruggiero, F., Lippiello, V., Ollero, A.: Aerial manipulation: a literature review. IEEE Robot. Autom. Lett. 3(3), 1957–1964 (2018). https://doi.org/10.1109/LRA.2018.2808541

  10. Unreal Engine: https://www.unrealengine.com/. Last accessed 26 Aug 2022

  11. ROS – Robot Operating System: https://www.ros.org. Last accessed 26 Aug 2022

  12. Koenig, N., Howard, A.: Design and use paradigms for gazebo, an open-source multi-robot simulator. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566), vol. 3, pp. 2149–2154 (2004)

    Google Scholar 

  13. Juliani, A., et al.: Unity: A general platform for intelligent agents. arXiv e-prints p. arXiv:1809.02627 (2018)

  14. Kohlbrecher, S., Meyer, J., Graber, T., Petersen, K., Klingauf, U., von Stryk, O.: Hector open source modules for autonomous mapping and navigation with rescue robots, pp. 624–631. Springer, Berlin Heidelberg, Berlin, Heidelberg (2014). roboCup 2013: Robot World Cup XVII

    Google Scholar 

  15. Furrer, F., Burri, M., Achtelik, M., Siegwart, R.: RotorS—a modular Gazebo MAV simulator framework. In: Koubaa, A. (ed.) Robot Operating System (ROS): The Complete Reference (Volume 1), pp. 595–625. Springer International Publishing, Cham (2016)

    Chapter  Google Scholar 

  16. Shah, S., Dey, D., Lovett, C., Kapoor, A.: AirSim: High-fidelity visual and physical simulation for autonomous vehicles. In: Hutter, M., Siegwart, R. (eds.) Field and Service Robotics, pp. 621–635. Springer International Publishing, Cham (2018)

    Chapter  Google Scholar 

  17. PhysX|GeForce: https://www.nvidia.cn/geforce/technologies/physx/. Last accessed 26 Aug 2022

  18. Song, Y., Naji, S., Kaufmann, E., Loquercio, A., Scaramuzza, D.: Flightmare: A flexible quadrotor simulator (2021)

    Google Scholar 

  19. Dai, X., Ke, C., Quan, Q., Cai, K.Y.: RFlySim: Automatic test platform for UAV autopilot systems with FPGA-based hardware-in-the-loop simulations. Aerosp. Sci. Technol. 114, 106727 (2021). https://doi.org/10.1016/j.ast.2021.106727

  20. Unreal Engine 5 – Unreal Engine. https://www.unrealengine.com/unreal-engine-5. Last accessed 28 Aug 2022

  21. Brandt, J., Deters, R., Ananda, G., Selig, M.: UIUC propeller database, University of Illinois at Urbana-Champaign. http://m-selig.ae.illinois.edu/props/propDB.html (2015)

  22. Nguyen, H.N., Park, S., Park, J., Lee, D.: A novel robotic platform for aerial manipulation using quadrotors as rotating thrust generators. IEEE Trans. Robot. 34(2), 353–369 (2018). https://doi.org/10.1109/TRO.2018.2791604

    Article  Google Scholar 

  23. Zilles, C., Salisbury, J.: A constraint-based god-object method for haptic display. In: Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots, vol. 3, pp. 146–151 (1995). https://doi.org/10.1109/IROS.1995.525876

  24. Qin, T., Li, P., Shen, S.: Vins-mono: a robust and versatile monocular visual-inertial state estimator. IEEE Trans. Robot. 34(4), 1004–1020 (2018). https://doi.org/10.1109/TRO.2018.2853729

    Article  Google Scholar 

  25. Hogan, N.: Impedance control: an approach to manipulation. In: 1984 American Control Conference, pp. 304–313 (1984). https://doi.org/10.23919/ACC.1984.4788393

  26. Hwangbo, J., Lee, J., Hutter, M.: Per-contact iteration method for solving contact dynamics IEEE Robotics and Automation Letters 3(2), 895–902 (2018), www.raisim.com

  27. Chaos Physics Overview|Unreal Engine Documentation. https://docs.unrealengine.com/5.0/zh-CN/physics-in-unreal-engine/. Last accessed 26 Aug 2022

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yushu Yu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Du, J., Fan, Y., Wang, K., Feng, Y., Yu, Y. (2023). AeroBotSim: A High-Photo-Fidelity Simulator for Heterogeneous Aerial Systems Under Physical Interaction. In: Sun, F., Cangelosi, A., Zhang, J., Yu, Y., Liu, H., Fang, B. (eds) Cognitive Systems and Information Processing. ICCSIP 2022. Communications in Computer and Information Science, vol 1787. Springer, Singapore. https://doi.org/10.1007/978-981-99-0617-8_19

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-0617-8_19

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-0616-1

  • Online ISBN: 978-981-99-0617-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics