Abstract
The popularity of Micro Aerial Vehicles (MAV) to be used in civilian applications has increased in the last years. However, in most of these applications, a MAV is used to acquire aerial images and video of areas and structures of interest. However, MAVs could become more useful if they can interact with the environment. For instance, in a parcel delivery task, the goal is for the MAV to deliver a package somewhere, but what about having to pick up a package autonomously? This task raises some challenges: i) the MAV has to recognize where the package or object of interest is; ii) the MAV has to plan its maneuver to achieve the picking. In this paper, we address both challenges, considering the scenario where the MAV has a suspended cable that moves freely with a hook attached at the end of the cable. A suspended cable saves weight, although it has to be indirectly controllable with the MAV’s flight. Thus, we present a solution based on a Convolutional Neural Network that is trained to recognize the object of interest, in this case, a bucket; and that simultaneously recognizes the hook. Both objects are expected to be observed with a camera on board the MAV. Our method uses the distance between these two objects in a state machine controller to position the MAV and trigger the lifting maneuver in a single upward motion action that reduces the effects of air current on the hook. We use synthetic datasets to train the bucket and hook detector, but the model is capable of performing the detection in real environments. We achieved an average lifting success rate of 70% for indoor and 60% for outdoor scenarios.
Similar content being viewed by others
Code Availability
The code can be reviewed in the following link: https://github.com/MLG-repos/mav-object-lifting
References
de Angelis, E.L., Giulietti, F., Pipeleers, G.: Two-time-scale control of a multirotor aircraft for suspended load transportation. Aerosp. Sci. Technol. 84, 193–203 (2019)
Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., Vedaldi, A.: Describing Textures in the Wild. arXiv:1311.3618 [cs] (2013)
Cruz, P.J., Fierro, R.: Cable-suspended load lifting by a quadrotor UAV: hybrid model, trajectory generation, and control. Auton. Robot. 41(8), 1629–1643 (2017)
Faust, A., Palunko, I., Cruz, P., Fierro, R., Tapia, L.: Automated aerial suspended cargo delivery through reinforcement learning. AI 247, 381–398 (2017)
Foehn, P., Falanga, D., Kuppuswamy, N., Tedrake, R., Scaramuzza, D.: Fast trajectory optimization for agile Quadrotor maneuvers with a cable-suspended payload. In: Robotics: science and systems XIII. robotics: science and systems foundation (2017)
IMAV: IMAV 2016 International Micro Air Vehicle Competition Rules v2.3. http://www.imavs.org/2016/competition.html (2016)
Lopez Garcia, M., Martinez Carranza, J.: A First CNN-based approach towards autonomous flight for object lifting. Comput. Sist. 24, 3 (2020)
Loquercio, A., Kaufmann, E., Ranftl, R., Dosovitskiy, A., Koltun, V., Scaramuzza, D.: Deep drone racing: From simulation to reality with domain randomization. IEEE Trans. Robot. 36(1), 1–14 (2019)
Ramon Soria, P., Arrue, B., Ollero, A.: Detection, location and grasping objects using a stereo sensor on UAV in outdoor environments. Sensors 17(12), 103 (2017)
Rojas-Perez, L.O., Martinez-Carranza, J.: Deeppilot: A cnn for autonomous drone racing. Sensors 20(16), 4524 (2020)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: Inverted Residuals and Linear Bottlenecks. arXiv:1801.04381 [cs] (2019)
Spurny, V., Baca, T., Saska, M., Penicka, R., Krajnik, T., Thomas, J., Thakur, D., Loianno, G., Kumar, V.: Cooperative autonomous search, grasping, and delivering in a treasure hunt scenario by a team of unmanned aerial vehicles. J Field Robot. 36(1), 125–148 (2019)
Suarez, A., Soria, P.R., Heredia, G., Arrue, B.C., Ollero, A.: Anthropomorphic, compliant and lightweight dual arm system for aerial manipulation. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, Vancouver, BC, pp 992–997 (2017)
Thomas, J., Loianno, G., Polin, J., Sreenath, K., Kumar, V.: Toward autonomous avian-inspired grasping for micro aerial vehicles. Bioinspir. Biomim. 9(2), 025,010 (2014)
Villa, D.K.D., Brandão, A.S., Sarcinelli-Filho, M.: A Survey on load transportation using multirotor UAVs. J Intell. Robot. Syst. 98(2), 267–296 (2020)
Zhang, G., He, Y., Dai, B., Gu, F., Yang, L., Han, J., Liu, G., Qi, J.: Grasp a Moving Target from the Air: System & Control of an Aerial Manipulator. In: 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, Brisbane, QLD, pp 1681–1687 (2018)
Zhou, X., Wang, D., Krahenbuhl, P.: Objects as Points. arXiv:1904.07850 [cs] (2019)
Funding
The first author is thankful to Consejo Nacional de Ciencia y Tecnología (CONACYT) for his scholarship with no. 719068.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by all authors. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflicts of interest/Competing interests
The authors have no conflicts of interest to declare that are relevant to the content of this article.
All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.
The authors have no financial or proprietary interests in any material discussed in this article.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lopez, M., Martinez-Carranza, J. A CNN-based Approach for Cable-Suspended Load Lifting with an Autonomous MAV. J Intell Robot Syst 105, 32 (2022). https://doi.org/10.1007/s10846-022-01637-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-022-01637-w