Skip to main content
SpringerLink
Log in

Visual feedback control of quadrotor by object detection in movies

  • Original Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

In the previous report, the authors developed a quadrotor mission planning function using a smartphone and an autopilot function using GPS signals. In this paper, a visual feedback control is considered so that the system can automatically recognize the existence of an object in images or movies acquired by a quadrotor with a surveillance camera and this enables the quadrotor to track the object within the operational environment without using GPS signals. The visual feedback control is realized by automatic object recognition based on techniques of computer vision, in which the center of gravity position in the image coordinate system can be obtained from color or shape information. A visual feedback control application for flight control of DJI’s quadrotor Matrice100, and image analysis software are developed using Xcode and MATLAB, respectively. The validity and effectiveness are evaluated through experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Hattori H (2008) Stereo vision technology for automotive applications. IEEE Intell Veh Sympos 63(5):48–51

    Google Scholar 

  2. Serizawa K, Ladig R, Shimonomura K (2019) Movable camera and stereoscopic image system for teleoperation of aerial robot working in high place. In: Proceedings of the 2019 JSME conference on robotics and mechatronics, 1P2-N08

  3. Otsuka H, Yajima R, Nagatani K, Kubo D (2015) Localization of a small multi-rotor UAV using dropping AR Markersh. In: Proceedings of the 2015 JSME conference on robotics and mechatronics, 2A2-G02

  4. Saito H, Takubo T, Ueno A, Cai K, Miyamoto R, Hara S (2018) Recognizing own UAV position and measuring position of a person using camera image. In: Proceedings of the 2018 JSME conference on robotics and mechatronics, 1P1-C08

  5. Lu Z, Nagata F, Watanabe K, Habib MK (2017) iOS application for quadrotor UAV remote control—implementation of basic functions with iPhone. Artif Life Robot 22(3):374–379

    Article  Google Scholar 

  6. Lu Z, Nagata F, Watanabe K (2018) Mission planning of iOS application for a quadrotor UAV. Artif Life Robot 23(3):428–433

    Article  Google Scholar 

  7. Gomez-Avila J, Lopez-Franco C, Alanis AY, Arana-Daniel N, Lopez-Franco M (2018) Ground vehicle tracking with a quadrotor using image based visual servoing. IFAC Papers Online, 2nd IFAC conference on modelling, identification and control of nonlinear systems MICNON 2018, vol 51, no 13, pp 344–349

  8. Yang L, Liu Z, Wang X (2019) A new image-based visual servo control algorithm for target tracking problem of fixed-wing unmanned aerial vehicle. 2019 Chinese control conference (CCC), pp 8142–8147

  9. Sheng H, Shi E, Zhang K (2019) Image-based visual servoing of a quadrotor with improved visibility using model predictive control. 2019 IEEE 28th international symposium on industrial electronics (ISIE), pp 551–556

  10. Cao Z, Chen X, Yu Y, Yu J, Liu X, Zhou C, Tan M (2020) Image dynamics-based visual servoing for quadrotors tracking a target with a nonlinear trajectory observer. IEEE Trans Syst Man Cybern Syst 50(1):376–384

    Article  Google Scholar 

  11. Shao L, Nagata F, Watanabe K (2019) Remote control application for a quadrotor supporting iOS and android. In: Proceedings of the 2019 JSME Conference on robotics and mechatronics, 1P1–N06

  12. Bay H, Ess A, Tuytelaars T, Gool LV (2008) SURF: speeded up robust features. Comput Vis Image Underst (CVIU) 110(3):346–359

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graduate School of Engineering, Sanyo-Onoda City University, 1-1-1 Daigaku-Dori, Sanyo-Onoda, 756-0884, Japan

    Lu Shao, Fusaomi Nagata, Hiroaki Ochi, Akimasa Otsuka & Takeshi Ikeda

  2. Okayama University, Okayama, Japan

    Keigo Watanabe

  3. American University in Cairo, Cairo, Egypt

    Maki K. Habib

Authors
  1. Lu Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fusaomi Nagata
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hiroaki Ochi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Akimasa Otsuka
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Takeshi Ikeda
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Keigo Watanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Maki K. Habib
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fusaomi Nagata.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was presented in part at the 25th International Symposium on Artificial Life and Robotics (Beppu, Oita, January 22–24, 2020).

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shao, L., Nagata, F., Ochi, H. et al. Visual feedback control of quadrotor by object detection in movies. Artif Life Robotics 25, 488–494 (2020). https://doi.org/10.1007/s10015-020-00609-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-020-00609-3

Keywords

Search

Navigation