Skip to main content
SpringerLink
Log in

PIE: a Tool for Data-Driven Autonomous UAV Flight Testing

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

In this paper, a novel technique is presented to test the flight of an unmanned aerial vehicle autonomously in a real-world scenario using a data-driven technique without intervening with its onboard software. With the growing applications of such vehicles, testing of autonomous flight is a very important task for rapid deployment. There are different tools for modeling and simulating unmanned vehicles in virtual worlds such as Gazebo, MATLAB, Simulink, and Webots to name a few. None of these simulation tools are able to model all possible physical parameters of a real-world environment. Hence, the flight controller or mission planning software has to be tested in the physical world in the presence of an expert before deployment for a specific task. A Perception Inference Engine evaluation tool is presented that can infer internal states of the autonomous system from external observations only. The Gazebo simulation platform is used to collect data to develop the perception model. For real-time data collection, a VICON motion capture system is used to observe the autonomous flight of a small unmanned aerial vehicle. A state-of-the-art decision tree algorithm is used to implement the data-driven approach. The technique was tested using simulation data and verified with real-time data from Intel Aero Ready to Fly and Parrot AR. 2.0 drones. Moreover, we analyzed the robustness of the proposed system by introducing noise in sensor measurement and ambiguity in the testing scenario. We compared the performance of the decision tree classifier with Naïve bayes and support vector machine classifiers. It is shown that the developed system can be used for the performance evaluation of a UAV operating in the physical world by significantly reducing uncertainty in mission failure due to environmental parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arbanas, B., Ivanovic, A., Car, M., Haus, T., Orsag, M., Petrovic, T., Bogdan, S.: Aerial-ground robotic system for autonomous delivery tasks. In: 2016 IEEE international conference on robotics and automation (ICRA), pp. 5463–5468. IEEE (2016)

  2. Bachr-ach, A., Prentice, S., He, R., Roy, N.: Range–robust autonomous navigation in gps-denied environments. J. Field Rob. 28(5), 644–666 (2011)

    Article  Google Scholar 

  3. Ben-Gal, I., Dana, A., Shkolnik, N., Singer, G.: Efficient construction of decision trees by the dual information distance method. Quality Technology & Quantitative Management 11(1), 133–147 (2014)

    Article  Google Scholar 

  4. Benić, Z., Piljek, P., Kotarski, D.: Mathematical modelling of unmanned aerial vehicles with four rotors. Interdisciplinary Description of Complex Systems: INDECS 14(1), 88–100 (2016)

    Article  Google Scholar 

  5. Berrahal, S., Kim, J.H., Rekhis, S., Boudriga, N., Wilkins, D., Acevedo, J.: Border surveillance monitoring using quadcopter uav-aided wireless sensor networks (2016)

  6. Bouabdallah, S., Noth, A., Siegwart, R.: Pid vs lq control techniques applied to an indoor micro quadrotor. In: Proceedings of the IEEE international conference on intelligent robots and systems (IROS), pp. 2451–2456. IEEE (2004)

  7. Brandmaier, A.M., von Oertzen, T., McArdle, J.J., Lindenberger, U.: Structural equation model trees. Psychol. Methods 18(1), 71 (2013)

    Article  Google Scholar 

  8. Breiman, L.: Classification and regression trees. Routledge (2017)

  9. Brescianini, D., D’Andrea, R.: Design, modeling and control of an omni-directional aerial vehicle. In: 2016 IEEE international conference on robotics and automation (ICRA), , pp. 3261–3266. IEEE (2016)

  10. Cabecinhas, D., Naldi, R., Marconi, L., Silvestre, C., Cunha, R.: Robust take-off and landing for a quadrotor vehicle. In: 2010 IEEE international conference on robotics and automation (ICRA), pp. 1630–1635. IEEE (2010)

  11. Cai, G., Chen, B.M., Peng, K., Dong, M., Lee, T.H.: Modeling and control of the yaw channel of a uav helicopter. IEEE Trans. Ind. Electron. 55(9), 3426–3434 (2008)

    Article  Google Scholar 

  12. Chaudhary, S., Prava, A., Nidhi, N., Nath, V.: Design of all-terrain rover quadcopter for military engineering services. In: Nanoelectronics, Circuits and Communication Systems, pp. 507–513. Springer (2019)

  13. Chen, Y., Pérez-Arancibia, N.O.: Generation and real-time implementation of high-speed controlled maneuvers using an autonomous 19-gram quadrotor. In: 2016 IEEE international conference on robotics and automation (ICRA), pp. 3204–3211. IEEE (2016)

  14. Chowdhury, D., Sarkar, M., Haider, M., Fattah, S., Shahnaz, C.: Design and implementation of a cyber-vigilance system for anti-terrorist drives based on an unmanned aerial vehicular networking signal jammer for specific territorial security. In: Humanitarian Technology Conference (R10-HTC), 2017 IEEE Region 10, pp. 444–448. IEEE (2017)

  15. Colomina, I., Molina, P.: Unmanned aerial systems for photogrammetry and remote sensing: A review. ISPRS J. Photogramm. Remote. Sens. 92, 79–97 (2014)

    Article  Google Scholar 

  16. Cook, Z., Zhao, L., Lee, J., Yim, W.: Unmanned aerial system for first responders. In: 2015 12th international conference on ubiquitous robots and ambient intelligence (URAI), pp. 306–310. IEEE (2015)

  17. Damerau, F.J., Johnson, D.E., Buskirk, M.C. Jr: Automatic labeling of unlabeled text data. US Patent 6,697,998 (2004)

  18. De Marina, H.G., Kapitanyuk, Y.A., Bronz, M., Hattenberger, G., Cao, M.: Guidance algorithm for smooth trajectory tracking of a fixed wing uav flying in wind flows. In: 2017 IEEE international conference on robotics and automation (ICRA), pp. 5740–5745. IEEE (2017)

  19. Deng, H., Runger, G., Tuv, E.: Bias of importance measures for multi-valued attributes and solutions. In: International conference on artificial neural networks, pp. 293–300. Springer (2011)

  20. Duggal, V., Sukhwani, M., Bipin, K., Reddy, G.S., Krishna, K.M.: Plantation monitoring and yield estimation using autonomous quadcopter for precision agriculture. In: 2016 IEEE international conference on robotics and automation (ICRA), pp. 5121–5127. IEEE (2016)

  21. Elhennawy, A.M., Habib, M.K.: Trajectory tracking of a quadcopter flying vehicle using sliding mode control. In: IECON 2017-43rd annual conference of the IEEE industrial electronics society, pp. 6264–6269. IEEE (2017)

  22. Fattah, S.A., Haider, M.Z., Chowdhury, D., Sarkar, M., Chowdhury, R.I., Islam, M.S., Karim, R., Rahi, A., Shahnaz, C.: An aerial landmine detection system with dynamic path and explosion mode identification features. In: 2016 IEEE global humanitarian technology conference (GHTC), pp. 745–752. IEEE (2016)

  23. Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2-3), 131–163 (1997)

    Article  Google Scholar 

  24. Fu, C., Sarabakha, A., Kayacan, E., Wagner, C., John, R., Garibaldi, J.M.: A comparative study on the control of quadcopter uavs by using singleton and non-singleton fuzzy logic controllers. In: 2016 IEEE international conference on fuzzy systems (FUZZ-IEEE), pp. 1023–1030. IEEE (2016)

  25. Gautam, D., Ha, C.: Control of a quadrotor using a smart self-tuning fuzzy pid controller. Int. J. Adv. Robot. Syst. 10(11), 380 (2013)

    Article  Google Scholar 

  26. Gawel, A., Kamel, M., Tonci, N., Widauer, J., et al.: Aerial picking and delivery of magnetic objects with mavs. In: 2017 IEEE international conference on robotics and automation (ICRA), pp. 5746–5752. IEEE (2017)

  27. He, Z., Zhao, L.: A simple attitude control of quadrotor helicopter based on ziegler-nichols rules for tuning pd parameters. The Scientific World Journal 2014 (2014)

  28. Hothorn, T., Hornik, K., Zeileis, A.: Unbiased recursive partitioning: A conditional inference framework. J. Comput. Graph. Stat. 15(3), 651–674 (2006)

    Article  MathSciNet  Google Scholar 

  29. How, J.P., Behihke, B., Frank, A., Dale, D., Vian, J.: Real-time indoor autonomous vehicle test environment. IEEE Control. Syst. 28(2), 51–64 (2008)

    Article  MathSciNet  Google Scholar 

  30. Huang, H., Hoffmann, G.M., Waslander, S.L., Tomlin, C.J.: Aerodynamics and control of autonomous quadrotor helicopters in aggressive maneuvering. In: IEEE international conference on robotics and automation, 2009. ICRA’09. pp. 3277–3282. IEEE (2009)

  31. James, G., Witten, D., Hastie, T., Tibshirani, R.: An introduction to statistical learning, vol. 112. Springer (2013)

  32. Khan, F., Ellenberg, A., Mazzotti, M., Kontsos, A., Moon, F., Pradhan, A., Bartoli, I.: Investigation on bridge assessment using unmanned aerial systems. In: Structures congress 2015, pp. 404–413 (2015)

  33. Koenig, N., Howard, A.: Design and use paradigms for gazebo, an open-source multi-robot simulator. In: 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS)(IEEE Cat. No. 04CH37566), vol. 3, pp. 2149–2154. IEEE (2004)

  34. Kohavi, R., et al.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Ijcai, vol. 14, pp 1137–1145. Montreal, Canada (1995)

  35. Kotsiantis, S.B., Zaharakis, I., Pintelas, P.: Supervised machine learning: A review of classification techniques. Emerging artificial intelligence applications in computer engineering 160, 3–24 (2007)

    Google Scholar 

  36. Kuantama, E., Tarca, I., Tarca, R.: Quadcopter modeling in virtual reality for dynamic visualization. In: 2018 5th international conference on control, decision and information technologies (CoDIT), pp. 671–676. IEEE (2018)

  37. Kuantama, E., Vesselenyi, T., Dzitac, S., Tarca, R.: Pid and fuzzy-pid control model for quadcopter attitude with disturbance parameter. International Journal of Computers Communications & Control 12(4), 519–532 (2017)

    Article  Google Scholar 

  38. Kuo, C., Kuo, C., Leber, A., Boller, C.: Vector thrust multi-rotor copter and its application for building inspection. In: International micro air vehicle conference and flight competition, Toulouse, France, Sept, pp. 17–20 (2013)

  39. Laurent, H., Rivest, R.L.: Constructing optimal binary decision trees is np-complete. Inf. Process. Lett. 5(1), 15–17 (1976)

    Article  MathSciNet  Google Scholar 

  40. Lee, J.Y., Song, S.H., Shon, H.W., Choi, H.R., Yim, W.: Modeling and control of a saucer type coandä effect uav. In: 2017 IEEE international conference on robotics and automation (ICRA), pp. 2717–2722. IEEE (2017)

  41. Ling, Y., Liu, T., Shen, S.: Aggressive quadrotor flight using dense visual-inertial fusion. In: 2016 IEEE international conference on robotics and automation (ICRA), pp. 1499–1506. IEEE (2016)

  42. Liu, L., Michael, N.: An mdp-based approximation method for goal constrained multi-mav planning under action uncertainty. In: 2016 IEEE international conference on robotics and automation (ICRA), pp. 56–62. IEEE (2016)

  43. Liu, S., Watterson, M., Tang, S., Kumar, V.: High speed navigation for quadrotors with limited onboard sensing. In: 2016 IEEE international conference on robotics and automation (ICRA), pp. 1484–1491. IEEE (2016)

  44. Lottes, P., Khanna, R., Pfeifer, J., Siegwart, R., Stachniss, C.: Uav-based crop and weed classification for smart farming. In: 2017 IEEE international conference on robotics and automation (ICRA), pp. 3024–3031. IEEE (2017)

  45. Luukkonen, T.: Modelling and control of quadcopter. Independent research project in applied mathematics. Espoo 22 (2011)

  46. Lyu, X., Gu, H., Wang, Y., Li, Z., Shen, S., Zhang, F.: Design and implementation of a quadrotor tail-sitter vtol uav. In: 2017 IEEE international conference on robotics and automation (ICRA), pp. 3924–3930. IEEE (2017)

  47. McArthur, D.R., Chowdhury, A.B., Cappelleri, D.J.: Design of the i-boomcopter uav for environmental interaction. In: 2017 IEEE international conference on robotics and automation (ICRA), pp. 5209–5214. IEEE (2017)

  48. Mendoza-Soto, J.L., Cortés, H.R.: Generalized predictive control for trajectory tracking of a quadcopter vehicle. In: 2017 international conference on unmanned aircraft systems (ICUAS), pp. 206–212. IEEE (2017)

  49. Merwaday, A., Guvenc, I.: Uav assisted heterogeneous networks for public safety communications. In: 2015 IEEE wireless communications and networking conference workshops (WCNCW), pp. 329–334. IEEE (2015)

  50. Modares, J., Ghanei, F., Mastronarde, N., Dantu, K.: Ub-anc planner: Energy efficient coverage path planning with multiple drones. In: 2017 IEEE international conference on robotics and automation (ICRA), pp. 6182–6189. IEEE (2017)

  51. Mullins, G.E., Stankiewicz, P.G., Gupta, S.K.: Automated generation of diverse and challenging scenarios for test and evaluation of autonomous vehicles. In: 2017 IEEE international conference on robotics and automation (ICRA), pp. 1443–1450. IEEE (2017)

  52. Murphy, K.P.: Machine learning: a probabilistic perspective. MIT press (2012)

  53. Myers, F.: The future of dod test and evaluation resources. Tech. rep. office of the under secretary of defense washington DC test resource (2009)

  54. Painsky, A., Rosset, S.: Cross-validated variable selection in tree-based methods improves predictive performance. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2142–2153 (2017)

    Article  Google Scholar 

  55. Paredes, J.A., Acevedo, J., Mogrovejo, H., Villalta, J., Furukawa, R.: Quadcopter design for medicine transportation in the peruvian amazon rainforest. In: 2016 IEEE XXIII international congress on electronics, electrical engineering and computing (INTERCON), pp. 1–6. IEEE (2016)

  56. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al.: Scikit-learn: Machine learning in python. J. Mach. Learn. Res. 12(Oct), 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  57. Praveen, V., Pillai, S.A.: Modeling and simulation of quadcopter using pid controller. International Journal of Control Theory and Applications 9(15), 7151–7158 (2016)

    Google Scholar 

  58. Quigley, M., Conley, K., Gerkey, B.P., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA workshop on open source software (2009)

  59. Quinlan, J.R.: Induction of decision trees. Machine learning 1(1), 81–106 (1986)

    Google Scholar 

  60. Quinlan, J.R.: C4. 5: programs for machine learning. Elsevier (2014)

  61. Raffo, G.V., Ortega, M.G., Rubio, F.R.: An integral predictive/nonlinear h\(\infty \) control structure for a quadrotor helicopter. Automatica 46(1), 29–39 (2010)

    Article  MathSciNet  Google Scholar 

  62. Renzaglia, A., Reymann, C., Lacroix, S.: Monitoring the evolution of clouds with uavs. In: 2016 IEEE international conference on robotics and automation (ICRA), pp. 278–283. IEEE (2016)

  63. Rohmer, E., Singh, S.P., Freese, M.: V-rep: A versatile and scalable robot simulation framework. In: 2013 IEEE/RSJ international conference on intelligent robots and systems, pp. 1321–1326. IEEE (2013)

  64. Sa, I., Corke, P.: Vertical infrastructure inspection using a quadcopter and shared autonomy control. In: Field and service robotics, pp. 219–232. Springer (2014)

  65. Adamchik, S.V.: Binary Trees https://www.cs.cmu.edu/∼adamchik/15-121/lectures/Trees/trees.html (2009)

  66. Sameh, H., Aiman, K., Dustegor, D.: A remotely piloted aerial system for a faster processing of traffic collisions towards reducing the resulting road congestion. In: 2017 9th IEEE-GCC conference and exhibition (GCCCE), pp. 1–9. IEEE (2017)

  67. Sammut, C., Webb, G.I.: Encyclopedia of machine learning. Springer Science & Business Media (2011)

  68. Steinberg, D., Colla, P.: Cart: classification and regression trees. The top ten algorithms in data mining 9, 179 (2009)

    Article  Google Scholar 

  69. Tan, P.N., et al.: Introduction to data mining. Pearson Education India (2007)

  70. Tengis, T., Batmunkh, A.: State feedback control simulation of quadcopter model. In: 2016 11th international forum on strategic technology (IFOST), pp. 553–557. IEEE (2016)

  71. Tomandl, D., Schober, A.: A modified general regression neural network (mgrnn) with new, efficient training algorithms as a robust ’black box’-tool for data analysis. Neural Netw. 14(8), 1023–1034 (2001)

    Article  Google Scholar 

  72. Yang, J., et al.: Automatically labeling video data using multi-class active learning. In: Proceedings Ninth IEEE international conference on computer vision, pp. 516–523. IEEE (2003)

  73. Zemalache, K.M., Beji, L., Marref, H.: Control of an under-actuated system: application a four rotors rotorcraft. In: 2005 IEEE international conference on robotics and biomimetics (ROBIO), pp. 404–409. IEEE (2005)

Download references

Acknowledgements

This paper is based on research sponsored by Air Force Research Laboratory and Office of the Secretary of Defense (OSD) under agreement number FA8750-15-2-0116. The authors would like to thank Air Force Research Laboratory and OSD.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, North Carolina A&T State University, Greensboro, NC, 27411, USA

    Mrinmoy Sarkar, Abdollah Homaifar, Berat A. Erol & Mohammadreza Behniapoor

  2. United Technologies Research Center, East Hartford, CT, 06108, USA

    Edward Tunstel

Authors
  1. Mrinmoy Sarkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdollah Homaifar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Berat A. Erol
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohammadreza Behniapoor
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Edward Tunstel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdollah Homaifar.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sarkar, M., Homaifar, A., Erol, B.A. et al. PIE: a Tool for Data-Driven Autonomous UAV Flight Testing. J Intell Robot Syst 98, 421–438 (2020). https://doi.org/10.1007/s10846-019-01078-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-019-01078-y

Keywords

Search

Navigation