Skip to main content
SpringerLink
Log in

Performance Analysis of Camera Rotation Estimation Algorithms in Multi-Sensor Fusion for Unmanned Aircraft Attitude Estimation

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

In this paper the possibility to use the camera of the onboard sense and avoid system as an aid in unmanned aircraft attitude estimation is examined. At first, the error analysis of four image processing algorithms targeting the reconstruction of camera orientation change is done. Camera spatial resolution, sampling frequency (temporal resolution), pixelization errors and image noises are all considered in the error analysis. After performing the error analysis, the camera rotation estimates from the two best algorithms are integrated into an IMU (inertial measurement unit)-GPS attitude estimator to obtain a Camera-IMU-GPS estimator. The attitude estimation results of the two algorithms are compared to each other considering hardware-in-the-loop simulated ’flight data’ and simulated camera images. After experiencing estimation error decrease with the augmented Camera-IMU-GPS algorithm a final proof of concept was conducted based-on real flight data. Off-line calculated IMU-GPS and Camera-IMU-GPS estimates were compared to the estimates of the onboard AHRS (attitude heading reference system) algorithm of the test aircraft. Estimation of errors relative to true orientation is done through the visualization of horizon line (obtained from the estimated Euler angles) in the camera image. This shows possible improvement in the precision by applying the Camera-IMU-GPS algorithm, but the considered very short (about 17sec) flight data section makes only the proof of concept possible. Final decision between the algorithms should be made considering long flight test results during the future development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alderton, S.: UAS Help New Zealand Farmer Count Sheep and Assess Grass Quality (2014). http://www.uasvision.com/2014/02/26/uas-helpnew-zealand-farmer-count-sheep-and-assess-grassquality/. Farmers Weekly

  2. Bauer, P., Bokor, J.: Multi-Mode Extended Kalman Filter for Aircraft Attitude Estimation. In: Proceedings of IFAC World Congress, p. 2011. IFAC, Milano, Italy (2011)

    Google Scholar 

  3. Beard, R.W., McLain, T.W.: Small unmanned aircraft: Theory and practice princeton university press (2012)

  4. Bouguet, J.Y.: Complete camera calibration toolbox for matlab (2004). http://www.vision.caltech.edu/bouguetj/calib_doc/index.html

  5. Campoy, P., Correa, J.F., Mondragn, I., Martnez, C., Olivares, M., Mejas, L., Artieda, J.: Computer vision onboard UAVs for civilian tasks. J. Intell. Robot. Syst. 54, 105–135 (2009)

    Article  Google Scholar 

  6. Choi, S., Kim, T., Yu, W.: Performance Evaluation of RANSAC Family. In: Proceedings of British Machine Vision Conference - BMVC, 2009 (2009)

  7. Chu, C.C., Lie, F.A.P., Lemay, L., Gebre-Egziabher, D.: Performance Comparison of Tight and Loose INS-Camera Integration. In: Proceedings of the 24Th International Technical Meeting of the Satellite Division of the Institute of Navigation (2011)

  8. Chu, T., Guo, N., Backn, S., Akos, D.: Monocular Camera/IMU/GNSS Integration for Ground Vehicle Navigation in Challenging GNSS Environments. Sensors 12, 3162–3185 (2012)

    Article  Google Scholar 

  9. Cox, T.H., Nagy, C.J., Skoog, M.A., Somers, I.A.: Civil UAV Capability Assessment. Technical Report NASA (2004)

  10. Nister, D.: An efficient solution to the five-point relative pose problem. IEEE Trans. Pattern Anal. Mach. Intell. 26, 756–770 (2004)

    Article  Google Scholar 

  11. Dillingham, G.L.: Unmanned Aircraft Systems: Measuring Progress and Addressing Potential Privacy Concerns Would Facilitate Integration into the National Airspace System. Technical Report GAO-12-981, U.S. Government Accountability Office, Washington DC (2012)

  12. Gleason, S., Gebre-Egziabher, D.: GNSS Applications and methods Artech House (2009)

  13. Gorton, S.: UAS Firefighting Technology in New South Wales, Australia (2014). http://www.uasvision.com/2014/02/11/uasfirefighting-technology-in-new-south-walesaustralia/. Naaroma News

  14. Hartley, R., Zisserman, A.: Multiple view geometry in computer vision cambridge university press (2004)

  15. Hartley, R.I.: Chirality. Int. J. Comput. Vis. 26, 41–61 (1998)

    Article  Google Scholar 

  16. Kannan, P.: UAE Plans to Use Delivery UAS Within a Year (2014). http://www.uasvision.com/2014/02/19/uae-plansto-use-delivery-uas-within-a-year/. The Vancouver Sun

  17. Koldaev, A.V.: Non-Military UAV Applications. Technical Report., IRKUT (2007)

  18. Magdirila, P.: Philippines TV Use UAS for News Reporting and Rescue Operations (2014). http://www.uasvision.com/2014/02/13/philippinestv-use-uas-for-news-reporting-and-rescueoperations/. Yahoo News

  19. Mariottini, G.L., Prattichizzo, D.: EGT For multiple view geometry and visual servoing: robotics vision with pinhole and panoramic cameras. IEEE Robot. Autom. Mag. 12, 26–39 (2005)

    Article  Google Scholar 

  20. Morton, B.: UAS to Help with Search & Rescue in Metro Vancouver (2014). http://www.uasvision.com/2014/02/11/uas-to-helpwith-search-rescue-in-metro-vancouver/

  21. Paw, Y.C.: Synthesis and Validation of Flight Control for UAV. Ph.D. thesis, University of Minnesota (2009)

  22. Pratt, W.K.: Digital Image Processing, PIKS Inside. Wiley (2001)

  23. Renault: Kwid concept-car (2013). http://www.renault.com/en/innovation/l-universdu-design/pages/kwid-concept-car.aspx

  24. Stewnius, H.: Calibrated Fivepoint Solver (2010). http://www.vis.uky.edu/~stewe/FIVEPOINT/

  25. Stewnius, H., Engels, C., Nistr, D.: Recent developments on direct relative orientation. ISPRS J. Photogramm. Remote. Sens. 60, 284–294 (2006)

    Article  Google Scholar 

  26. Torra, P., Zisserman, A.: MLESAC: A new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. 78, 138–156 (2000)

    Article  Google Scholar 

  27. Vanek, B., Bauer, P., Gozse, I., Lukatsi, M., Reti, I., Bokor, J.: Safety Critical Platform for Mini UAS Insertion into the Common Airspace. In: Proceedings of AIAA Guidance, Navigation and Control Conference 2014 (2014)

  28. Vanek, B., Peni, T., Zarandy, A., Bokor, J., Zsedrovits, T., Roska, T.: Performance Analysis of a Vision Only Sense and Avoid System for Small UAVs. In: Proceedings of AIAA Guidance, Navigation, and Control Conference (2011)

  29. Vanek, B., Peni, T., Zarandy, A., Bokor, J., Zsedrovits, T., Roska, T.: Performance Analysis of a Vision Only Sense and Avoid System for Small UAVs. In: Proceedings of ACD 2011 (9Th European Workshop on Advanced Control and Diagnosis) (2011)

  30. Werner, D.: Making way for unmanned aircraft. Aerosp. Am., 28–32 (2014)

  31. Zarandy, A., Nagy, Z., Vanek, B., Kiss, T.Z.A., Nemeth, M.: A five-camera vision system for UAV visual attitude calculation and collision warning. Comput. Vis. Syst. 7963, 11–20 (2013)

    Article  Google Scholar 

  32. Zsedrovits, T., Bauer, P., Zarandy, A., Vanek, B., Bokor, J., Roska, T.: Towards Real-Time Visual and IMU Data Fusion. In: Proceedings of AIAA Guidance, Navigation, and Control Conference 2014 (2014)

  33. Zsedrovits, T., Zarandy, A., Vanek, B., Peni, T., Bokor, J., Roska, T.: Collision Avoidance for UAV Using Visual Detection. In: Proceedings of IEEE ISCAS 2011 (International Symposium on Circuits and Systems), pp. 2173–2176. IEEE, Rio de Janeiro (2011)

    Chapter  Google Scholar 

  34. Zsedrovits, T., Zarandy, A., Vanek, B., Peni, T., Bokor, J., Roska, T.: Visual Detection and Implementation Aspects of a UAV See and Avoid System. In: Proceedings of ECCTD 2011 (20Th European Conference on Circuit Theory and Design), pp. 472–475, LinkÖping Sweden (2011)

Download references

Author information

Authors and Affiliations

  1. Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Budapest, Hungary

    Tamas Zsedrovits

  2. Institute for Computer Science and Control, Hungarian Academy of Sciences, Budapest, Hungary

    Peter Bauer, Antal Hiba, Mate Nemeth, Borbala Jani Matyasne Pencz, Akos Zarandy, Bálint Vanek & József Bokor

Authors
  1. Tamas Zsedrovits
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter Bauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antal Hiba
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mate Nemeth
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Borbala Jani Matyasne Pencz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Akos Zarandy
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bálint Vanek
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. József Bokor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tamas Zsedrovits.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zsedrovits, T., Bauer, P., Hiba, A. et al. Performance Analysis of Camera Rotation Estimation Algorithms in Multi-Sensor Fusion for Unmanned Aircraft Attitude Estimation. J Intell Robot Syst 84, 759–777 (2016). https://doi.org/10.1007/s10846-016-0346-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-016-0346-z

Keywords

Search

Navigation