Abstract
In this paper the possibility to use the camera of the onboard sense and avoid system as an aid in unmanned aircraft attitude estimation is examined. At first, the error analysis of four image processing algorithms targeting the reconstruction of camera orientation change is done. Camera spatial resolution, sampling frequency (temporal resolution), pixelization errors and image noises are all considered in the error analysis. After performing the error analysis, the camera rotation estimates from the two best algorithms are integrated into an IMU (inertial measurement unit)-GPS attitude estimator to obtain a Camera-IMU-GPS estimator. The attitude estimation results of the two algorithms are compared to each other considering hardware-in-the-loop simulated ’flight data’ and simulated camera images. After experiencing estimation error decrease with the augmented Camera-IMU-GPS algorithm a final proof of concept was conducted based-on real flight data. Off-line calculated IMU-GPS and Camera-IMU-GPS estimates were compared to the estimates of the onboard AHRS (attitude heading reference system) algorithm of the test aircraft. Estimation of errors relative to true orientation is done through the visualization of horizon line (obtained from the estimated Euler angles) in the camera image. This shows possible improvement in the precision by applying the Camera-IMU-GPS algorithm, but the considered very short (about 17sec) flight data section makes only the proof of concept possible. Final decision between the algorithms should be made considering long flight test results during the future development.
Similar content being viewed by others
References
Alderton, S.: UAS Help New Zealand Farmer Count Sheep and Assess Grass Quality (2014). http://www.uasvision.com/2014/02/26/uas-helpnew-zealand-farmer-count-sheep-and-assess-grassquality/. Farmers Weekly
Bauer, P., Bokor, J.: Multi-Mode Extended Kalman Filter for Aircraft Attitude Estimation. In: Proceedings of IFAC World Congress, p. 2011. IFAC, Milano, Italy (2011)
Beard, R.W., McLain, T.W.: Small unmanned aircraft: Theory and practice princeton university press (2012)
Bouguet, J.Y.: Complete camera calibration toolbox for matlab (2004). http://www.vision.caltech.edu/bouguetj/calib_doc/index.html
Campoy, P., Correa, J.F., Mondragn, I., Martnez, C., Olivares, M., Mejas, L., Artieda, J.: Computer vision onboard UAVs for civilian tasks. J. Intell. Robot. Syst. 54, 105–135 (2009)
Choi, S., Kim, T., Yu, W.: Performance Evaluation of RANSAC Family. In: Proceedings of British Machine Vision Conference - BMVC, 2009 (2009)
Chu, C.C., Lie, F.A.P., Lemay, L., Gebre-Egziabher, D.: Performance Comparison of Tight and Loose INS-Camera Integration. In: Proceedings of the 24Th International Technical Meeting of the Satellite Division of the Institute of Navigation (2011)
Chu, T., Guo, N., Backn, S., Akos, D.: Monocular Camera/IMU/GNSS Integration for Ground Vehicle Navigation in Challenging GNSS Environments. Sensors 12, 3162–3185 (2012)
Cox, T.H., Nagy, C.J., Skoog, M.A., Somers, I.A.: Civil UAV Capability Assessment. Technical Report NASA (2004)
Nister, D.: An efficient solution to the five-point relative pose problem. IEEE Trans. Pattern Anal. Mach. Intell. 26, 756–770 (2004)
Dillingham, G.L.: Unmanned Aircraft Systems: Measuring Progress and Addressing Potential Privacy Concerns Would Facilitate Integration into the National Airspace System. Technical Report GAO-12-981, U.S. Government Accountability Office, Washington DC (2012)
Gleason, S., Gebre-Egziabher, D.: GNSS Applications and methods Artech House (2009)
Gorton, S.: UAS Firefighting Technology in New South Wales, Australia (2014). http://www.uasvision.com/2014/02/11/uasfirefighting-technology-in-new-south-walesaustralia/. Naaroma News
Hartley, R., Zisserman, A.: Multiple view geometry in computer vision cambridge university press (2004)
Hartley, R.I.: Chirality. Int. J. Comput. Vis. 26, 41–61 (1998)
Kannan, P.: UAE Plans to Use Delivery UAS Within a Year (2014). http://www.uasvision.com/2014/02/19/uae-plansto-use-delivery-uas-within-a-year/. The Vancouver Sun
Koldaev, A.V.: Non-Military UAV Applications. Technical Report., IRKUT (2007)
Magdirila, P.: Philippines TV Use UAS for News Reporting and Rescue Operations (2014). http://www.uasvision.com/2014/02/13/philippinestv-use-uas-for-news-reporting-and-rescueoperations/. Yahoo News
Mariottini, G.L., Prattichizzo, D.: EGT For multiple view geometry and visual servoing: robotics vision with pinhole and panoramic cameras. IEEE Robot. Autom. Mag. 12, 26–39 (2005)
Morton, B.: UAS to Help with Search & Rescue in Metro Vancouver (2014). http://www.uasvision.com/2014/02/11/uas-to-helpwith-search-rescue-in-metro-vancouver/
Paw, Y.C.: Synthesis and Validation of Flight Control for UAV. Ph.D. thesis, University of Minnesota (2009)
Pratt, W.K.: Digital Image Processing, PIKS Inside. Wiley (2001)
Renault: Kwid concept-car (2013). http://www.renault.com/en/innovation/l-universdu-design/pages/kwid-concept-car.aspx
Stewnius, H.: Calibrated Fivepoint Solver (2010). http://www.vis.uky.edu/~stewe/FIVEPOINT/
Stewnius, H., Engels, C., Nistr, D.: Recent developments on direct relative orientation. ISPRS J. Photogramm. Remote. Sens. 60, 284–294 (2006)
Torra, P., Zisserman, A.: MLESAC: A new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. 78, 138–156 (2000)
Vanek, B., Bauer, P., Gozse, I., Lukatsi, M., Reti, I., Bokor, J.: Safety Critical Platform for Mini UAS Insertion into the Common Airspace. In: Proceedings of AIAA Guidance, Navigation and Control Conference 2014 (2014)
Vanek, B., Peni, T., Zarandy, A., Bokor, J., Zsedrovits, T., Roska, T.: Performance Analysis of a Vision Only Sense and Avoid System for Small UAVs. In: Proceedings of AIAA Guidance, Navigation, and Control Conference (2011)
Vanek, B., Peni, T., Zarandy, A., Bokor, J., Zsedrovits, T., Roska, T.: Performance Analysis of a Vision Only Sense and Avoid System for Small UAVs. In: Proceedings of ACD 2011 (9Th European Workshop on Advanced Control and Diagnosis) (2011)
Werner, D.: Making way for unmanned aircraft. Aerosp. Am., 28–32 (2014)
Zarandy, A., Nagy, Z., Vanek, B., Kiss, T.Z.A., Nemeth, M.: A five-camera vision system for UAV visual attitude calculation and collision warning. Comput. Vis. Syst. 7963, 11–20 (2013)
Zsedrovits, T., Bauer, P., Zarandy, A., Vanek, B., Bokor, J., Roska, T.: Towards Real-Time Visual and IMU Data Fusion. In: Proceedings of AIAA Guidance, Navigation, and Control Conference 2014 (2014)
Zsedrovits, T., Zarandy, A., Vanek, B., Peni, T., Bokor, J., Roska, T.: Collision Avoidance for UAV Using Visual Detection. In: Proceedings of IEEE ISCAS 2011 (International Symposium on Circuits and Systems), pp. 2173–2176. IEEE, Rio de Janeiro (2011)
Zsedrovits, T., Zarandy, A., Vanek, B., Peni, T., Bokor, J., Roska, T.: Visual Detection and Implementation Aspects of a UAV See and Avoid System. In: Proceedings of ECCTD 2011 (20Th European Conference on Circuit Theory and Design), pp. 472–475, LinkÖping Sweden (2011)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zsedrovits, T., Bauer, P., Hiba, A. et al. Performance Analysis of Camera Rotation Estimation Algorithms in Multi-Sensor Fusion for Unmanned Aircraft Attitude Estimation. J Intell Robot Syst 84, 759–777 (2016). https://doi.org/10.1007/s10846-016-0346-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-016-0346-z