Skip to main content
SpringerLink
Log in

Homography vs similarity transformation in aerial mosaicking: which is the best at different altitudes?

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Aerial image mosaicking of an area of interest is the process of combining multiple images, of an area with overlapping regions, into a single comprehensive view. In this process, image registration, i.e., the operation of geometric transformation to align and overlay two or more images of the same scene taken from different viewpoints, starting from their common parts, plays a key role in terms of artifacts reduction. In the current state-of-the-art, image registration of aerial images is usually performed through the use of the homography transformation. This occurs because these images are frequently acquired at high altitudes (more than 100 meters) and the homography has always provided excellent performance. The recent widespread of Unmanned Aerial Vehicles (UAVs) has enabled the development of several applications where mosaics are used as reference images for high precision tasks, including Detection, Recognition, and Identification (hereinafter DRI) of people and objects. These tasks need to acquire images at very low altitudes (below 50 meters), in which the homography tends to introduce artifacts during the registration process. Therefore, a different transformation able to limit how an image can be morphed, i.e., the similarity transformation, is necessary to perform the image registration, thus improving the overall accuracy of the obtained mosaics. In this paper, for the first time in literature, a comparison between the homography and similarity transformations is performed. In particular, the comparison is carried out by using three recently released public datasets, i.e., NPU Drone-Map, senseFly, and UAV Mosaicking and Change Detection (UMCD), containing challenging aerial video sequences acquired at high and low altitudes. The experimental tests have pointed out the direct relationship among best image transformation, UAV altitude, and spatial resolution, required to accomplish the DRI tasks reported above.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://www.sensefly.com/education/datasets/

  2. https://msis.jsc.nasa.gov/sections/section03.htm

References

  1. Abraham R, Simon P (2013) Review on mosaicing techniques in image processing. In: International Conference on Advanced Computing and Communication Technologies (ACCT), pp 63–68

  2. Abughalieh KM, Sababha BH, Rawashdeh NA (2019) A video-based object detection and tracking system for weight sensitive UAVs. Multimed Tools Appl 78 (7):9149–9167

    Google Scholar 

  3. Ammour N, Alhichri H, Bazi Y, Benjdira B, Alajlan N, Zuair M (2017) Deep learning approach for car detection in UAV imagery. Remote Sens 9(4):1–15

    Google Scholar 

  4. Avola D, Foresti GL, Martinel N, Micheloni C, Pannone D, Piciarelli C (2017) Aerial video surveillance system for small-scale UAV environment monitoring. In: IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS), pp 1–6

  5. Avola D, Foresti GL, Martinel N, Micheloni C, Pannone D, Piciarelli C (2017) Real-time incremental and geo-referenced mosaicking by small-scale UAVs. In: International Conference on Image Analysis and Processing (ICIAP), pp 694–705

  6. Avola D, Cinque L, Foresti GL, Martinel N, Pannone D, Piciarelli C (2018) Low-Level Feature Detectors and Descriptors for Smart Image and Video Analysis: A Comparative Study. Springer International Publishing, Cham, pp 7–29

    Google Scholar 

  7. Avola D, Cinque L, Foresti GL, Martinel N, Pannone D, Piciarelli C (2018). A UAV video dataset for mosaicking and change detection from low-altitude flights. IEEE Transactions on Systems, Man, and Cybernetics: Systems pp 1–11 (in press)

  8. Avola D, Cinque L, Foresti GL, Pannone D (2019) Visual cryptography for detecting hidden targets by small-scale robots. In: International Conference on Pattern Recognition Applications and Methods (ICPRAM) - Revised Selected Papers, pp 186–201

  9. Bay H, Ess A, Tuytelaars T, Gool LV (2008) Speeded-up robust features (surf). Comput Vis Image Underst 110(3):346–359

    Google Scholar 

  10. Bejiga MB, Zeggada A, Nouffidj A, Melgani F (2017) A convolutional neural network approach for assisting avalanche search and rescue operations with UAV imagery. Remote Sens 9(2):1–22

    Google Scholar 

  11. Brown M, Lowe DG (2007) Automatic panoramic image stitching using invariant features. Int J Comput Vis 74(1):59–73

    Google Scholar 

  12. Bu S, Zhao Y, Wan G, Liu Z (2016) Map2DFusion: Real-time incremental UAV image mosaicing based on monocular SLAM. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 4564–4571

  13. Camargo A, Schultz RR, Wang Y, Fevig RA, He Q (2010) GPU-CPU implementation for super-resolution mosaicking of unmanned aircraft system (UAS) surveillance video. In: IEEE Southwest Symposium on Image Analysis Interpretation (SSIAI), pp 25–28

  14. Chen J, Luo L, Wang S, Wu H (2018) Automatic panoramic uav image mosaic using ORB features and robust transformation estimation. In: Chinese Control Conference (CCC), pp 4265–4270

  15. Chen Y, Hakala T, Karjalainen M, Feng Z, Tang J, Litkey P, Kukko A, Jaakkola A, Hyyppä J (2017) UAV-Borne profiling radar for forest research. Remote Sens 9(1):1–17

    Google Scholar 

  16. Dhana Lakshmi M, Mirunalini P, Priyadharsini R, Mirnalinee TT (2019) Review of feature extraction and matching methods for drone image stitching. In: Proceedings of the International Conference on ISMAC in Computational Vision and Bio-Engineering (ISMAC-CVB), pp 595–602

  17. Fan DP, Cheng MM, Liu Y, Li T, Borji A (2017) Structure-measure: A new way to evaluate foreground maps. In: International Conference on Computer Vision (ICCV), pp 4558–4567

  18. Fang S, Li J, Tian Y, Huang T, Chen X (2017) Learning discriminative subspaces on random contrasts for image saliency analysis. IEEE Transactions on Neural Networks and Learning Systems 28(5):1095–1108

    Google Scholar 

  19. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    MathSciNet  Google Scholar 

  20. Fu K, Li J, Shen H, Tian Y (2018). How Drones Look:, Crowdsourced Knowledge Transfer for Aerial Video Saliency Prediction. arXiv:1811.05625

  21. Ghosh D, Kaabouch N (2016) A survey on image mosaicing techniques. J Vis Commun Image Represent 34:1–11

    Google Scholar 

  22. Guo T, Kujirai T, Watanabe T (2012) Mapping crop status from an unmanned aerial vehicle for precision agriculture applications. International Archives of the Photogrammetry Remote Sensing and Spatial Information Sciences 39(B1):485–490

    Google Scholar 

  23. He B, Yu S (2015) Parallax-robust surveillance video stitching. Sensors 16 (1):1–12

    MathSciNet  Google Scholar 

  24. Huang Y, Li J, Fan N (2008) Image mosaicing for UAV application. In: International Symposium on Knowledge Acquisition and Modeling (KAM), pp 663–667

  25. Johnson J (1985) Analysis of image forming systems, Proc SPIE 513

  26. Li J, Tian Y, Huang T, Gao W (2010) Probabilistic multi-task learning for visual saliency estimation in video. Int J Comput Vis 90(2):150–165

    Google Scholar 

  27. Li J, Duan L, Chen X, Huang T, Tian Y (2015) Finding the secret of image saliency in the frequency domain. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(12):2428–2440

    Google Scholar 

  28. Li J, Fu K, Zhao S, Ge S (2020) Spatiotemporal knowledge distillation for efficient estimation of aerial video saliency. IEEE Trans Image Process 29:1902–1914

    MathSciNet  Google Scholar 

  29. Lipinski D, Mohseni K (2016) Micro/miniature aerial vehicle guidance for hurricane research. IEEE Syst J 10(3):1263–1270

    Google Scholar 

  30. Liu F, Yang A (2019) Application of gcforest to visual tracking using UAV image sequences. Multimed Tools Appl 78(19):27, 933-27, 956

  31. Liu Y, Bai B, Zhang C (2017) UAV image mosaic for road traffic accident scene. In: Youth Academic Annual Conference of Chinese Association of Automation (YAC), pp 1048–1052

  32. Oettershagen P, Stastny T, Hinzmann T, Rudin K, Mantel T, Melzer A, Wawrzacz B, Hitz G, Siegwart R (2018) Robotic technologies for solar-powered UAVs: Fully autonomous updraft-aware aerial sensing for multiday search-and-rescue missions. Journal of Field Robotics 35(4):612–640

    Google Scholar 

  33. Piciarelli C, Avola D, Pannone D, Foresti GL (2019) A vision-based system for internal pipeline inspection. IEEE Transactions on Industrial Informatics 15 (6):3289–3299

    Google Scholar 

  34. Prathap KSV, Jilani SAK, Reddy PR (2016) A critical review on image mosaicing. In: International Conference on Computer Communication and Informatics (ICCCI), pp 1–8

  35. Sakla W, Konjevod G, Mundhenk TN (2017) Deep multi-modal vehicle detection in aerial isr imagery. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp 916–923

  36. Song H, Yang C, Zhang J, Hoffmann WC, He D, Thomasson JA (2016) Comparison of mosaicking techniques for airborne images from consumer-grade cameras. J Appl Remote Sens 10(1):1–14

    Google Scholar 

  37. Sun S, Zeng Z (2013) Uav image mosaic based on adaptive SIFT algorithm. In: International Conference on Geoinformatics, pp 1–6

  38. Wang S, Zhang Y, Wang W, Zhao Y, Zhu S (2018 ) A novel image mosaic method based on improved ORB and its application in police-UAV. In: IEEE International conference on internet of things (iThings) and IEEE green computing and communications (GreenCom) and IEEE Cyber Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pp 1707–1713

  39. Wang W, Shen J (2018) Deep visual attention prediction. IEEE Trans Image Process 27(5):2368–2378

    MathSciNet  Google Scholar 

  40. Wei Q, Lao S, Bai L (2017) Panorama stitching, moving object detection and tracking in UAV videos. In: International Conference on Vision, Image and Signal Processing (ICVISP), 46–50

  41. Wischounig-Strucl D, Rinner B (2015) Resource aware and incremental mosaics of wide areas from small-scale UAVs. Mach Vis Appl 26(7-8):885–904

    Google Scholar 

  42. Zhang Y, Jin X, Wang Z (2017) A new modified panoramic uav image stitching model based on the GA-SIFT and adaptive threshold method. Memetic Computing 9:231–244

    Google Scholar 

  43. Zhang Y, Yuan X, Li W, Chen S (2017) Automatic power line inspection using UAV images. Remote Sensing 9(8):1–19

    Google Scholar 

  44. Zhao J, Zhang X, Gao C, Qiu X, Tian Y, Zhu Y, Cao W (2019) Rapid mosaicking of unmanned aerial vehicle (UAV) images for crop growth monitoring using the SIFT algorithm. Remote Sens 11(10):1–19

    Google Scholar 

  45. Zhou Z, Zhang C, Xu C, Xiong F, Zhang Y, Umer T (2018) Energy-efficient industrial internet of UAVs for power line inspection in smart grid. IEEE Transactions on Industrial Informatics 14(6):2705–2714

    Google Scholar 

  46. Zhou W, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612

    Google Scholar 

  47. Zitová B, Flusser J (2003) Image registration methods: a survey. Image Vis Comput 21(11):977–1000

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the MIUR under grant “Departments of Excellence 2018-2022” of the Department of Computer Science of Sapienza University.

Author information

Authors and Affiliations

  1. Department of Computer Science, Sapienza University, Via Salaria 113, 00198, Rome, Italy

    Danilo Avola, Luigi Cinque & Daniele Pannone

  2. Department of Mathematics, Computer Science and Physics, University of Udine, Via delle Scienze 206, 33100, Udine, Italy

    Gian Luca Foresti

Authors
  1. Danilo Avola
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luigi Cinque
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gian Luca Foresti
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniele Pannone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Danilo Avola.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Avola, D., Cinque, L., Foresti, G.L. et al. Homography vs similarity transformation in aerial mosaicking: which is the best at different altitudes?. Multimed Tools Appl 79, 18387–18404 (2020). https://doi.org/10.1007/s11042-020-08758-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08758-0

Keywords

Search

Navigation