SIFT optimization and automation for matching images from multiple temporal sources

https://doi.org/10.1016/j.jag.2016.12.017Get rights and content

Highlights

  • An optimization of SIFT algorithm is proposed.

  • The optimization is applied to match multi-temporal images captured from different sources.

  • The optimization model has been validated with different imagery.

Abstract

Scale Invariant Feature Transformation (SIFT) was applied to extract tie-points from multiple source images. Although SIFT is reported to perform reliably under widely different radiometric and geometric conditions, using the default input parameters resulted in too few points being found. We found that the best solution was to focus on large features as these are more robust and not prone to scene changes over time, which constitutes a first approach to the automation of processes using mapping applications such as geometric correction, creation of orthophotos and 3D models generation. The optimization of five key SIFT parameters is proposed as a way of increasing the number of correct matches; the performance of SIFT is explored in different images and parameter values, finding optimization values which are corroborated using different validation imagery. The results show that the optimization model improves the performance of SIFT in correlating multitemporal images captured from different sources.

Introduction

Unmanned Aerial Vehicles (UAVs) have the following main advantages: low cost, greater flexibility, less affected by certain atmospheric conditions (clouds, humidity, etc.), high-precision position data and high temporal and spatial resolution images captured in low-level flight, large scale environmental monitoring, multi-spectral sensor capacity, etc.

UAVs can be piloted remotely, but they can also achieve autonomous flight using an embedded integrated avionic system, and their parameters can be monitored and modified using a ground control system (GCS) via a wireless link. This link can be implemented using xbee modules, which has been proven to be low cost and efficient in sensor networks (Mayalarp et al., 2010), and which allows UAVs to link not only GCS but also other sensor networks (Todd et al., 2007) with a maximum theoretical distance of 40km using dipole antennas without obstacles.

UAVs are used in a wide range of markets such as digital cities, ground station buildings, ground management, city planning, disaster forecast and evaluation, digital agriculture and mapping environment inspection. Some authors Gonçalves and Henriques (2015) suggest that UAVs will become increasingly common in different environments (urban, coastal, rural, etc.) and will be in widespread use in a few decades.

For applications such as making high precision orthophotos and digital elevation models, the image and other data from the UAV needs to be processed first. How to carry out these processes quickly and accurately is one of the key technologies for UAV remote sensing applications. New image acquisition technologies provide distorted images, but the absence of automated algorithms to fix them makes it impossible to use the full potential of the imagery produced by these new technologies; over the last few years, computer vision has been developed actively for image processing, focusing on mobile devices with embedded generic sensors, and developing new and advanced software applications and systems which would allow image matching and correction of distortion through correlation of images. However, it does not always gives good results when the images are obtained from different sources.

One solution is given in this paper, which focuses on the optimization and automation of methods and techniques to process and correlate images from multiple sources, providing a new methodology for use in other fields where manual intervention is required.

Section snippets

Related work

Image correlation techniques rely on imagery feature extraction, which uses detectors to search for features which are geometrically stable under different transformations, including distinctness, uniqueness and marker-less automated orientation procedures, etc.; these features are called interest points (Remondino, 2006), and define a descriptor as a 2D vector which provides pixel information. In photogrammetry, the most popular applications are in image orientation and 3D reconstruction.

In

SIFT

Scale Invariant Feature Transformation (SIFT) (Lowe, 2004) was originally developed for general purpose object recognition. SIFT detects distinctive invariant features from images and performs matching based on the descriptor representing each feature that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale, rotation, illumination, change in 3D viewpoint, affine distortion, addition of noise, and blurring. SIFT is

Results and discussion

To generate a set of aerial images, it was necessary to set up the UAV, hardware and sensors correctly, and produce a number of algorithms through testing of different configurations. The methodology followed and the results obtained are described below.

Conclusions

The correlation of images is a common application of SIFT, where the default parameters can be used for generic scenes, although some authors claim that certain applications present special and complex scenes, and a tuning of parameters could provide the best results. UAVs have become popular as common image capture platforms, providing more and more images captured over time with diverse sensors, but not enough studies have been made in the literature into the correlation of multitemporal and

References (50)

  • B. Arbab-Zavar et al.

    On model-based analysis of ear biometrics

    IEEE Conference on Biometrics: Theory, Applications and Systems, BTAS’07

    (2007)
  • Ardupilot mega, 2016. http://www.ardupilot.co.uk/ (accessed on...
  • S. Battiato et al.

    Sift features tracking for video stabilization

  • M. Bicego et al.

    On the use of sift features for face authentication

    Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

    (2006)
  • A. Cesetti et al.

    A vision-based guidance system for UAV navigation and safe landing using natural landmarks

    J. Intell. Robot. Syst.: Theory Appl.

    (2010)
  • D. Chen et al.

    City-scale landmark identification on mobile devices

    Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

    (2011)
  • W. Cheung et al.

    n-sift: n-dimensional scale invariant feature transform

    IEEE Trans. Image Process.

    (2009)
  • K. Chureesampant et al.

    Automatic GCP extraction of fully polarimetric SAR images

    IEEE Trans. Geosci. Remote Sens.

    (2014)
  • Exiv2 , 2016. http://www.exiv2.org/ (accessed on...
  • M.A. Fischler et al.

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography

    Commun. ACM

    (1981)
  • Futaba ff9, 2016. http://www.futaba-rc.com (accessed on...
  • L.-C. Geng et al.

    A new camera self-calibration method based on CSA

    IEEE VCIP 2013 – 2013 IEEE International Conference on Visual Communications and Image Processing

    (2013)
  • B. Girod et al.

    Mobile visual search

    IEEE Signal Process. Mag.

    (2011)
  • Gopro yhdc5170, 2016. http://es.gopro.com/ (accessed on...
  • N. Henze et al.

    What is that? Object recognition from natural features on a mobile phone

    Proceedings of the Workshop on Mobile Interaction with the Real World

    (2009)
  • Cited by (0)

    View full text