Symmetric stability of low level feature detectors☆
Introduction
There are many feature detectors documented in the literature, and used in research and practical applications to fulfil the common need to identify interest points within an image. Information at these positions can then be extracted into a descriptor and used for correspondence matching in image retrieval and classification, image alignment, image stitching, and many other applications. The two stages are often combined into one in discussion, but each are independent and the algorithms used in each can often be interchanged.
Most popular and useful feature descriptors are invariant to scale and rotation, and matching features from two images where they appear at different sizes or are rotated can still be successful. Invariance to bilateral symmetry in feature detectors, however, is less well documented. We describe our interest in this invariance and investigate the property for some popular feature detectors, assessing their consistency in finding interest points within an image and a horizontal reflection of the image. Our goal is to identify which popular feature detectors are most invariant to bilateral symmetry, and what degree of error exists in the interest point position, size and orientation.
To the best of our knowledge, no assessment of low level feature detectors with respect to their invariance to bilateral symmetry has previously appeared in the literature. The main contributions in this paper are:
- •
We introduce five measurements of error that we show to be useful in determining the invariance to bilateral symmetry of a feature detector; mean distance error, mean size error, mean angle error, mean descriptor distance error and the mean descriptor match error (Section 4).
- •
We measure the accuracy of bilateral keypoint position, size and angle of orientation in an established dataset [5] of 8677 images (Section 5).
- •
We evaluate the capability of popular detectors to find consistent interest points (Section 6).
Section snippets
Bilateral symmetry
Bilateral symmetry describes a symmetry through a vertical plane in an image, and can occur at different scales. Fig. 1 shows two examples; (a) the image as a whole is bilaterally symmetrical because the right hand side of the plane (the dotted blue line down the centre of the image) is a mirror image of the left hand side and (b) the highlighted section of the image is bilaterally symmetrical although the image as a whole is not. Detected keypoints in an image are generally very small and
Feature detectors
Feature detectors are used extensively in all areas of computer vision to identify parts of an image which contain pixel information that can be useful in many applications. Numerous detector methods have been described in the literature, and many have become popular for different tasks. Two distinct categories of feature detectors exist; keypoint detectors and region detectors. Recent trends in Deep Learning use features that are discovered automatically during the training process. In this
Experiments and data
We assess the eight keypoint detectors and two region detectors described above, using the well established CALTECH101 dataset [5]. The dataset consists of 8677 JPEG images grouped into 101 categories, and contains a variety of image styles including cartoons and photographs of objects, human faces, animals and natural scenes. MSCR is the only detector that works with 3-channel colour images and for all other detectors, the original colour images are first converted to intensity images.
To
Results
Using the ten detectors, 41.78 million keypoints where found in the 8677 images (Table 1). Overall, 1, 330 more keypoints were found in mirror images than in the original images, but this varied by descriptor. BRISK, SIFT, and MSER for example found 735, 644 and 437 more keypoints in the mirror images, but this represents only 0.05%, 0.02% and 0.06% increases. SURF found 494 fewer keypoints in the mirror image, 0.01%. Columns 4 and 5 of Table 1 show the number of category (of a total of 101)
Error measurements
The mean descriptor distance error and mean descriptor match error are perhaps the most important measurements in assessing invariance to bilateral symmetry. Mean descriptor distance error is the average Euclidean distance between matched descriptors measured in 128-dimension descriptor space. A mean of 0.0 indicates perfect matching. Mean descriptor match error is a measure of the matching accuracy based on descriptors against matching spatially. In a perfect set of bilateral feature
Related work
Most work on bilateral symmetry concentrates on the detection of symmetry and accurate positioning of the line of symmetry within a single image (e.g. [16], [17], [22]; see [14] for background work on Symmetry). Yang and Patras [24] investigated object localisation methods and concluded that all of the methods that they evaluated on two representative problems struggle to get mirror symmetric results. The authors introduced the concept of mirrorability in their assessment of accurate face
Conclusion
We have assessed ten popular image feature detectors to determine their invariance to bilateral symmetry. We focussed on the accuracy and consistency of feature detection between an image and its mirror reflection. We conclude (Table 4) that FAST and CenSurE detectors are perfectly invariant and GFTT and the Harris Corner detector are invariant after feature matching and filtering algorithms are applied to find the correct correspondences in uneven sized sets of detected interest points. BRISK,
Acknowledgements
This work is funded by the European Union’s Seventh Framework Programme, under Grant agreement number 607480 (LASIE IP project). The authors extend their thanks to the Metropolitan Police at Scotland Yard, London, UK, for the supply of and permission to use CCTV images (Fig. 2).
References (26)
- et al.
Speeded-up robust features (SURF)
Comput. Vis. Image Underst.
(2008) Maximally stable colour regions for recognition and matching
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
(2007)- et al.
ORB: An efficient alternative to SIFT or SURF
Proceedings of the IEEE International Conference on Computer Vision
(2011) - et al.
CenSurE: Center surround extremas for realtime feature detection and matching
Proceedings of the 2008 European Conference on Computer Vision, ECCV 2008
(2008) The openCV Library
Dr. Dobb’s J. Softw. Tools
(2000)- et al.
BRIEF: Binary robust independent elementary features
Proceedings of the 2010 European Conference on Computer Vision (ECCV’10)
(2010) - et al.
Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories
Comput. Vis. Image Underst.
(2007) - et al.
Shape descriptors for maximally stable extremal regions
Proceedings of the IEEE International Conference on Computer Vision
(2007) - et al.
FIND: A neat flip invariant descriptor
Proceedings of the International Conference on Pattern Recognition
(2010) - et al.
MIFT: A mirror reflection invariant feature descriptor
Proceedings of the Ninth Asian Conference on Computer Vision, ACCV2009
(2010)
A combined corner and edge detector
Proceedings of the 1988 Alvey Vision Conference
Mirror reflection invariant HOG descriptors for object detection
Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP)
BRISK: Binary robust invariant scalable keypoints
Proceedings of the 2011 International Conference on Computer Vision
Cited by (3)
A fast technique to detect copy-move image forgery with reflection and non-affine transformation attacks
2021, Expert Systems with ApplicationsConstruction and application of a probabilistic atlas of 3D landmark points for initialization of hippocampus mesh models in brain MR images
2018, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
- ☆
This paper has been recommended for acceptance by Dr. J. Laaksonen.