Object detection with feature stability over scale space
Highlights
► We propose a segmentation method based on scale space endowed with feature stability. ► The stability of blob’s features is taken into account for blob lifetime measurement. ► The algorithm outperforms conventional space scale algorithm. ► Variable size and variable shape objects are efficiently detected by the algorithm. ► It can also be used as a preprocessing step in pattern recognition applications.
Introduction
One of the most challenging tasks in computer vision is the segmentation of objects in the image. Typically, the image segmentation is used to locate regions of interest within the image. The process usually starts from dividing the original image into several homogenous parts with respect to some properties such as intensity, color or texture. The obtained results will be used in subsequent tasks such as object recognition or classification. Obviously, the performance of object recognition and classification depends on the quality of the image segmentation process.
A number of segmentation methods have been proposed. According to features used, they can be divided into two groups, namely, region-based [1], [2], [3] and edge-based segmentations [4], [5], [6], [7].
Many previous works related to region-based segmentation or blob detection were proposed. Kawaguchi et al. [8] present blob analysis used to detect eyes from human face images. The algorithm extracts blobs by searching for intensity peaks and valleys obtained from binarized image and computes a cost using a Hough transform for each pair of blobs. A pair of blobs which has the smallest cost will be selected as the irises of both eyes. However, the performance of this method relies on the selection of a proper intensity threshold and type of template used to detect blobs. Damerval and Meignen [9] present a blob detection based on maxima lines of the continuous wavelet transform. The detection method is not based on a thresholding step but uses properties of maxima lines of regions of interest. A framework for detecting interesting blobs in the color domain is presented in [10]. The method consists of a weighted multi-scale blob detector, hue-based color histogram and Forstner operator for roundness calculation. In [11], an intelligent vehicle counting method based on blob analysis in traffic surveillance is presented. The algorithm consists of three steps, namely, moving object segmentation, blob analysis, and vehicle tracking. The velocity of each vehicle can be calculated by analyzing blobs of vehicles. Another blob analysis method used for detecting moving objects is presented in [12]. The method consists of three steps: the symmetric difference is used to extract a rough moving object, blob analysis is used to update the background model, and a proposed classification strategy is used to extract foreground.
Even though these algorithms are claimed to work efficiently and produce high accuracy segmentation results, most schemes require prior knowledge about the region of interest, e.g., color, size or shape. This information will then be used to specify parameters in order to improve the segmentation such as intensity thresholds, the size of template or window operator sizes, etc. Without the accuracy of the parameters, the proper segmentation could not be well executed. However, once the parameters are set to suit one set of test images, they might have to be readjusted again when a new set arrives. Even in the same image, the problem may be more prominent if the target objects appear in different sizes or uncertain shapes. Besides, analysis of only similarity of the region cannot yield the best result especially for segmenting objects from an image with a complex scene, such as traffic images, or objects which consist of several parts, such as buildings or flowers.
Since details of objects in an image only exist and make sense in some observed scales, so the concept of hierarchical structures of an image is required to describe the structure of an image in different aspects. The idea of creating multi-scale representation of signals is first proposed by Witkin [13], called scale-space theory, to analyze a 1-D signal. Later, Lindeberg [14], [15] employs the scale space approach to detect local maxima with extent in a 2-D intensity image, called grey-level blobs, at multiple scales. The relationship of all detected blobs at all scales can be considered by constructing a scale-space blob tree. In the absence of further information, the significance of a blob can be measured using its attributes, e.g., grey-level intensity, color, etc. The important structures in the image can be obtained by selecting the blobs with more significance.
The concept of scale space has been widely used in several applications for detecting image features such as blobs, edges, ridges and corners. Carvalho et al. [16] proposed the method to segment yeast cells based on watershed and scale space analysis. Trees and calculated node attributes are built such as survival time, shape and gray-scale, in order to perform segmentation analysis. The automatic scale selection for edge and ridge detection is presented in [17]. The number of strongest edge responses is selected by analyzing the integrated normalized gradient magnitude along the scale-space edges. For ridge detection, the ridge strength is considered by maximizing a normalized measure of ridge strength over scales. The proposed method for detecting corners in [18] employs the scale-space method and Plessey operator to detect corners belonging to different scales instead of a certain scale. The final result is obtained by combining the corners detected at every scale and a tracking back algorithm is applied to get the accurate localization. A multi-scale method for shape recognition is presented by Jalba et al. [19]. The method is based on two morphological scale-space representations and the hat-transform in scale space. They use maximum heights of the extrema of the curvature function as a shape descriptor.
In normal blob linking processes (to form a scale space tree), only spatial information of the blobs in consecutive scales is used. The main flaw within this process is that the blurring does not take into account with any other information on the image. For example, two different blobs with different colors will still be merged if they stay close together spatially.
In this paper, we propose an object detection method based on scale space by incorporating features of blobs into the scale space blob linking process as well as spatial information. We demonstrate in the paper that blobs are linked if their features are stable over scales, we get a better performance in the object detection. The evolution of linked blobs over scales using feature stability presents how stable image structures are in scale-space.
Section snippets
Scale space representation
In scale-space theory, a multi-scale representation of a two-dimension image, f(x, y), is defined by a convolution with the Gaussian kernel f(x, y, σ). The successive smoothing process generates a set of output images in various scales, σ. A scale parameter, σ, of the kernel is gradually increased many times to create a series of smoothed images. During the blurring process, less important details of the image are suppressed while prominent structures and features still remain. In other words, the
Numerical experiments
To demonstrate the robustness and generality of the algorithm we consider the stability of three features: the entropy, the average value of the gray level of the image and the standard deviation of the gray level. These three values represent the texture, the gray level and the gray level distribution, respectively. No systematic feature selection was performed. The feature selection was intuitive but supported by some standard pattern recognition schemes [20], [21], [22], [23], [24], [25],
Conclusion
We have proposed a novel object detection method based on scale-space endowed with feature stability. The algorithm is robust for detecting variable size and variable shape objects without a priori information of the object of interest. The shapes and sizes of multiple objects such as the flying birds or sunflowers are not uniform throughout the image but the algorithm can detect most of them.
The algorithm is flexible. The feature vectors can be extended or modified to suit particular
Acknowledgments
This project is funded by Guy’s and St. Thomas’ Hospital Special Trustees, UK and Thailand Toray Science Foundation (TTSF) and National Research University Project of Thailand Office of Higher Education Commission.
References (28)
- et al.
Edge detectorevaluation using empirical roc curves
Computer Vision and Image Understanding
(2001) - et al.
Automatic detection of diabetic retinopathy exudates from non-dilated retinal images using mathematical morphology methods
Computerized Medical Imaging and Graphics
(2008) - et al.
Watershed-based textural image segmentation
Intelligent Signal Processing and Communication Systems
(2007) - et al.
Extensive partition operators, gray-level connected operators, and region merging/classification segmentation algorithms: theoretical links
IEEE Transactions on Image Processing
(2001) - et al.
Anti-geometric diffusion for adaptive thresholding and fast segmentation
IEEE Transactions on Image Processing
(2003) - et al.
Inconsistencies in edge detector evaluation
Computer Vision and Pattern Recognition
(2000) - et al.
Statistical edge detection: learning and evaluating edge cues
IEEE Transactions on Pattern Analysis and Machine Intelligence
(2003) - H. Gomez-Moreno, S. Maldonado-Bascon, F. Lopez-Ferreras, Edge detection in noisy images using the support vector...
- T. Kawaguchi, D. Hidaka, M. Rizon, Robust extraction of eyes from face, in: Proceedings on the 15th International...
- et al.
Blob detection with wavelet maxima lines
IEEE Signal Processing Letters
(2007)
Tracking scale-space blobs for video description
IEEE Multimedia
Cited by (7)
Multi-target location of infrared image of power equipment
2018, Chemical Engineering TransactionsMobile video communication based on augmented reality
2017, Multimedia Tools and ApplicationsHigh-speed multiple spatial band-pass filtering using a resistive network
2013, 2013 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2013Learning scale ranges for the extraction of regions of interest
2012, Proceedings - International Conference on Image Processing, ICIPGrowing least squares for the analysis of manifolds in scale-space
2012, Eurographics Symposium on Geometry ProcessingGrowing least squares for the analysis of manifolds in scale-space
2012, Computer Graphics Forum