Object detection with feature stability over scale space

https://doi.org/10.1016/j.jvcir.2011.02.004Get rights and content

Abstract

This paper proposes a novel segmentation method based on the scale space techniques endowed with a feature stability approach. The novelty of the paper is the lifetime of the space-scale blobs measured not only by their presence and disappearance but by the stability of the features characterizing the objects of interest as well. Our numerical experiments show that the algorithm outperforms the conventional space scale algorithm applied to variable size and variable shape objects. The proposed algorithm can be used as a preprocessing step in object or pattern recognition applications to produce seeds for more accurate image segmentation methods such as the snakes or the level set techniques.

Highlights

► We propose a segmentation method based on scale space endowed with feature stability. ► The stability of blob’s features is taken into account for blob lifetime measurement. ► The algorithm outperforms conventional space scale algorithm. ► Variable size and variable shape objects are efficiently detected by the algorithm. ► It can also be used as a preprocessing step in pattern recognition applications.

Introduction

One of the most challenging tasks in computer vision is the segmentation of objects in the image. Typically, the image segmentation is used to locate regions of interest within the image. The process usually starts from dividing the original image into several homogenous parts with respect to some properties such as intensity, color or texture. The obtained results will be used in subsequent tasks such as object recognition or classification. Obviously, the performance of object recognition and classification depends on the quality of the image segmentation process.

A number of segmentation methods have been proposed. According to features used, they can be divided into two groups, namely, region-based [1], [2], [3] and edge-based segmentations [4], [5], [6], [7].

Many previous works related to region-based segmentation or blob detection were proposed. Kawaguchi et al. [8] present blob analysis used to detect eyes from human face images. The algorithm extracts blobs by searching for intensity peaks and valleys obtained from binarized image and computes a cost using a Hough transform for each pair of blobs. A pair of blobs which has the smallest cost will be selected as the irises of both eyes. However, the performance of this method relies on the selection of a proper intensity threshold and type of template used to detect blobs. Damerval and Meignen [9] present a blob detection based on maxima lines of the continuous wavelet transform. The detection method is not based on a thresholding step but uses properties of maxima lines of regions of interest. A framework for detecting interesting blobs in the color domain is presented in [10]. The method consists of a weighted multi-scale blob detector, hue-based color histogram and Forstner operator for roundness calculation. In [11], an intelligent vehicle counting method based on blob analysis in traffic surveillance is presented. The algorithm consists of three steps, namely, moving object segmentation, blob analysis, and vehicle tracking. The velocity of each vehicle can be calculated by analyzing blobs of vehicles. Another blob analysis method used for detecting moving objects is presented in [12]. The method consists of three steps: the symmetric difference is used to extract a rough moving object, blob analysis is used to update the background model, and a proposed classification strategy is used to extract foreground.

Even though these algorithms are claimed to work efficiently and produce high accuracy segmentation results, most schemes require prior knowledge about the region of interest, e.g., color, size or shape. This information will then be used to specify parameters in order to improve the segmentation such as intensity thresholds, the size of template or window operator sizes, etc. Without the accuracy of the parameters, the proper segmentation could not be well executed. However, once the parameters are set to suit one set of test images, they might have to be readjusted again when a new set arrives. Even in the same image, the problem may be more prominent if the target objects appear in different sizes or uncertain shapes. Besides, analysis of only similarity of the region cannot yield the best result especially for segmenting objects from an image with a complex scene, such as traffic images, or objects which consist of several parts, such as buildings or flowers.

Since details of objects in an image only exist and make sense in some observed scales, so the concept of hierarchical structures of an image is required to describe the structure of an image in different aspects. The idea of creating multi-scale representation of signals is first proposed by Witkin [13], called scale-space theory, to analyze a 1-D signal. Later, Lindeberg [14], [15] employs the scale space approach to detect local maxima with extent in a 2-D intensity image, called grey-level blobs, at multiple scales. The relationship of all detected blobs at all scales can be considered by constructing a scale-space blob tree. In the absence of further information, the significance of a blob can be measured using its attributes, e.g., grey-level intensity, color, etc. The important structures in the image can be obtained by selecting the blobs with more significance.

The concept of scale space has been widely used in several applications for detecting image features such as blobs, edges, ridges and corners. Carvalho et al. [16] proposed the method to segment yeast cells based on watershed and scale space analysis. Trees and calculated node attributes are built such as survival time, shape and gray-scale, in order to perform segmentation analysis. The automatic scale selection for edge and ridge detection is presented in [17]. The number of strongest edge responses is selected by analyzing the integrated normalized gradient magnitude along the scale-space edges. For ridge detection, the ridge strength is considered by maximizing a normalized measure of ridge strength over scales. The proposed method for detecting corners in [18] employs the scale-space method and Plessey operator to detect corners belonging to different scales instead of a certain scale. The final result is obtained by combining the corners detected at every scale and a tracking back algorithm is applied to get the accurate localization. A multi-scale method for shape recognition is presented by Jalba et al. [19]. The method is based on two morphological scale-space representations and the hat-transform in scale space. They use maximum heights of the extrema of the curvature function as a shape descriptor.

In normal blob linking processes (to form a scale space tree), only spatial information of the blobs in consecutive scales is used. The main flaw within this process is that the blurring does not take into account with any other information on the image. For example, two different blobs with different colors will still be merged if they stay close together spatially.

In this paper, we propose an object detection method based on scale space by incorporating features of blobs into the scale space blob linking process as well as spatial information. We demonstrate in the paper that blobs are linked if their features are stable over scales, we get a better performance in the object detection. The evolution of linked blobs over scales using feature stability presents how stable image structures are in scale-space.

Section snippets

Scale space representation

In scale-space theory, a multi-scale representation of a two-dimension image, f(x, y), is defined by a convolution with the Gaussian kernel f(x, y, σ). The successive smoothing process generates a set of output images in various scales, σ. A scale parameter, σ, of the kernel is gradually increased many times to create a series of smoothed images. During the blurring process, less important details of the image are suppressed while prominent structures and features still remain. In other words, the

Numerical experiments

To demonstrate the robustness and generality of the algorithm we consider the stability of three features: the entropy, the average value of the gray level of the image and the standard deviation of the gray level. These three values represent the texture, the gray level and the gray level distribution, respectively. No systematic feature selection was performed. The feature selection was intuitive but supported by some standard pattern recognition schemes [20], [21], [22], [23], [24], [25],

Conclusion

We have proposed a novel object detection method based on scale-space endowed with feature stability. The algorithm is robust for detecting variable size and variable shape objects without a priori information of the object of interest. The shapes and sizes of multiple objects such as the flying birds or sunflowers are not uniform throughout the image but the algorithm can detect most of them.

The algorithm is flexible. The feature vectors can be extended or modified to suit particular

Acknowledgments

This project is funded by Guy’s and St. Thomas’ Hospital Special Trustees, UK and Thailand Toray Science Foundation (TTSF) and National Research University Project of Thailand Office of Higher Education Commission.

References (28)

  • K. Bowyer et al.

    Edge detectorevaluation using empirical roc curves

    Computer Vision and Image Understanding

    (2001)
  • A. Sopharak et al.

    Automatic detection of diabetic retinopathy exudates from non-dilated retinal images using mathematical morphology methods

    Computerized Medical Imaging and Graphics

    (2008)
  • S. Wang et al.

    Watershed-based textural image segmentation

    Intelligent Signal Processing and Communication Systems

    (2007)
  • D. Gatica-Perez et al.

    Extensive partition operators, gray-level connected operators, and region merging/classification segmentation algorithms: theoretical links

    IEEE Transactions on Image Processing

    (2001)
  • S. Manay et al.

    Anti-geometric diffusion for adaptive thresholding and fast segmentation

    IEEE Transactions on Image Processing

    (2003)
  • L.A. Forbes et al.

    Inconsistencies in edge detector evaluation

    Computer Vision and Pattern Recognition

    (2000)
  • S. Konishi et al.

    Statistical edge detection: learning and evaluating edge cues

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2003)
  • H. Gomez-Moreno, S. Maldonado-Bascon, F. Lopez-Ferreras, Edge detection in noisy images using the support vector...
  • T. Kawaguchi, D. Hidaka, M. Rizon, Robust extraction of eyes from face, in: Proceedings on the 15th International...
  • C. Damerval et al.

    Blob detection with wavelet maxima lines

    IEEE Signal Processing Letters

    (2007)
  • A. Ming H. Ma, A blob detector in color images, in: Proceedings of the 6th ACM International Conference on Image and...
  • C. Thou-Ho, L. Yu-Feng, C. Tsong-Yi, Intelligent vehicle counting method based on blob analysis in traffic...
  • R. Megret et al.

    Tracking scale-space blobs for video description

    IEEE Multimedia

    (2002)
  • A.P. Witkin, Scale-space filtering, in: Proc. 8th Int. Joint Conf. Art. Intell., 1983, pp....
  • Cited by (7)

    View all citing articles on Scopus
    View full text