Reduced set density estimator for object segmentation based on shape probabilistic representation

Abstract

In this paper, a nonparametric statistical shape model based on shape probabilistic representation is proposed for object segmentation. Given a set of training shapes, Cremers et al.’s probabilistic method is adopted to represent the shape, and then principal components analysis (PCA) on shape probabilistic representation is computed to capture the variation of the training shapes. To encode complex shape variation in training set, reduced set density estimator is used to model nonlinear shape distributions in a finite-dimensional subspace. This statistical shape prior is integrated to convex segmentation functional to guide the evolving contour to the object of interest. In addition, in contrast to the commonly used signed distance functions, PCA on shape probabilistic representation needs less number of eigenmodes to capture certain details of the training shapes. Numerical experiments show promising results and the potential of the model for object segmentation.

Introduction

Object segmentation is a fundamental task in image processing and computer vision. Its essential goal is to extract desired objects from the given images. Since the object and background may exhibit very similar intensity characteristics in numerous real-world applications, it is normally not enough to only use the low-level information of the images, such as intensity, color or texture for segmentation, especially when misleading information due to occlusion, clutter and noises exist in the input images. This naturally leads to a need for integrating prior knowledge such as shape information into the segmentation process in order to improve segmentation results. In this paper, by assuming that prior knowledge given by a set of training shapes of expected objects, we focus on the problem of how to exploit such shape priors for object segmentation.

Level set methods were introduced by Osher and Sethian [1]. Since such methods allow implicit representation of the evolving object boundary and automatic changes of its topology, level set methods have become increasingly popular for image segmentation [2], [3]. Recently, to segment images of low quality or with missing data, level set based variational approaches have gained significant attention toward the integration of shape prior into the image segmentation processes [4], [5], [6], [7], [8], [9], [10], [11], [12], [13]. Almost all these works can be considered as a linear combination of two terms: a data-driven term and a shape constraint term. Geometric active contours model [14] and Chan-Vese’s model [15] have become two popular data-driven terms to guide the motion of the active contour. There are two ways to define the shape constraint term. One is commonly defined by an explicit dissimilarity measure between the evolving contour and a given prior contour, and the other is to estimate a statistical distribution from training shapes to guide the evolving contour to the most likely shape of the estimated distribution. Given a set of training shapes, one may impose simple or more complicated distribution functions such as uniform distribution [7], Gaussian distribution [16], or non-parametric estimator [17] to improve segmentation results in the presence of noise or occlusion. In applications, the distribution of training shapes is generally not uniform distribution or Gaussian distribution due to a large variability of shape. Kernel density estimation (KDE) is an efficient approach to model nonlinear distributions of training shapes [12], [13]. In this technique, the density function is estimated by a sum of kernel functions. The kernel number is equal to the size of the training data. When the training data set is very large, the KDE suffers from high computational cost and becomes intractable for subsequent use (e.g., in a real-time applications). Reduced set density estimator (RSDE) was proposed by Girolami and He [18] to solve the above problem by providing a kernel density estimator which employs a small subset of the available data sample to provide similar levels of performance.

Shape is represented implicitly by signed distance function (SDF), and can be easily integrated into level set variational methods as a shape constraint term. These representations have gained much popularity in recent years [4], [5], [6], [7], [8], [9], [10], [11], [12], [13]. The idea is to represent the shape contour C by embedding it in a higher dimension level set functional ϕ, as follows:

$$\varphi (x)=\left\{\begin{array}{cc}\mathit{Dist}(x\text{,}C)\text{,}\hfill & x\in \mathit{in}(C)\hfill \\ 0\text{,}\hfill & x\in C\hfill \\ -\mathit{Dist}(x\text{,}C)\text{,}\hfill & x\in \mathit{out}(C)\hfill \end{array}\right.\text{,}$$

where Dist(x, C) denotes the Euclidean distance from x to the closet point on C, and out(C) and in(C) represent the regions outside and inside of the contour C, respectively. The contour C can be reconstructed from such representation by taking its zero level set C = {x|ϕ(x) = 0}. Hence, any shape in the plane corresponds to a unique SDF. This shape representation is consistent with the level set framework, and has its advantages since parameterization free and easy handling of topological changes. However, the use of principal component analysis (PCA) on a set of SDF embedding a set of sample shapes has two drawbacks:

• The space of SDF is not a linear space, e.g., the mean shape and linear combinations of sample shapes are typically no longer SDF. Most existing works only consider very similar shape priors.

• While the first few principal components are used to capture the most variation on the space of SDF, they will not necessarily capture the variation on the space of the embedded shape contours. Therefore, in contrast to PCA on explicit shape contours, PCA on SDF need to include a larger number of eigenmodes in order to capture certain details of the sample shapes.

Recently, there has been significant research exploring methods to solve these non-convex problems by using convex relaxation methods [19], [20], [21], [22]. In [20], Cremers et al. proposed a shape probabilistic representation (SPR) by relaxing the binary constraint and allowing the binary function to take on values in the interval [0, 1], defined as a mapping

$$q=\Omega \to [0\text{,}1]\text{,}$$

that assigns to every pixel x of the shape domain $\Omega \subset {\mathbb{R}}^{\mathbb{2}}$ the probability that this pixel is inside the given shape. In traditional definition of shape, pixels are part of the shape, and only take values 1 (members) or 0 (non-members). It can be described as q:Ω → {0, 1}. Based on the probabilistic definitions, it is easy to get the shape region of the object $(q{)}_{\tau }=\{x|q(x)⩾\tau \}$ and the background of image $(q{)}_{\tau }^C=1-(q{)}_{\tau }$ by selecting a τ ϵ [0, 1]. In the experiment, τ is chosen as 0.5. It was shown that the space Q of all probabilistic shapes forms a convex set, and the space spanned by a few training shapes χ = {q₁, q₂, ⋯q_N} forms a convex subset. Arbitrary convex combinations of the set again correspond to a valid shape. For example, the mean $\mu (x)=\frac{1}{\text{N}}{\sum }_{i=1}^N{q}_i(x)\text{,}\mu (x)\in [0\text{,}1]$ is a function which assigns to each point x ϵ Ω the average of all probabilities (Fig. 1). This shape probabilistic representation leads to convex segmentation functional on convex shape spaces.

In this paper, we are building up on the above developments and propose two contributions in order to overcome the discussed limitations:

• We use probabilistic representation to model the shape prior, and then compute PCA on shape probabilistic representation to capture the variation of the training shapes. In contrast to the commonly used signed distance functions, PCA on shape probabilistic representation needs less number of eigenmodes to capture certain details of the training shapes.

• RSDE is used to learn the shape prior information in the low-dimensional subspace, and then is integrated to convex segmentation functional to restrict the segmenting contour to a manifold of training shapes during the segmentation process. In contrast to existing statistic approaches of shape priors (uniform distribution, Gaussian distribution, KDE), RSDE can provide a high accuracy estimator of a probability density function which employs a small percentage of the available data sample.

The remainder of this paper is organized as follows. In Section 2, the shape prior based level set segmentation and RSDE are described. RSDE based on shape probabilistic representation for variational segmentation is presented in Section 3. The implementation procedure is proposed in Section 4 to minimize the energy. Experimental results are provided in Section 5, and the conclusions are given in Section 6.