Weak boundary preserved superpixel segmentation based on directed graph clustering

Highlights

• We analysis the relationship between image boundary extraction and pixel K-NN-graph.

• A superpixel segmentation methods via directed graph clustering is proposed.

• We construct the problem of stitching trivial regions by an integer programming-based region merging.

Abstract

Weak boundary preservation is a challenge for superpixel segmentation. Existing methods that measure pixels’ similarity based on pairwise distance could not efficiently describe relationship among high-dimensional image data, which plays key roles in extraction for image boundaries. In this paper, we present a new directed graph clustering (DGC)-based superpixel segmentation method via K-nearest-neighbor (K-NN) graph and distance information. It can efficiently deal with segmentation for the weak boundary of complex and irregular object. The basic idea is motivated by following observations: compared with smooth region, image boundary points have much lower pixel density and directed connectivity in local region. Based on K-NN directed graph, we introduce indegree and outdegree to describe above observation, which are the foundation to evaluate pixels’ similarity. Then, a patch-based segmentation generates superpixel borders by even overlapping regions. Finally, we solve an integer programming to stitch small noise regions into final superpixels. Experimental results on two benchmarks demonstrate that our method outperforms the state-of-the-arts.

Introduction

Superpixel segmentation has been widely used in image and video segmentation [[1], [2]], saliency detection [[3], [4]], object proposal extraction [5], biomedical image analysis [6], object recognition [7] and target tracking [8], which benefits from the region consistency, cluster perceptibility and computation efficiency.

Generally, superpixel segmentation can be coarsely divided into two classes: (1) cut edge searching on graph structure and (2) seed region growing. The first class models image pixels as graph structure and seeks cut edges with minimum cost to disjoint the graph into subgraph as individual superpixel, such as Ncut [9], FH [10], LatticCut [[11], [12]] and RegularSeg [13]. Those methods introduce adaptive similarity predicate [10] or regularity constraint [[9], [11], [12], [13]] to produce consistent superpixel segmentation. The second class grows superpixel from some initial seeds. For example, SLIC [14] and HOSup [15] assign pixels’ label of adjacent region by their nearest seeds. TP [16] expands the superpixel region from initial seeds until satisfies some stop conditions. Seed expansion-based superpixel segmentation obtain efficient computation performance due to their sophistical energy models. Although those two class superpixel segmentation methods produce simple and efficient calculation, they all calculate the affinity based on pairwise distance of pixels in feature space.

Pairwise distance-based measure can easily describe the dissimilarity of two pixels from obviously difference regions. However, it is difficult to reflect the fundamental difference for two pixels in image weak boundary. Intuitively, image boundary points have lower density distribution compared with smooth regions. Meanwhile, boundary points have different connectivity with non-boundary points based on K-NN directed graph. Existing methods for graph-based superpixel segmentation ignore these two key cues to describe pixels’ similarity, which result into low boundary adherence rate, such as results of FH [10] and NC [9] for fish image in Fig. 1. Seed expansion-based methods, i.e., TP [16], SNIC [17] will yield large region leakage for some long and narrow regions due to lack of seed location on fine structure by regular grids location, such as segmentation for branch in crow image as shown in Fig. 1.

In this paper, we focus on the preservation for image weak boundary by combining pixels’ density distribution and its local connectivity in K-NN directed graph, inspired by DGC [18]. Weak boundary preservation is the ability to extract the object boundary with low contrast, which is due to the similarity of foreground and background, illumination and blur. For example, the fish’s lip in Fig. 1 has the similar appearance with the surroundings and its border is blur. The state-of-the-art methods use the pairwise distance to depict the pixel similarity, which results into the inaccurate weak boundary extraction. In contrast, the proposed method define the similarity by some corresponding neighbor pixels based on the K-NN adjacent graph. It is more accurate to distinguish the boundary pixels, such as our segmentation result of fish in Fig. 1. Although the fish’s lip has the similar appearance with background, our method is more robust to extract the weak boundary by considering the similarity of pixel to some its neighbor pixels.

In DGC [18], Zhang et al. introduce indegree and outdegree to describe the spatial structure of irregular cluster, which is suitable to segment irregular and low contrast object boundary in complex and noise background. We utilize indegree and outdegree to construct graph structure to describe the relationship between pixel and its neighbors. Intuitively, the external points of object boundary have lower indegree compared with object smooth regions, which reflects much lower density distribution than its nearest pixels. Meanwhile, those boundary points have difference local connectivity with smooth region points, which can be captured by pixel’s outdegree. That is to say, pixels with lower indegree tend to locate in image boundary. Moreover, those points tend to be some region boundary if they are with higher outdegree. Utilizing these two indications, it is easy to discriminate those points and capture the topology structure of fine object.

Specifically, we firstly introduce DGC-based boundary extraction. Then patch-based superpixel segmentation has been proposed to partition the image pixel into consistent regions. Finally, we model region stitching process as an integer programming problem. We compare our method with several state-of-the-arts on BSD500 benchmark and shows that it outperforms other methods. The result proves that our method can more efficiently preserves the image weak boundary.

Our main contributions are:

• We first analysis the relationship between image boundary extraction and pixel K-NN graph.

• Then we propose superpixel segmentation methods via directed graph clustering.

• We construct the problem of stitching trivial regions by an integer programming-based region merging.