A graph based superpixel generation algorithm

Abstract

In recent years, superpixels have become a prevailing tool in computer vision and many methods have been proposed. However, due to the problems such as high time complexity, low object boundary adherence and irregular shape, only a few methods are widely used. To improve these issues, we propose a novel general superpixel segmentation method called minstpixel, which relies on energy functional minimization. Minstpixel introduces an energy functional based on minimal spanning tree and designs a strategy to gain the global optimum. It never needs sophisticated optimization scheme, complicated mathematical deduction or fussy iteration process. At the same time, the time complexity of minstpixel is approximately linear with respect to the number of image pixels. The benchmark on Berkeley segmentation database shows that minstpixel could rival state-of-the-art in every aspect.

Appendix

This section prepares an proof for the global optimality of the algorithm 1. Proposition 1 and proposition 2 demonstrate that the segmentation $\mathcal {S}$ generated by algorithm 1 is a valid superpixel segmentation. Proposition 3 demonstrates that algorithm 1 will always converge to the optimum of (4).

Proposition 1

There is one and only one seed point in each connected component produced by algorithm 1.

Proof

On the one hand, the number of seed points in each component is less than 2 (1 or 0) according to the step 3 of algorithm 1. On the other hand, we suppose that in segmentation $\mathcal {S}$ there exists one connected component c_i without seed points, then the number of connected components of segmentation is greater than k. There is a contradiction between assumption and termination condition of algorithm 1. So, the number of seed points in each component cannot be other except 1.□

Proposition 2

Each connected component produced by algorithm 1 is a minimum spanning tree.

Proof

Let $G(\mathcal {V},\mathcal {E})$ be an input graph, $G(\mathcal {V},\mathcal {A})$ is the segmentation result produced by algorithm 1, c_i(v_i, a_i) is the ith connected component of graph $(\mathcal {V},\mathcal {A})$ i.e. $\bigcup _{i} \mathbf {v}_{i}=\mathcal {V}$ and $\bigcup _{i} \mathbf {a}_{i}$ = $\mathcal {A}$. Obviously, if we remove an edge e ∉ $\mathcal {A}$ from graph $(\mathcal {V},\mathcal {E})$, then the segmentation result $(\mathcal {V},\mathcal {A})$ produced by algorithm 1 remains the same. Now, we build a graph $(\mathcal {V}^{\prime },\mathcal {E}^{\prime })$ from graph $(\mathcal {V},\mathcal {E})$ by removing all edges e(v₁, v₂) satisfying v₁ ∈ c_i, v₂ ∈ c_j and c_i≠c_j. Suppose that $(\mathcal {V}^{\prime },\mathcal {A}^{\prime })$ is the segmentation result produced by algorithm 1 on graph $(\mathcal {V}^{\prime },\mathcal {E}^{\prime })$, $c^{\prime }_{i}(\mathbf {v}^{\prime }_{i},\mathbf {a}^{\prime }_{i})$ is the ith connected component of graph $(\mathcal {V}^{\prime },\mathcal {A}^{\prime })$ i.e. $\bigcup _{i} \mathbf {v}^{\prime }_{i}$ = $\mathcal {V}^{\prime }$ and $\bigcup _{i} \mathbf {a}^{\prime }_{i}=\mathcal {A}^{\prime }$. Graph $(\mathcal {V},\mathcal {A})$ and graph $(\mathcal {V}^{\prime },\mathcal {A}^{\prime })$ is identical because all removed edges don’t belong to $\mathcal {A}$, and then each c_i is identical to the corresponding $c^{\prime }_{i}$. On the other side, the process of algorithm 1 for graph $(\mathcal {V}^{\prime },\mathcal {E}^{\prime })$ is perfectly equivalent to the process of Kruskal algorithm for every $c^{\prime }_{i}$ [40]. So, each connected component $c^{\prime }_{i}$ is a minimum spanning tree, and then each c_i is also a minimum spanning tree.□

Proposition 3

The algorithm 1 can converge to the global minimum point.

Proof

We define proposition P : If $\mathcal {A}$ is the set of edges chosen at any stage of the algorithm 1, then there are some optimal segmentations that contain $\mathcal {A}$. Clearly, proposition 3 is equivalent to proposition P. We show that proposition P is true by induction.

• 1. Clearly, P is true at the beginning, when $\mathcal {A}$ is empty: any optimal segmentation will do, and there exists one because a weighted connected graph always has some segmentation satisfying the constraint condition of energy functional (4).

• 2. Now assume that P is true for a non-final edge set $\mathcal {A}$ and let $\mathcal {S}$ be an optimal segmentation that contains $\mathcal {A}$. If the next chosen edge e is also in $\mathcal {S}$, then P is true for $\mathcal {A} + e$. Otherwise, $\mathcal {S} +e$ has a cycle C or leads to a superpixel which contains two seed points p₁ and p₂.

  • If, $\mathcal {S} + e$ has a cycle C and there is another edge f that is in C but not $\mathcal {A}$. (If there were no such edge f , then e could not have been added to $\mathcal {A}$, since doing so would have created the cycle C.) Then $\mathcal {S} - f + e$ is a valid superpixel segmentation, and it has the same weight as $\mathcal {S}$, since $\mathcal {S}$ has minimum weight and the weight of f cannot be less than the weight of e, otherwise the algorithm 1 would have chosen f instead of e. So $\mathcal {S} - f + e$ is a minimum spanning tree containing $\mathcal {A} + e$ and again, P holds.

  • If, $\mathcal {S} + e$ leads to a superpixel contains two seed points p₁ and p₂. In other words, there is a path between p₁ and p₂ and there is another edge f that is in path p₁p₂ but not $\mathcal {A}$. Then $\mathcal {S} - f + e$ is a valid superpixel segmentation, and it has the same weight as $\mathcal {S}$, since $\mathcal {S}$ has minimum weight and the weight of f cannot be less than the weight of e, otherwise the algorithm 1 would have chosen f instead of e. So $\mathcal {S} - f + e$ is a minimum spanning tree containing $\mathcal {A} + e$ and again, P holds.

• 3. Therefore, by the principle of induction, P holds when $\mathcal {A}$ has become a valid segmentation, which is possible only if $\mathcal {A}$ is a optimal superpixel segmentation itself.

□