Bisection approach for pixel labelling problem
Introduction
Many early vision tasks require assigning labels to pixels based on the observed images. The labels can denote quantities such as gray, disparity and so on. The label fields can be elegantly expressed as Markov random fields (MRFs). Then, the pixel labelling problem can be formulated as maximum a posterior estimation of the Markov random fields (MAP-MRF) in a Bayesian framework, it results in an energy minimization problem [1], [2].
Simulated annealing is easy to implement and can optimize an arbitrary energy function. Theoretically, if annealing's temperature parameter is sufficiently low, it should eventually find the global minimum. Nevertheless, it gives results far from global minimum [3]. Moreover, it requires exponential time and it is very slow. Iterated conditional modes (ICM) adopts a greedy technique to find a local minimum [4]. It starts with an initial labelling and updates the labelling until a minimum is reached. Since only one pixel can change its label at each step, the results are extremely sensitive to the initial labelling and their quality is usually low.
Recently, people have developed graph cut for energy minimization. Graph cut methods [5], [6], [7] construct a weighted graph such that the minimum cut corresponds to a configuration minimizing the energy function. The minimum cut can be found efficiently by max flow algorithms such as the “push-relabel” [8]. When there are two labels involved, the global minimum can be found by a single minimum cut computation [3]. Usually, there are more than two labels involved, for example, image gray and disparity have many levels. Then, it is necessary to solve a multiway minimum cut problem. Unfortunately, multiway minimum cut problem is NP-hard [5].
Boykov et al. proposed and algorithm to compute an approximate optimal solution [9]. These algorithms start with an initial labelling and update it iteratively. In each iteration, they perform a swap move for every pair of labels and an expansion move for every label, respectively, to minimize the energy. There are only two possible labels involved in the moves, and the optimal moves are found via standard minimum cut algorithm. Since a large number of pixels are allowed to change their labels simultaneously, they find local minimum with respect to very large moves and produce results with high quality [10]. The complexities of algorithm and algorithm are and , respectively, where k is the number of iterations and n is the size of the label set.
Moreover, people have developed belief propagation algorithms for energy minimization. Belief propagation algorithms [11], [12] adopt a message propagating mechanism to assist pixel labelling. Each pixel receives messages from neighboring pixels. The message propagation is iterated until its convergence. Each pixel accepts the label that supports its maximal belief. Since the graphical model for pixel labelling consists of many loops, the belief propagation algorithm can eventually find an approximate solution. Belief propagation algorithms produce results with comparable quality as that of graph cut algorithms [10], [13], [14]. The complexity of the algorithm is , where p is the number of pixels in the image. The complexities of belief propagation and graph cut algorithms are either linear or quadric in n, they are not efficient enough to support realtime applications.
This paper proposes an efficient method for pixel labelling. We treat labels as indicators of categories, i.e. pixels with same label belong to same category while pixels with different labels belong to different categories. Then, we formulate the pixel labelling problem as a classification problem, and classify pixels by a series of two-category classification. In other words, a label set instead of a determinate label is assigned to each pixel and it is shrunk step by step until the label set consists of only one label. It is not necessary to test every label or every pair of labels as that of algorithm and algorithm. Using bisection technique, determinate label can be achieved within steps. As a result, the whole process consists only steps, and the complexity is reduced to .
This rest of this paper is organized as follows. Section 2 formulates pixel labelling problem as a classification problem and Section 3 shows how to solve it using graph cut algorithm. A bitwise algorithm for pixel labelling is proposed in Section 4. Experimental results are presented in Section 5 and conclusions are drawn in Section 6.
Section snippets
Pixel labelling problem
A MRF is a set of random variables adhering to a field of pixels ; each random variable can take a value in some label set . It has the local characteristics: the value on pixel p only depends on the value of its neighboring pixels ; is the set of neighbors of p and is a neighborhood system on the field.
Pixel labelling problem is a common task in computer vision. It needs to assign one label l in L to each pixel p in P such that the
Energy minimization via graph cut
Kolmogorov and Zabih presented a theoretic foundation on what energy can be minimized via graph cut [17]. The main result is that: Theorem 1 Let E be a function of m binary variables Then E can be minimized via graph cut if and only if each term satisfies the inequality
Following Assumption 1 and the classification schema described in Section 2.2, we have . Moreover, we have: Theorem 2
Bitwise algorithm
As shown in Section 2.2.3, each step of the classification corresponds to one layer on the decision tree and determines one bit of the labels. Without loss of generality, let us assume that the labels have 8 bits and focus on the first step of the classification. As shown in Fig. 4, the highest bit of is 0 and 1 when and , respectively. Since represents the decision for p, we can use to set the highest bit of . The rest bits of the labels can also be determined in the
Experimental results
Stereo matching and image restoration can be formulated as pixel labelling problem. Graph cut and belief propagation are the state of the art algorithms for pixel labelling. Since bitwise algorithm is built on graph cut algorithm, it is reasonable to compare bitwise algorithm with graph cut algorithms. and algorithms have nearly the same performance while as algorithm is more efficient than . Therefore, this section presents results on both stereo
Conclusion
This paper has demonstrated that pixel labelling problem can be expressed as a classification problem, and then formulated it as a series of two-category classification. In contrast with existing techniques, which assign a determinate label to each pixel, our new approach assigns a set of labels to each pixel and bisect the label set step by step until it contains only one label. Each step is formulated as a bi-label labelling problem and can be solved by standard graph cut method. Based on
Acknowledgements
The authors would like to thank the anonymous reviewers for their critical and constructive comments and suggestions.
About the Author—DENGFENG CHAI received Bachelor's degree from Wuhan University in 1997, then received Master's degree from State Key Lab of Information Engineering in surveying, mapping and remote sensing at Wuhan University in 2000, and then received Doctor's degree from State Key Lab of CAD&CG at Zhejiang University in 2006. Since 2000, he serves as an assistant professor at College of Science in Zhejiang University. He has done research in computer vision and pattern recognition, and has
References (22)
- et al.
Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images
IEEE Transactions on Pattern Analysis and Machine Intelligence
(1984) Markov Random Field Modeling in Computer Vision
(1995)- et al.
Exact maximum a posteriori estimation for binary images
Journal of the Royal Statistical Society, Series B
(1989) On the statistical analysis of dirty pictures (with discussion)
Journal of the Royal Statistical Society, Series B
(1986)- et al.
Markov random fields with efficient approximations
- et al.
A maximum-flow formulation of the n-camera stereo correspondence problem
- H. Ishikawa, Global optimization using embedded graphs, Ph.D. Thesis, New York University,...
- et al.
A new approach to the maximum flow problem
Journal of the Association for Computing Machinery
(1988) - et al.
Fast approximate energy minimization via graph cuts
IEEE Transactions on Pattern Analysis and Machine Intelligence
(2001) - et al.
A taxonomy and evaluation of dense two-frame stereo correspondence algorithms
International Journal of Computer Vision
(2002)
Learning low-level vision
International Journal of Computer Vision
Cited by (2)
SQL: Superpixels via quaternary labeling
2019, Pattern RecognitionLarge margin filtering
2012, IEEE Transactions on Signal Processing
About the Author—DENGFENG CHAI received Bachelor's degree from Wuhan University in 1997, then received Master's degree from State Key Lab of Information Engineering in surveying, mapping and remote sensing at Wuhan University in 2000, and then received Doctor's degree from State Key Lab of CAD&CG at Zhejiang University in 2006. Since 2000, he serves as an assistant professor at College of Science in Zhejiang University. He has done research in computer vision and pattern recognition, and has published many papers on ICCV, ACCV and other related conferences.
About the Author—HONGWEI LIN received his B.Sc. from Department of Applied Mathematics at Zhejiang University in 1996, and Ph.D. from Department of Mathematics at Zhejiang University in 2004. He worked as a communication engineer from 1996 to 1999. Now, he is an associate professor in State Key Laboratory of CAD&CG, Zhejiang University. His current research interests are in computer aided geometric design, computer graphics, and computer vision. He has published over twenty papers on these areas.
About the Author—QUNSHENG PENG graduated from Beijing Mechanical College in 1970 and received a Ph.D. from the Department of Computing Studies, University of East Anglia, UK in 1983. He is a Professor of computer graphics at Zhejiang University. His research interests include realistic image synthesis, computer animation, scientific data visualization, virtual reality, bio-molecule modeling. Prof. Peng serves currently as a member of the editorial boards of several international and Chinese journals.