Skip to main content
SpringerLink
Log in

Parsing Images into Regions, Curves, and Curve Groups

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper, we present an algorithm for parsing natural images into middle level vision representations—regions, curves, and curve groups (parallel curves and trees). This algorithm is targeted for an integrated solution to image segmentation and curve grouping through Bayesian inference. The paper makes the following contributions. (1) It adopts a layered (or 2.1D-sketch) representation integrating both region and curve models which compete to explain an input image. The curve layer occludes the region layer and curves observe a partial order occlusion relation. (2) A Markov chain search scheme Metropolized Gibbs Samplers (MGS) is studied. It consists of several pairs of reversible jumps to traverse the complex solution space. An MGS proposes the next state within the jump scope of the current state according to a conditional probability like a Gibbs sampler and then accepts the proposal with a Metropolis-Hastings step. This paper discusses systematic design strategies of devising reversible jumps for a complex inference task. (3) The proposal probability ratios in jumps are factorized into ratios of discriminative probabilities. The latter are computed in a bottom-up process, and they drive the Markov chain dynamics in a data-driven Markov chain Monte Carlo framework. We demonstrate the performance of the algorithm in experiments with a number of natural images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • August, J. and Zucker, S.W. 2003. Sketches with curvature: The Curve Indicator Random Field and Markov processes. IEEE Trans. on Pattern Analysis and Machine Intelligence, 25(4): 387–400.

    Article  Google Scholar 

  • Barbu, A. and Zhu, S.C. 2003. Graph partition by swendsen-wang cuts. In Proc. Int’l Conf. Comp. Vis., Nice, France.

  • Barbu, A. and Zhu, S.C. 2005. Cluster sampling and its applications in image analysis. Preprint 409 Department of Statistics, UCLA.

  • Bremaud, P. 1999. Markov Chains: Gibbs Fields, Monte Carlo Simulation and Queues. Springer. (chapter 6).

  • Bubley, R. and Dyer, M. 1997. Path coupling: A Technique for Proving Rapid Mixing in Markov Chains. In Proc. of 38th Annual IEEE Symposium on Foundations of Computer Science.

  • Candes, E.J. 1998. Ridgelets: Theory and applications. Ph.D. Dissertation, Yale University.

  • Carlsson, S. 1998. Sketch Based Coding of Grey Level Images, Signal Processing, Vol. 15.

  • Comaniciu, D. and Meer, P. 1999. Mean Shift Analysis and Applications. In Proc. Int’l Conf. Comp. Vis.

  • Cooper, C. and Frieze, A.M. 1999. Mixing Properties of the Swendsen-Wang Process on Classes of Graphs. Random Structures and Algorithms, Vol. 15, pp. 242–261.

    Article  MathSciNet  Google Scholar 

  • Cootes, T.F., Cooper, D., Taylor, C.J., and Graham, J. 1995. Active shape models–-Their training and application. Computer Vision and Image Understanding, 61(1):38–59.

    Article  Google Scholar 

  • Dellaert, F., Seitz, S., Thorpe, C., and Thrun, S. 2003. EM, MCMC, and Chain flipping for structure from motion with unknown correspondence. Machine Learning, No. 50, pp. 45–71.

    Article  Google Scholar 

  • Dick, A.R., Torr, P.H.S., and Cipolla, R. 2002. A Bayesian estimation of building shaping using MCMC. In Proc. of 6th European Conference on Computer Vision.

  • Forsyth, D.A. 2001. The joy of sampling. Int’l J. of Computer Vision, 41(1/2):109–134.

    Article  MATH  Google Scholar 

  • Geman, S. and Geman, D. 1984. Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images. IEEE Trans. PAMI, 6:721–741.

    Google Scholar 

  • Geman, D. and Jedynak, B. 1996. An Active Testing Model for Tracking Roads from Satellite Images. IEEE Trans. PAMI, 18(1).

  • Parida, L., Geiger, D., and Hummel, R. 1998. Junctions: Detection, Classification, and Reconstruction. IEEE Trans. PAMI, 20(7).

  • Green, P.J. 1995. Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination. Biometrika, 82(4):711–732.

    Article  MATH  MathSciNet  Google Scholar 

  • Grenander, U. and Miller, M.I. 1994. Representations of Knowledge in Complex Systems. J. of the Royal Stat. Soc. Series B, 56(4):549–603.

    MathSciNet  Google Scholar 

  • Han, F. and Zhu, S.C. 2003. Bayesian Reconstruction of 3D Shapes and Scenes from a Single Image. In Proc. of Int’l Workshop on High Level Knowledge in 3D Modeling and Motion, Nice, France.

  • Hastings, W.K. 1970. Monte Carlo Sampling Methods Using Markov Chains and Their Applications. Biometrika, 57:97–109.

    Article  MATH  Google Scholar 

  • Isard, M. and Blake, A. 1996. Contour Tracking by Stochastic Propagation of Conditional Density. In Proc. of 4th European Conference on Computer Vision.

  • Kaess, M., Zboinske, R., and Dellaert, F. 2004. Multiview Reconstruction of Piecewise Smooth Subdivision Curves with a variable Number of Control Points. Proc. of 8th European Conference on Computer Vision.

  • Khan, Z., Balch, T., and Dellaert, F. 2004. An MCMC-based Particle Filter for Tracking Multiple Interacting Targets. In Proc. of 8th European Conference on Computer Vision.

  • Kass, M., Witkin, A., and Terzopoulos, D. 1988. Snakes: Active contour models. Int’l J. Computer Vision, 1(4):321–332.

    Article  Google Scholar 

  • Lanterman, A.D. 2001. Jump-Diffusion Algorithm for Multiple Target Recognition using Laser Radar Range Data. Optical Engineering, 40(8):1724–1728.

    Article  Google Scholar 

  • Lee, M.W. and Cohen, I. 2004. Proposal maps driven MCMC for estimating human body pose. In Proc. of CVPR, Washington.

  • Liu, J.S. 2001. Monte Carlo Strategies in Scientific Computing. Springer-Verlag: NY INC.

    Google Scholar 

  • Maciuca, R. and Zhu, S.C. 2005. First-hitting-time Analysis of Independence Metropolis Sampler. Journal of Theoretical Probability(Accepted).

  • Malik, J., Belongie, S., Leung, T., and Shi, J. 2001. Contour and Texture Analysis for Image Segmentation. Int’l J. Comp. Vis.

  • Martin, D., Fowlkes, C., Tal, D., and Malik, J. 2001. A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. In Proc. Int’l Conf. Computer Vision, 2:416–423.

    Google Scholar 

  • Mallat, S.G. and Zhang, Z. 1993. Matching Pursuits With Time-frequency Dictionaries. IEEE Tranc. on Signal Processing, 41(12).

  • Metropolis, N., Rosenbluth, M.N., Rosenbluth, A.W., Teller, A.H., and Teller, E. 1953. Equations of State Calculations by Fast Computing Machines. J. Chem. Phys., 21:1087–1092.

    Article  Google Scholar 

  • Nitzberg, M. and Mumford, D.B. 1990. The 2.1D Sketch. In Proc. Int’l Conf. Comp. Vis., pp.138–144.

  • Puskun, P.H. 1973. Optimum Monte Carlo sampling using Markov chains. Biometrika, 60(3):607–612.

    Article  MathSciNet  Google Scholar 

  • Schapire, R.E. 2000. The boosting approach to machine learning: An overview. MSRI Workshop on Nonlinear Estimation and Classification.

  • Skiena, S. 1990. Partial orders. Implementing Discrete Mathematics: Combinatorics and Graph Theory with Mathematica. Addison-Wesley: MA, pp. 203–209.

    Google Scholar 

  • Shi, J. and Malik, J. 2000. Normalized Cuts and Image Segmentation. IEEE Trans. PAMI, 22(8).

  • Srivastava, A, Grenander, U., Jensen, G.R., and Miller, M.I. 2002. Jump-diffusion Markov Processes on Orthogonal Groups for Object Post Estimation. J. of Statistical Planning and Inference, 103:15-37.

    Article  MathSciNet  Google Scholar 

  • Swendsen, R.H. and Wang, J.S. 1987. Nonuniversal Critical Dynamics in Monte Carlo Simulation. Physical Review Letters, 58(2):86–88.

    Article  Google Scholar 

  • Tu, Z. and Zhu, S.C. 2002. Image Segmentation by Data-Driven Markov Chain Monte Carlo. IEEE Trans. PAMI, 24(5).

  • Tu, Z. and Zhu, S.C. 2002. Parsing Images into Region and Curve Processes. In Proc. of 7th European Conference on Computer Vision, Copenhagen.

  • Tu, Z., Chen, X., Yuille, A.L., and Zhu, S.C. 2003. Image parsing: Segmentation,Detection and Recognition. In Proc. Int’l Conf. Comp. Vis., Nice, France.

  • Wang, J. and Adelson, E. 1994. Representing Moving Images with Layers. IEEE Trans. on Image Processing, 6(3).

  • Zhao, T. and Nevita, R. 2004. Tracking Multiple Humans in Crowded Environment. In Proc of CVPR, Washington.

  • Zhu, S.C. and Yuille, A.L. 1996. Region competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multi-band Image Segmentation. In IEEE Trans. PAMI, 18(9):884–900.

    Google Scholar 

  • Zhu, S.C., Zhang, R., and Tu, Z.W. 2000. Integrating Top-down/Bottom-up for Object Recognition by Data Driven Markov Chain Monte Carlo. In Proc. of Int’l Conf. on Computer Vision and Pattern Recognition, Hilton Head, SC.

  • Zimmer, C., Labruyere, E., Meas-Yedid, V., Guillen, N., Olivo-Marin, J.C. 2002. Segmentation and Tracking of Migrating Cells in Videomicroscopy With Parametric Active Contours. IEEE Trans. Medical Imaging, 21(10).

Download references

Author information

Authors and Affiliations

  1. Lab of Neuro Imaging, Department of Neurology, University of California, Los Angeles, CA, 90095

    Zhuowen Tu

  2. Departments of Statistics and Computer Science, University of California, Los Angeles, CA, 90095

    Song-Chun Zhu

Authors
  1. Zhuowen Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Song-Chun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhuowen Tu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tu, Z., Zhu, SC. Parsing Images into Regions, Curves, and Curve Groups. Int J Comput Vision 69, 223–249 (2006). https://doi.org/10.1007/s11263-006-6995-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-006-6995-9

Keywords

Search

Navigation