Skip to main content
SpringerLink
Log in

The Joy of Sampling

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

A standard method for handling Bayesian models is to use Markov chain Monte Carlo methods to draw samples from the posterior. We demonstrate this method on two core problems in computer vision—structure from motion and colour constancy. These examples illustrate a samplers producing useful representations for very large problems. We demonstrate that the sampled representations are trustworthy, using consistency checks in the experimental design. The sampling solution to structure from motion is strictly better than the factorisation approach, because: it reports uncertainty on structure and position measurements in a direct way; it can identify tracking errors; and its estimates of covariance in marginal point position are reliable. Our colour constancy solution is strictly better than competing approaches, because: it reports uncertainty on surface colour and illuminant measurements in a direct way; it incorporates all available constraints on surface reflectance and on illumination in a direct way; and it integrates a spatial model of reflectance and illumination distribution with a rendering model in a natural way. One advantage of a sampled representation is that it can be resampled to take into account other information. We demonstrate the effect of knowing that, in our colour constancy example, a surface viewed in two different images is in fact the same object. We conclude with a general discussion of the strengths and weaknesses of the sampling paradigm as a tool for computer vision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Amit, Y., Grenander, U., and Piccioni, M. 1991. Structural image restoration through deformable templates. J. Am. Statist. Ass., 86:376-387.

    Google Scholar 

  • Beardsley, P.A., Zisserman, A.P., and Murray, D.W. 1997. Sequential updating of projective and affine structure from motion. Int. J. Computer Vision, 23(3):235-259.

    Google Scholar 

  • Besag, J., Green, P., Higdon, D., and Mengersen, K. 1995. Bayesian computation and stochastic systems. Statistical Science, 10(1): 3-41.

    Google Scholar 

  • Binford, T.O. and Levitt, T.S. 1994. Model-based recognition of objects in complex scenes. In Image Understanding Workshop, pp. 149-155.

  • Blake, A. and Isard, M. 1998. Condensation—conditional density propagation for visual tracking. Int. J. Computer Vision, 29(1): 5-28.

    Google Scholar 

  • Blake, A. and Zisserman, A. 1987. Visual Reconstruction. Cambridge, MA: MIT Press.

    Google Scholar 

  • Brainard, D.H. and Freeman, W.T. 1997. Bayesian colour constancy. J. Opt. Soc. Am.-A, 14:1393-1411.

    Google Scholar 

  • Buchsbaum, G. 1980. A spatial processor model for object colour perception. J. Franklin Inst., 310:1-26.

    Google Scholar 

  • Carlin, B.P. and Louis, T.A. 1996. Bayes and Empirical Bayes Methods for Data Analysis. Chapman and Hall.

  • Carpenter, J., Clifford, P., and Fearnhead, P. 1999. Improved particle filter for non-linear problems. IEEE Proc. Radar, Sonar and Navigation, 146(1):2-7.

    Google Scholar 

  • Chou, P.B. and Brown, C.M. 1990. The theory and practice of bayesian image labeling. Int. J. Computer Vision, 4(3):185-210.

    Google Scholar 

  • Collins, N.E., Englese, R.W., and Golden, B.L. 1988. Simulated annealing—an annotated bibliography. Technical report, University of Maryland at College Park, College of Business and Management.

  • Costeira, J.P. and Kanade, T. 1998. A multibody factorisation method for independently moving objects. Int. J. Computer Vision, 29(3):159-180.

    Google Scholar 

  • Debevec, P.E., Taylor, C.J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In '96, pp. 11-20.

  • Dellaert, F., Seitz, S., Thorpe, C., and Thrun, S. 2000. Structure from motion without correspondence. In IEEE Conf. on Computer Vision and Pattern Recognition, pp. 557-564.

  • Doucet, A., De Freitas, N., and Gordon, N. 2001. Sequential Monte Carlo Methods in Practice. Springer-Verlag, New York.

    Google Scholar 

  • Duane, S., Kennedy, A.D., Pendleton, B.J., and Roweth, D. 1987. Hybrid monte carlo. Physics Letters B, 195:216-22.

    Google Scholar 

  • Evans, M. and Swartz, T. 2000. Approximating Integrals via Monte Carlo and Deterministic Methods. Oxford University Press: New York.

    Google Scholar 

  • Faugeras, O. D. and Robert, L. 1996. What can two images tell us about a third one? Int. J. Computer Vision, 18(1):5-19.

    Google Scholar 

  • Faugeras, O., Robert, L., Laveau, S., Csurka, G., Zeller, C., Gauclin, C., and Zoghlami, I. 1998. 3d reconstruction of urban scenes from image sequences. Computer Vision and Image Understanding, 69(3):292-309.

    Google Scholar 

  • Finlayson, G. Colour in Perspective. 1996. IEEE T. pattern Analysis and machine Intelligence, 18:1034-1038.

    Google Scholar 

  • Fischler, M.A. and Bolles, R.C. 1981. Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography. Comm. ACM, 24:381-395.

    Google Scholar 

  • Forsyth, D.A. 1990. A novel algorithm for colour constancy. Int. J. Computer Vision, 5:5-36.

    Google Scholar 

  • Funt, B.V., Barnard, K., and Martin, L. 1998. Is machine colour constancy good enough? In ECCV, pp. 445-459.

  • Gamerman, D. 1997. Markov Chain Monte Carlo. Chapman-Hall: New York.

    Google Scholar 

  • Gelman, A. and Rubin, D.B. 1993. Inference from iterative simulation using multiple sequences. Statistical Science, 7: 457-511.

    Google Scholar 

  • Gelman, A., Carlin, J.B., Stern, H.S., and Rubin, D.B. 1995. Bayesian Data Analysis. Chapman and Hall: London.

    Google Scholar 

  • Geman, S. and Geman, D. 1984. Stochastic relaxation, gibbs distributions and the bayesian restoration of images. IEEE T. Pattern Analysis and Machine Intelligence, 6:721-741.

    Google Scholar 

  • Geman, S. and Graffigne, C. 1986. Markov random field image models and their application to computer vision. In Proc. Int. Congress of Math.

  • Geweke, J. 1992. Evaluating the accuracy of sampling based approaches to the calculation of posterior moments. In Bayesian Statistics 4, J.M. Bernardo, J. Berger, A.P. Dawid, and A.F.M. Smith (Eds.). Oxford: Clarendon Press.

    Google Scholar 

  • Geyer, C. 1999. Likelihood inference for spatial point processes. In Stochastic Geometry: Likelihood and Computation. O.E. Barndorff-Nielsen, W.S. Kendall, and M.N.W. van Lieshout (Eds.), Chapman and Hall: Boca Raton.

    Google Scholar 

  • Gilks, W.R. and Roberts, G.O. 1996. Strategies for improving mcmc. In Markov Chain Monte Carlo in Practice, W.R. Gilks, S. Richardson, and D.J. Spiegelhalter (Eds.), Chapman and Hall: New York.

    Google Scholar 

  • Gilks, W.R., Richardson, S., and Spiegelhalter, D.J. 1996. Introducing Markov Chain Monte Carlo. In Markov Chain Monte Carlo in Practice, W.R. Gilks, S. Richardson, and D.J. Spiegelhalter (Eds.), Chapman and Hall: New York.

    Google Scholar 

  • Gilks, W.R., Richardson, S., and Spiegelhalter, D.J. 1996. Introduction to markov chain monte carlo. In Markov Chain Monte Carlo in Practice,W.R. Gilks, S. Richardson, and D.J. Spiegelhalter (Eds.), Chapman and Hall: New York.

    Google Scholar 

  • Gilks, W.R., Richardson, S., and Spiegelhalter, D.J. (Eds.). 1996. Markov Chain Monte Carlo in Practice. Chapman and Hall: New York.

    Google Scholar 

  • Golden, B.L. and Skiscim, C.C. 1986. Using simulated annealing to solve routing and location problems. Naval Res. Log. Quart., 33:261–279.

    Google Scholar 

  • Van, Gool L. and Zisserman, A.P. 1997. Automatic 3d model building from video sequences. European Transactions on Telecommunications, 8(4):369–378.

    Google Scholar 

  • Green, P.J. 1995. Reversible jump markov chain monte carlo computation and bayesian model determination. Biometrika, 82(4):711–732.

    Google Scholar 

  • Green, P.J. 1996. Mcmc in image analysis. In Markov chain Monte Carlo in practice, W.R. Gilks, S. Richardson, and D.J. Spiegelhalter (Eds.), Chapman and Hall, New York, pp. 381–400.

    Google Scholar 

  • Grenander, U. 1983. Tutorial in pattern theory. Technical report, Brown University, Providence, Rhode Island.

    Google Scholar 

  • Grenander, Ulf. 1993. General Pattern Theory. Oxford University Press: New York.

    Google Scholar 

  • Hartley, R. and Zisserman, A. 2000. Multiple View Geometry. Cambridge University Press: New York.

    Google Scholar 

  • Huang, T., Koller, D., Malik, J., Ogasawara, G., Rao, B., Russell, S., and Weber, J. 1994. Automatic symbolic traffic scene analysis using belief networks. In AAAI, pp. 966-972.

  • Ioffe, S. and Forsyth, D.A. 1999. Finding people by sampling. In Int. Conf. on Computer Vision, pp. 1092-1097.

  • Jacobs, D. 1997. Linear fitting with missing data: Applications to structure-from-motion and to characterizing intensity images. In IEEE Conf. on Computer Vision and Pattern Recognition.

  • Jerrum, Mark and Sinclair, Alistair. 1996. The Markov chain Monte Carlo method: An approach to approximate counting and integration. In Approximation Algorithms for NP-Hard Problems, D.S. Hochbaum (Ed.), PWS Publishing: Boston.

    Google Scholar 

  • Dubuisson Jolly, M.-P., Lakshmanan, S., and Jain, A.K. 1996.Vehicle segmentation and classification using deformable templates. IEEE T. Pattern Analysis and Machine Intelligence, 18(3):293-308.

    Google Scholar 

  • Kanazawa, K., Koller, D., and Russell, S. 1995. Stochastic simulation algorithms for dynamic probabilistic networks. In Proc Uncertainty in AI.

  • Kitagawa, G. 1987. Non-gaussian state space modelling of nonstationary time series with discussion. J. Am. Stat. Assoc., 82: 1032-1063.

    Google Scholar 

  • Land, E.H. and McCann, J.J. 1971. Lightness and retinex theory. J. Opt. Soc. Am., 61(1):1-11.

    Google Scholar 

  • Lee, H.C. 1986. Method for computing the scene-illuminant chromaticity from specular highlights. J. Opt. Soc. Am.-A, 3:1694-1699.

    Google Scholar 

  • Li, S.Z. 1995. Markov Random Field Modeling in Computer Vision. Springer-Verlag: New York.

    Google Scholar 

  • Maloney, L.T. and Wandell, B.A. 1986. A computational model of color constancy. J. Opt. Soc. Am., 1:29-33.

    Google Scholar 

  • Marimont, D.H. and Wandell, B.A. 1992. Linear models of surface and illuminant spectra. J. Opt. Soc. Am.-A, 9:1905-1913.

    Google Scholar 

  • Maybank, S.J. and Sturm, P.F. 1999. Minimum description length and the inference of scene structure from images. In IEE Colloquium on Applied Statistical Pattern Recognition, pp. 9-16.

  • McLachlan, G.J. and Krishnan, T. 1996. The EM Algorithm and Extensions. John Wiley and Sons: New York.

    Google Scholar 

  • Moller, J. 1999. Markov chain Monte Carlo and spatial point processes. In Stochastic Geometry: Likelihood and Computation, O.E. Barndorff-Nielsen, W.S. Kendall, and M.N.W. van Lieshout (Eds.), Chapman and Hall: Boca Raton.

    Google Scholar 

  • Morris, D.D. and Kanade, T. 1998. A unified factorization algorithm for points, line segments and planes with uncertainty models. In Int. Conf. on Computer Vision, pp. 696-702.

  • Mumford, D. and Shah, J. 1989. Optimal approximations by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math., 42:577-684.

    Google Scholar 

  • Neal, R.M. 1993. Probabilistic inference using markov chain monte carlo methods. Computer science tech report crg-tr-93-1, University of Toronto.

  • Noble, J.A. and Mundy, J. 1993. Toward template-based tolerancing from a bayesian viewpoint. In IEEE Conf. on Computer Vision and Pattern Recognition, pp. 246-252.

  • Pavlovic, V., Frey, B.J., and Huang, T.S. 1999. Time-series classifi-cation using mixed-state dynamic bayesian networks. In IEEE Conf. on Computer Vision and Pattern Recognition, pp. 609-612.

  • Pavlovic, V., Rehg, J.M., Cham, Tat-Jen, and Murphy, K.P. 1999. A dynamic bayesian network approach to figure tracking using learned dynamic models. In Int. Conf. on Computer Vision, pp. 94-101.

  • Phillips, D.B. and Smith, A.F.M. 1996. Bayesian model comparison via jump diffusion. In Markov Chain Monte Carlo in Practice, W.R. Gilks, S. Richardson, and D.J. Spiegelhalter (Eds.), Chapman and Hall.

  • Poelman, C. 1993. The paraperspective and projective factorisation method for recovering shape and motion. Cmu cs-93-219, Carnegie-Mellon University.

  • Richardson, S. and Green, P.J. 1987. On bayesian analysis of mixtures with an unknown number of components. Proc. Roy. Stat. Soc. B, 59:731-792.

    Google Scholar 

  • Ripley, B. 1987. Stochastic Simulation. Wiley.

  • Ripley, B.D. 1996. Pattern Recognition and Neural Networks. Cambridge University Press.

  • Roberts, G.O. 1992. Convergence diagnostics of the gibbs sampler. In Bayesian Statistics 4, J.M. Bernardo, J. Berger, A.P. Dawid, and A.F.M. Smith (Eds.), Oxford: Clarendon Press.

    Google Scholar 

  • Roberts, G.O. 1996. Markov chain concepts related to sampling algorithms. In Markov chain Monte Carlo in Practice, W.R. Gilks, S. Richardson, and D.J. Spiegelhalter (Eds.), Chapman and Hall: New York.

    Google Scholar 

  • Rousseeuw, P.J. 1987. Robust Regression and Outlier Detection. Wiley: New York.

    Google Scholar 

  • Sanz-Serna, J.M. and Calvo, M.P. 1994. Numerical Hamiltonian Problems. Chapman and Hall: New York.

    Google Scholar 

  • Sarkar, S. and Boyer, K.L. 1992. Perceptual organization using bayesian networks. In IEEE Conf. on Computer Vision and Pattern Recognition, pp. 251-256.

  • Sarkar, S. and Boyer, K.L. 1994. Automated design of bayesian perceptual inference networks. In IEEE Conf. on Computer Vision and Pattern Recognition, pp. 98-103.

  • Sullivan, J., Blake, A., Isard, M., and MacCormick, J. 1999. Object localization by bayesian correlation. In Int. Conf. on Computer Vision, pp. 1068-1075.

  • Tierney, L. 1996. Introduction to general state-space markov chain theory. In Markov Chain Monte Carlo in Practice, W.R. Gilks, S. Richardson, and D.J. Spiegelhalter (Eds.), Chapman and Hall: New York.

    Google Scholar 

  • Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. Int. J. of Comp. Vision, 9(2):137-154.

    Google Scholar 

  • Torr, P.H.S. and Murray, D.W. 1997. The development and comparison of robust methods for estimating the fundamental matrix. Int. J. Computer Vision, 24:271-300.

    Google Scholar 

  • Torr, P. and Zisserman, A. 1998. Robust computation and parametrization of multiple view relations. In Int. Conf. on Computer Vision, pp. 485-491.

  • Traub, J.F. and Werschulz, A. 1999. Complexity and Information. Cambridge University Press.

  • Triggs, B. 1995. Factorization methods for projective structure and motion. In IEEE Conf. on Computer Vision and Pattern Recognition, pp. 845-851.

  • Yuille, A.L. and Coughlan, J. 1999. High-level and generic models for visual search: When does high level knowledge help? In IEEE Conf. on Computer Vision and Pattern Recognition, pp. 631-637.

  • Zhu, S.C., Wu, Y., and Mumford, D. 1998. Filters, random fields and maximum entropy (frame): Towards a unified theory for texture modelling. Int. J. Computer Vision, 27:107-126.

    Google Scholar 

  • Zhu, S.C. 1998. Stochastic computation of medial axis in markov random fields. In IEEE Conf. on Computer Vision and Pattern Recognition, pp. 72-80.

  • Zhu, S.C., Zhang, R., and Tu, Z. 2000. Integrating bottom-up/topdown for object recognition by data driven markov chain monte carlo. In IEEE Conf. on Computer Vision and Pattern Recognition, pp. 738-745.

Download references

Author information

Authors and Affiliations

  1. Computer Science Division, University of California at Berkeley, Berkeley, CA, 94720, USA

    D.A. Forsyth, J. Haddon & S. Ioffe

Authors
  1. D.A. Forsyth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Haddon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Ioffe
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Forsyth, D., Haddon, J. & Ioffe, S. The Joy of Sampling. International Journal of Computer Vision 41, 109–134 (2001). https://doi.org/10.1023/A:1011165200654

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1011165200654

Search

Navigation