Skip to main content
Log in

Building Blocks for Computer Vision with Stochastic Partial Differential Equations

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We discuss the basic concepts of computer vision with stochastic partial differential equations (SPDEs). In typical approaches based on partial differential equations (PDEs), the end result in the best case is usually one value per pixel, the “expected” value. Error estimates or even full probability density functions PDFs are usually not available. This paper provides a framework allowing one to derive such PDFs, rendering computer vision approaches into measurements fulfilling scientific standards due to full error propagation. We identify the image data with random fields in order to model images and image sequences which carry uncertainty in their gray values, e.g. due to noise in the acquisition process.

The noisy behaviors of gray values is modeled as stochastic processes which are approximated with the method of generalized polynomial chaos (Wiener-Askey-Chaos). The Wiener-Askey polynomial chaos is combined with a standard spatial approximation based upon piecewise multi-linear finite elements. We present the basic building blocks needed for computer vision and image processing in this stochastic setting, i.e. we discuss the computation of stochastic moments, projections, gradient magnitudes, edge indicators, structure tensors, etc. Finally we show applications of our framework to derive stochastic analogs of well known PDEs for de-noising and optical flow extraction. These models are discretized with the stochastic Galerkin method. Our selection of SPDE models allows us to draw connections to the classical deterministic models as well as to stochastic image processing not based on PDEs. Several examples guide the reader through the presentation and show the usefulness of the framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Amiaz, T., & Kiryati, N. (2006). Piecewise-smooth dense optical flow via level sets. International Journal of Computer Vision, 68(2), 111–124.

    Article  Google Scholar 

  • Avriel, M. (2003). Nonlinear programming: Analysis and methods. New York: Dover.

    MATH  Google Scholar 

  • Bao, Y., & Krim, H. (2004). Smart nonlinear diffusion: A probabilistic approach. Pattern Analysis and Machine Intelligence, 26(1), 63–72.

    Article  Google Scholar 

  • Black, M. J., & Anandan, P. (1991). Robust dynamic motion estimation over time. In Proc. computer vision and pattern recognition, CVPR-91 (pp. 296–302), June 1991.

  • Black, M. J., & Anandan, P. (1993). A framework for the robust estimation of optical flow. In Proc. ICCV93 (pp. 231–236).

  • Bruhn, A., Weickert, J., & Schnörr, C. (2002). Combining the advantages of local and global optic flow methods. In Proc. DAGM (pp. 454–462).

  • Bruhn, A., Weickert, J., & Schnörr, C. (2005). Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. International Journal of Computer Vision, 61(3), 211–231.

    Article  Google Scholar 

  • Catté, F., Lions, P.-L., Morel, J.-M., & Coll, T. (1992). Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 29(1), 182–193.

    Article  MATH  MathSciNet  Google Scholar 

  • Chorin, A. J. (1971). Hermite expansions in Monte Carlo computation. Journal of Computational Physics, 8, 471–482.

    Article  MathSciNet  Google Scholar 

  • Chorin, A. J. (1974). Gaussian fields and random flow. Journal of Fluid Mechanics, 63, 21–32.

    Article  MATH  MathSciNet  Google Scholar 

  • Cohen, I. (1993). Nonlinear variational method for optical flow computation. In SCIA93 (pp. 523–530).

  • de Laplace, P. S. (1812). Théorie analytique des probabilites. Paris: Courcier Imprimeur.

    Google Scholar 

  • Deb, M. K., Babuška, I. M., & Oden, J. T. (2001). Solutions of stochastic partial differential equations using Galerkin finite element techniques. Computer Methods in Applied Mechanics Engineering, 190, 6359–6372.

    Article  MATH  Google Scholar 

  • Fermüller, C., Shulman, D., & Aloimonos, Y. (2001). The statistics of optical flow. Computer Vision and Image Understanding, 82(1), 1–32.

    Article  MATH  Google Scholar 

  • Forsyth, D. A., & Ponce, J. (2003). Computer vision: a modern approach. Englewood Cliffs: Prentice Hall.

    Google Scholar 

  • Ghanem, R. G. (1999). Higher order sensitivity of heat conduction problems to random data using the spectral stochastic finite element method. ASME Journal of Heat Transfer, 121, 290–299.

    Article  Google Scholar 

  • Ghanem, R. G., & Spanos, P. (1991). Stochastic finite elements: a spectral approach. New York: Springer.

    MATH  Google Scholar 

  • Haussecker, H., & Spies, H. (1999). Motion. In B. Jähne, H. Haußecker, & P. Geißler (Eds.), Handbook of computer vision and applications (pp. 309–396). San Diego: Academic Press.

    Google Scholar 

  • Haussecker, H., Spies, H., & Jähne, B. (1998). Tensor-based image sequence processing techniques for the study of dynamical processes. In Proceedings of the international symposium on real-time imaging and dynamic analysis, ISPRS, commission V, working group IC V/III, Hakodate, Japan, June 1998.

  • Horn, B. K. P., & Schunck, B. (1981). Determining optical flow. Artificial Intelligence, 17, 185–204.

    Article  Google Scholar 

  • Van Huffel, S., & Vandewalle, J. (1991). Frontiers in applied mathematics: Vol. 9. The total least squares problem: Computational aspects and analysis. Philadelphia: SIAM.

    MATH  Google Scholar 

  • Iijima, T. (1962). Basic theory on normalization of pattern (in case of typical one-dimensional pattern). Bulletin of the Electrotechnical Laboratory, 26, 368–388 (in Japanese).

    Google Scholar 

  • Iijima, T. (1963). Theory of pattern recognition. Electronics and Communications in Japan (pp. 123–134).

  • Jähne, B. (1993). Spatio-temporal image processing: Theory and scientific applications. Lecture notes in computer science. Berlin: Springer.

    MATH  Google Scholar 

  • Kearney, J. K., Thompson, W. B., & Boley, D. L. (1987). Optical flow estimation: An error analysis of gradient-based methods with local optimization. PAMI, 9(2), 229–244.

    Google Scholar 

  • Keese, A. (2004). Numerical solution of systems with stochastic uncertainties: A general purpose framework for stochastic finite elements. Ph.D. thesis, Technical University Braunschweig.

  • Kichenassamy, S. (1997). The Perona-Malik paradox. SIAM Journal on Applied Mathematics, 57(5), 1328–1342.

    Article  MATH  MathSciNet  Google Scholar 

  • Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In DARPA image understanding workshop (pp. 121–130).

  • Lucor, D., Su, C.-H., & Karniadakis, G. E. (2004). Generalized polynomial chaos and random oscillators. International Journal for Numerical Methods in Engineering, 60, 571–596.

    Article  MATH  MathSciNet  Google Scholar 

  • Le Maître, O. P., Reagan, M., Najm, H. N., Ghanem, R. G., & Knio, O. M. (2002). A stochastic projection method for fluid flow II: random process. Journal of Computational Physics, 181(1), 9–44.

    Article  MATH  MathSciNet  Google Scholar 

  • Malliavin, P. (1997). Stochastic analysis. New York: Springer.

    MATH  Google Scholar 

  • Maltz, F. H., & Hitzl, D. L. (1979). Variance reduction in Monte Carlo computations using multi-dimensional Hermite polynomials. Journal of Computational Physics, 32, 345–376.

    Article  MATH  MathSciNet  Google Scholar 

  • Meecham, W. C., & Jeng, D. T. (1968). Use of Wiener-Hermite expansion for nearly normal turbulence. Journal of Fluid Mechanics, 32, 225–249.

    Article  MATH  Google Scholar 

  • Mikula, K., Preusser, T., & Rumpf, M. (2004). Morphological image sequence processing. Computing and Visualization in Science, 6(4), 197–209.

    Article  MathSciNet  Google Scholar 

  • Narayanan, V. A., & Zabaras, N. (2004). Stochastic inverse heat conduction using a spectral approach. International Journal for Numerical Methods in Engineering, 60, 1569–1593.

    Article  MATH  MathSciNet  Google Scholar 

  • Nestares, O., & Fleet, D. J. (2003). Error-in-variables likelihood functions for motion estimation. In IEEE international conference on image processing (ICIP) (Vol. III, pp. 77–80). Barcelona.

  • Nestares, O., Fleet, D. J., & Heeger, D. (2000). Likelihood functions and confidence bounds for total-least-squares problems. In CVPR’00 (Vol. 1).

  • Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158.

    Article  Google Scholar 

  • Perona, P., & Malik, J. (1990). Scale space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 629–639.

    Article  Google Scholar 

  • Preusser, T., & Rumpf, M. (1999). An adaptive finite element method for large scale image processing. In Proceedings scale-space ’99, scale space theories in computer vision, second international conference (pp. 223–234).

  • Reagan, M. T., Najm, H. N., Debusschere, B. J., Le Maître, O. P., Knio, O. M., & Ghanem, R. G. (2004). Spectral stochastic uncertainty quantification in chemical systems. Combustion Theory and Modelling, 8, 607–632.

    Article  Google Scholar 

  • Reagan, M. T., Najm, H. N., Pebay, P. P., Knio, O. M., & Ghanem, R. G. (2005). Quantifying uncertainty in chemical systems modeling. International Journal of Chemical Kinetics, 37, 386–382.

    Article  Google Scholar 

  • Scharr, H. (2006). Diffusion-like reconstruction schemes from linear data models. In Lecture notes in computer science : Vol. 4174. Pattern recognition 2006 (pp. 51–60). Berlin: Springer.

    Chapter  Google Scholar 

  • Scharr, H., Black, M. J., & Haussecker, H. W. (2003). Image statistics and anisotropic diffusion. In Int. conf. on computer vision, ICCV 2003 (pp. 840–847), Nice, France.

  • Sühling, M. (2006). Myocardial motion and deformation analysis from echocardiograms. Ph.D. thesis, Swiss Federal Institute of Technology Lausanne (EPFL), July 2006.

  • Thomee, V. (1984). Galerkin—finite element methods for parabolic problems. New York: Springer.

    MATH  Google Scholar 

  • Gauss, C. F. (1987). Theory of the combination of observations least subject to errors, part one and part two. Supplement (Classics in Applied Mathematics 11) (trans: Stewart, G. W.). Society for Industrial Mathematics, Facsimile edition. English version in 1987. Original version in Latin in 1820s.

  • Weber, J., & Malik, J. (1994). Robust computation of optical flow in a multi-scale differential framework. International Journal of Computer Vision, 14(1), 5–19.

    Google Scholar 

  • Weickert, J. (1998). On discontinuity-preserving optic flow. In Proc. computer vision and mobile robotics workshop (pp. 115–122).

  • Weickert, J., & Schnörr, C. (2001). Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision, 14(3), 245–255.

    Article  MATH  Google Scholar 

  • Wiener, N. (1938). The homogeneous chaos. American Journal of Mathematics, 60(4), 897–936.

    Article  MathSciNet  Google Scholar 

  • Witkin, A. P. (1983). Scale-space filtering. In Proc. eighth int. joint conf. on artificial intelligence (IJCAI) (Vol. 2, pp. 1019–1022).

  • Xiu, D. B., & Karniadakis, G. E. (2002a). Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos. Computational Methods in Applied Mechanics and Engineering, 191, 4927–4948.

    Article  MATH  MathSciNet  Google Scholar 

  • Xiu, D. B., & Karniadakis, G. E. (2002b). The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM Journal on Scientific Computing, 24, 619–644.

    Article  MATH  MathSciNet  Google Scholar 

  • Xiu, D. B., & Karniadakis, G. E. (2003a). Modeling uncertainty in flow simulations via generalized polynomial chaos. Journal of Computational Physics, 187, 137–167.

    Article  MATH  MathSciNet  Google Scholar 

  • Xiu, D. B., & Karniadakis, G. E. (2003b). A new stochastic approach to transient heat conduction modeling with uncertainty. International Journal of Heat and Mass Transfer, 46, 4681–4693.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Tobias Preusser.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Preusser, T., Scharr, H., Krajsek, K. et al. Building Blocks for Computer Vision with Stochastic Partial Differential Equations. Int J Comput Vis 80, 375–405 (2008). https://doi.org/10.1007/s11263-008-0145-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-008-0145-5

Keywords

Navigation