Skip to main content
SpringerLink
Log in

Local Adaptivity to Variable Smoothness for Exemplar-Based Image Regularization and Representation

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

A novel adaptive and exemplar-based approach is proposed for image restoration (denoising) and representation. The method is based on a pointwise selection of similar image patches of fixed size in the variable neighborhood of each pixel. The main idea is to associate with each pixel the weighted sum of data points within an adaptive neighborhood. We use small image patches (e.g. 7×7 or 9×9 patches) to compute these weights since they are able to capture local geometric patterns and texels seen in images. In this paper, we mainly focus on the problem of adaptive neighborhood selection in a manner that balances the accuracy of approximation and the stochastic error, at each spatial position. The proposed pointwise estimator is then iterative and automatically adapts to the degree of underlying smoothness with minimal a priori assumptions on the function to be recovered. The method is applied to artificially corrupted real images and the performance is very close, and in some cases even surpasses, to that of the already published denoising methods. The proposed algorithm is demonstrated on real images corrupted by non-Gaussian noise and is used for applications in bio-imaging.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aujol, J. F., Aubert, G., Blanc-Féraud, L., & Chambolle, A. (2005). Image decomposition into a bounded variation component and an oscillating component. Journal of Mathematical Imaging and Vision, 22(1), 71–88.

    Article  MathSciNet  Google Scholar 

  • Aurich, V., & Weule, J. (1995). Nonlinear Gaussian filters performing edge preserving diffusion. In Proceedings of the 17th DAGM symposium (pp. 538–545), Bielefeld, Germany.

  • Awate, S. P., & Whitaker, R. T. (2005). Higher-order image statistics for unsupervised, information-theoretic, adaptive image filtering. In Proceedings of the computer vision and pattern recognition (CVPR’05) (Vol. 2, pp. 44–51), San Diego, CA.

  • Azzabou, N., Paragios, N., Cao, F., & Guichard, F. (2007). Variable bandwidth image denoising using image-based noise models. In Proceedings of the IEEE international conference on computer vision and pattern recognition (CVPR’07) (pp. 1–7). Minneapolis, MN.

  • Barash, D. (2002). A fundamental relationship between bilateral filtering, adaptive smoothing and the nonlinear diffusion equation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(6), 844–847.

    Article  Google Scholar 

  • Barash, D., & Comaniciu, D. (2004). A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean-shift. Image and Video Computing, 22(1), 73–81.

    Article  Google Scholar 

  • Blake, A., & Zisserman, A. (1987). Visual reconstruction. Cambridge: MIT Press.

    Google Scholar 

  • Black, M. J., & Sapiro, G. (1999). Edges as outliers: anisotropic smoothing using local image statistics. In Lecture notes in computer science : Vol. 1682. Proceedings of the scale-space theories in computer vision (Scale-Space’99) (pp. 259–270), Kerkyra, Greece. Berlin: Springer.

    Chapter  Google Scholar 

  • Black, M. J., Sapiro, G., Marimont, D. H., & Heeger, D. (1998). Robust anisotropic diffusion. IEEE Transactions on Image Processing, 7(3), 421–432.

    Article  Google Scholar 

  • Boulanger, J., Kervrann, K., & Bouthemy, P. (2007). Space–time adaptation for patch based image sequence restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(6), 1096–1102.

    Article  Google Scholar 

  • Boykov, Y., Veksler, O., & Zabih, R. (1998). A variable window approach to early vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12), 1283–1294.

    Article  Google Scholar 

  • Brox, T., & Cremers, D. (2007). Iterated non-local means for texture restoration. In In Proceedings of the conference on scale-space and variational methods (SSVM’ 07), Ischia, Italy.

  • Brox, T., & Weickert, J. (2004). A TV flow based local scale measure for texture discrimination. In Proceedings of the European conference on computer vision (ECCV’04) (Vol. 2, pp. 578–590), Prague, Czech Republic.

  • Buades, A., Coll, B., & Morel, J.-M. (2005a). A review of image denoising algorithms, with a new one. Multiscale Modeling and Simulation, 4(2), 490–530.

    Article  MathSciNet  MATH  Google Scholar 

  • Buades, A., Coll, B., & Morel, J.-M. (2005b). A non local algorithm for image denoising. In Proceedings of the computer vision and pattern recognition (CVPR’05) (Vol. 2, pp. 60–65), San Diego, CA.

  • Catte, F., Lions, P.-L., Morel, J.-M., & Coll, T. (1992). Image selective smoothing and edge-detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 29(1), 182–193.

    Article  MathSciNet  MATH  Google Scholar 

  • Chan, T. F., Osher, S., & Shen, J. (2001). The digital TV filter and nonlinear denoising. IEEE Transactions on Image Processing, 10(2), 231–241.

    Article  MATH  Google Scholar 

  • Cheng, I. (1995). Mean-shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(8), 790–799.

    Article  Google Scholar 

  • Chu, C. K., Glad, K., Godtliebsen, F., & Marron, J. S. (1998). Edge-preserving smoothers for image processing. Journal of the American Statistical Association, 93(442), 526–555.

    Article  MathSciNet  MATH  Google Scholar 

  • Comaniciu, D., & Meer, P. (2002). Mean-shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5), 603–619.

    Article  Google Scholar 

  • Comaniciu, D., Ramesh, V., & Meer, P. (2001). The variable bandwidth mean-shift and data-driven scale selection. In Proceedings of the international conference on computer vision (ICCV’01) (Vol. 1, pp. 438–445), Vancouver, Canada.

  • Criminisi, A., Pérez, P., & Toyama, K. (2004). Region filling and object removal by exemplar-based inpainting. IEEE Transactions on Image Processing, 13(9), 1200–1212.

    Article  Google Scholar 

  • Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8).

  • De Bonet, J. S. (1997).Noise reduction through detection of signal redundancy. In Rethinking artificial intelligence. MIT AI Lab.

  • Donoho, D. L., & Johnston, I. M. (1994). Ideal spatial adaptation via wavelet shrinkage. Biometrika, 81, 425–455.

    Article  MathSciNet  MATH  Google Scholar 

  • Donoho, D. L., & Johnston, I. M. (1995). Denoising by soft-thresholding. IEEE Transactions on Information Theory, 41, 613–627.

    Article  MATH  Google Scholar 

  • Efros, A., & Leung, T. (1999) Texture synthesis by non-parametric sampling. In Proceedings of the international conference on computer vision (ICCV’99) (pp. 1033–1038), Kerkyra, Greece.

  • Elad, M. (2002). On the bilateral filter and ways to improve it. IEEE Transactions on Image Processing, 11(10), 1141–1151.

    Article  MathSciNet  Google Scholar 

  • Elad, M., Aharon, M. (2006) Image denoising via learned dictionaries and sparse representation. In Proceedings of the conference on computer vision and pattern recognition (CVPR’06) (Vol. 1, pp. 895–900), New York.

  • Fischl, B., & Schwartz, E. L. (1999). Adaptive nonlocal filtering: a fast alternative to anisotropic diffusion for image enhancement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(1), 42–48.

    Article  Google Scholar 

  • Fitzgibbon, A., Wexler, Y., & Zisserman, A. (2003). Image-based rendering using image-based priors. In Proceedings of the international conference on computer vision (ICCV’03), Nice, France.

  • Freeman, W. T., Pasztor, E. C., & Carmichael, O. T. (2000). Learning low-level vision. International Journal of Computer Vision, 40(1), 25–47.

    Article  MATH  Google Scholar 

  • Gasser, T., Sroka, L., & Jennen Steinmetz, C. (1986). Residual variance and residual pattern in nonlinear regression. Biometrika, 73, 625–633.

    Article  MathSciNet  MATH  Google Scholar 

  • Geman, D., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6), 721–741.

    Article  MATH  Google Scholar 

  • Geman, D., Geman, S., Graffigne, C., & Dong, P. (1990). Boundary detection by constrained optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7), 609–628.

    Article  Google Scholar 

  • Ghazel, M., Freeman, G. H., & Vrscay, E. R. (2003). Fractal image denoising. IEEE Transactions on Image Processing, 12(12), 1560–1578.

    Article  Google Scholar 

  • Gijbels, I., Lambert, A., & Qiu, P. (2006). Edge-preserving image denoising and estimation of discontinuous surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(7), 1075–1087.

    Article  Google Scholar 

  • Gilboa, G., & Osher, S. (2007). Nonlocal linear image regularization and supervised segmentation. SIAM Journal on Multiscale Modelling and Simulation, 6, 595–630.

    Article  MathSciNet  MATH  Google Scholar 

  • Gilboa, G., Zeevi, Y. Y., & Sochen, N. (2003). Texture preserving variational denoising using an adaptive fidelity term. In Proceedings VLSM’03, Nice, France.

  • Godtliebsen, F., Spjotvoll, E., & Marron, J. S. (1997). A nonlinear Gaussian filter applied to images with discontinuities. Journal of Nonparametric Statistics, 8, 21–43.

    Article  MathSciNet  MATH  Google Scholar 

  • Goldenshluger, A., & Nemirovsky, A. (1997). On spatial adaptive estimation of nonparametric regression. Mathematical Methods of Statistics, 6(2), 135–170.

    MathSciNet  MATH  Google Scholar 

  • Gomez, G., Marroquin, J. L., & Sucar, L. E. (2000). Probabilistic estimation of local scale. In Proceedings of the international conference on pattern recognition (ICPR’00) (Vol. 3, pp. 798–801), Barcelona, Spain.

  • Hardle, W., & Linton, O. (1994). Applied nonparametric methods. In R.F. Engle & D.L. McFadden (Eds.), Handbook of econometrics (Vol. IV, pp. 2295–2381). North Holland: Amsterdam.

    Google Scholar 

  • Jojic, N., Frey, B., & Kannan, A. (2003). Epitomic analysis of appearance and shape. In Proceedings of the international conference on computer vision (ICCV’03) (Vol. 1, pp. 34–41), Nice, France.

  • Juditsky, A. (1997). Wavelet estimators: adapting to unknown smoothness. Mathematical Methods of Statistics, 1, 1–20.

    MathSciNet  Google Scholar 

  • Katkovnik, V., Egiazarian, K., & Astola, J. (2002). Adaptive window size image denoising based on intersection of confidence intervals (ICI) rule. Journal of Mathematical Imaging and Vision, 16(3), 223–235.

    Article  MathSciNet  MATH  Google Scholar 

  • Kervrann, C. (2004). An adaptive window approach for image smoothing and structures preserving. In Proceedings of the European conference on computer vision (ECCV’04) (Vol. 3, pp. 132–144), Prague, Czech Republic.

  • Kervrann, C., & Boulanger, J. (2005). Local adaptivity to variable smoothness for exemplar-based image denoising and representation. INRIA Research Report, RR-5624, July 2005.

  • Kervrann, C., & Boulanger, J. (2006). Unsupervised patch-based image regularization and representation. In Proceedings of the European conference on computer vision (ECCV’06) (Vol. 4, pp. 555–567), Graz, Austria.

  • Kervrann, C., & Heitz, F. (1995). A Markov random field model-based approach to unsupervised texture segmentation using local and global spatial statistics. IEEE Transactions on Image Processing, 4(6), 856–862.

    Article  Google Scholar 

  • Kervrann, C., Boulanger, J., & Coupé, P. (2007). Bayesian non-local means filter, image redundancy and adaptive dictionaries for noise removal. In Proceedings of the conference on scale-space and variational methods (SSVM’ 07), Ischia, Italy.

  • Kinderman, S., Osher, S., & Jones, P. W. (2005). Deblurring and denoising of images by nonlocal functionals. Multiscale Modeling and Simulation, 4, 1091–1115.

    Article  MathSciNet  Google Scholar 

  • Lee, J. S. (1983). Digital image smoothing and the sigma filter. Computer Vision, Graphics, and Image Processing, 24, 255–269.

    Article  Google Scholar 

  • Le Pennec, E., & Mallat, S. (2005). Sparse geometric image representation with bandelets. IEEE Transactions on Image Processing, 14(4), 423–438.

    Article  MathSciNet  Google Scholar 

  • Lepskii, O. (1990). On a problem of adaptive estimation on white Gaussian noise. Theory of Probability and Its Applications, 35, 454–466.

    Article  MathSciNet  Google Scholar 

  • Lepskii, O. (1991). Asymptotically minimax adaptive estimation, 1: uppers bounds. Theory of Probability and Its Applications, 36(4), 654–659.

    Article  MathSciNet  Google Scholar 

  • Lepski, O. V., Mammen, E., & Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors. Annals of Statistics, 25(3), 929–947.

    Article  MathSciNet  MATH  Google Scholar 

  • Lindeberg, T. (1998). Edge detection and ridge detection with automatic scale selection. International Journal of Computer Vision, 30(2), 117–154.

    Article  Google Scholar 

  • Maurizot, M., Bouthemy, P., Delyon, B., Juditski, A., & Odobez, J.-M. (1995). Determination of singular points in 2D deformable flow fields. In IEEE proceedings of the international conference on image processing (ICIP’95) (Vol. 3, pp. 488–491), Washington, DC.

  • Meyer, Y. (2002). University lecture series: vol. 22. Oscillating patterns in image processing and nonlinear evolution equations. Providence: AMS.

    Google Scholar 

  • Mairal, J., Sapiro, G., & Elad, M. (2007). Multiscale sparse image representation with learned dictionaries. In Proceedings of the international conference on image processing (ICIP’07), San Antonio, TX, USA.

  • Mrazek, P. (2003). Selection of optimal stopping time for nonlinear diffusion filtering. International Journal of Computer Vision, 52(2/3), 189–203.

    Article  Google Scholar 

  • Mrazek, P., Weickert, J., & Bruhn, A. On robust estimation and smoothing with spatial and tonal kernels. Preprint No. 51, University of Bremen, Germany.

  • Mumford, D., & Shah, J. (1989). Optimal approximations by piecewise smooth functions and variational problems. Communications on Pure and Applied Mathematics, 42(5), 577–685.

    Article  MathSciNet  MATH  Google Scholar 

  • Nitzberg, M., & Shiota, T. (1992). Nonlinear image filtering with edge and corner enhancement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(8), 826–833.

    Article  Google Scholar 

  • Osher, S., Solé, A., & Vese, L. (2003). Image decomposition and restoration using total variation minimization and the H −1 norm. Multiscale Modeling and Simulation, 1(3), 349–370.

    Article  MathSciNet  MATH  Google Scholar 

  • Perona, P., & Malik, J. (1990). Scale space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7), 629–639.

    Article  Google Scholar 

  • Pizurica, A., & Philips, W. (2006). Estimating probability of presence of a signal of interest in multiresolution single and multiband image denoising. IEEE Transactions on Image Processing, 15(3), 654–665.

    Article  Google Scholar 

  • Polzehl, J., & Spokoiny, V. (2000). Adaptive weights smoothing with application to image restoration. Journal of the Royal Statistical Society, Series B, 62(2), 335–354.

    Article  MathSciNet  Google Scholar 

  • Portilla, J., Strela, V., Wainwright, M., & Simoncelli, E. (2003). Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Transactions on Image Processing, 12(11), 1338–1351.

    Article  MathSciNet  Google Scholar 

  • Roth, S., & Black, M. J. (2005). Fields of experts: a framework for learning image priors with applications. In Proceedings of the conference on computer vision and pattern recognition (CVPR’05) (Vol. 2, pp. 860–867), San Diego, CA.

  • Rudin, L., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60, 259–268.

    Article  MATH  Google Scholar 

  • Saint-Marc, P., Chen, J. S., & Médioni, G. (1991). Adaptive smoothing: a general tool for early vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6), 514–529.

    Article  Google Scholar 

  • Scott, D. W. (1992). Multivariate density estimation. New York: Wiley.

    MATH  Google Scholar 

  • Singh, M., & Ahuja, N. Regression based bandwidth selection for segmentation using Parzen windows. In Proceedings of the international conference on computer vision (ICCV’03) (Vol. 1, pp. 2–9), Nice, France.

  • Smith, S. M., & Brady, M. (1997). SUSAN—a new approach to low-level image processing. International Journal of Computer Vision, 23(1), 45–78.

    Article  Google Scholar 

  • Spokoiny, V. G. (1998). Estimation of a function with discontinuities via local polynomial fit with an adaptive window choice. Annals of Statistics, 26(4), 141–170.

    MathSciNet  Google Scholar 

  • Stankovic, L. (2004). Performance analysis of the adaptive algorithm for bias-to-variance trade-off. IEEE Transactions on Signal Processing, 52(5), 1228–1234.

    Article  MathSciNet  Google Scholar 

  • Sochen, N., Kimmel, R., & Bruckstein, A. M. (2001). Diffusions and confusions in signal and image processing. Journal of Mathematical Imaging and Vision, 14(3), 237–244.

    Article  MathSciNet  Google Scholar 

  • Spira, A., Kimmel, R., & Sochen, N. (2003). Efficient Beltrami flow using a short-time kernel. In Proceedings of the international conference on scale-space theories in computer vision (Scale-Space’03) (pp. 551–522), Isle of Skye, Scotland.

  • Starck, J. L., Candes, E., & Donoho, D. L. (2002). The curvelet transform for image denoising. IEEE Transactions on Image Processing, 11(6), 670–684.

    Article  MathSciNet  Google Scholar 

  • Stewart, C. V., Tsai, C.-L., & Roysam, B. (2003). The dual-bootstrap iterative closest point algorithm with application to retinal image registration. IEEE Transactions on Medical Imaging, 22(11), 1379–1394.

    Article  Google Scholar 

  • Tomasi, C., & Manduchi, R. (1998). Bilateral filtering for gray and color images. In Proceedings of the international confernce on computer vision (ICCV’98) (pp. 839–846), Bombay, India.

  • Tschumperlé, D. (2006). Curvature-preserving regularization of multi-valued images using PDE’s. In Proceedings of the European confernce on computer vision (ECCV’06) (Vol. 2, pp. 295–307), Graz, Austria.

  • van de Weijer, J., & van den Boomgaard, R. (2001). Local mode filtering. In Proceedings of the confernce on computer vision and pattern recognition (CVPR’01) (Vol. II, pp. 428–433), Kauai, HI.

  • van den Boomgaard, R., & van de Weijer, J. (2002). On the equivalence of local-mode finding, robust estimation and mean-shift analysis as used in early vision tasks. In Proceedings of the international confernce on pattern recognition (ICPR’02) (Vol. III, pp. 927–930), Quebec City, Canada.

  • Wang, Z., & Zhang, D. (1998). Restoration of impulse noise corrupted images using long-range correlation. IEEE Signal Processing Letters, 5(0), 4–6.

    Article  Google Scholar 

  • Weickert, J. (1998). Anisotropic diffusion in image processing. Stuttgart: Teubner.

    MATH  Google Scholar 

  • Weickert, J. (1999). Coherence-enhancing diffusion filtering. International Journal of Computer Vision, 31(2–3), 111–127.

    Article  Google Scholar 

  • Yang, G. Z., Burger, P., Firmin, D. N., & Underwood, S. R. (1996). Structure adaptive anisotropic image filtering. Image and Vision Computing, 14, 135–145.

    Article  Google Scholar 

  • Yaroslavsky, L. P., & Eden, M. (1996). Fundamentals of digital optics. Boston: Birkhäuser.

    MATH  Google Scholar 

  • Zhang, D., & Wang, Z. (2002). Image information restoration based on long-range correlation. IEEE Transactions on Circuits and Systems for Video Technology, 12, 331–341.

    Article  Google Scholar 

  • Zhu, S. C., Wu, Y., & Mumford, D. (1998). Filters, random fields and maximum entropy (FRAME): towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2), 107–126.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IRISA/INRIA Rennes, Campus Universitaire de Beaulieu, 35042, Rennes cedex, France

    Charles Kervrann & Jérôme Boulanger

  2. INRA, UR341 Mathématiques et Informatique Appliquées, 78352, Jouy-en-Josas, France

    Charles Kervrann & Jérôme Boulanger

Authors
  1. Charles Kervrann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jérôme Boulanger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles Kervrann.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kervrann, C., Boulanger, J. Local Adaptivity to Variable Smoothness for Exemplar-Based Image Regularization and Representation. Int J Comput Vis 79, 45–69 (2008). https://doi.org/10.1007/s11263-007-0096-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-007-0096-2

Keywords

Search

Navigation