Skip to main content
SpringerLink
Log in

Local Smoothing Neighborhood Filters

  • Reference work entry
Handbook of Mathematical Methods in Imaging

Abstract

Denoising images can be achieved by a spatial averaging of nearby pixels. However, although this method removes noise, it creates blur. Hence, neighborhood filters are usually preferred. These filters perform an average of neighboring pixels, but only under the condition that their gray level is close enough to the one of the pixel in restoration. This very popular method unfortunately creates shocks and staircasing effects. It also excessivelly blurs texture and fine structures when noise dominates the signal.In this chapter, we perform an asymptotic analysis of neighborhood filters as the size of the neighborhood shrinks to zero. We prove that these filters are asymptotically equivalent to the Perona-Malik equation, one of the first nonlinear PDEs proposed for image restoration. As a solution to the shock effect, we propose an extremely simple variant of the neighborhood filter using a linear regression instead of an average. By analyzing its subjacent PDE, we prove that this variant does not create shocks: it is actually related to the mean curvature motion.We also present a generalization of neighborhood filters, the nonlocal means (NL-means) algorithm, addressing the preservation of structure in a digital image. The NL-means algorithm tries to take advantage of the high degree of redundancy of any natural image. By this, we simply mean that every small window in a natural image has many similar windows in the same image. Now in a very general sense inspired by the neighborhood filters, one can define as “neighborhood of a pixel” any set of pixels with a similar window around. All pixels in that neighborhood can be used for predicting its denoised value.We finally analyze the recently introduced variational formulations of neighborhood filters and their application to segmentation and seed diffusion.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 1,200.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Andreu, F., Ballester, C., Caselles, V., Mazon, J.M.: Minimizing total variation flow. Comptes Rendus de l’Academie des Sciences Series I Mathematics 331(11), 867–872 (2000)

    MATH  MathSciNet  Google Scholar 

  2. Arias, P., Caselles, V., Sappiro, G.: A variational framework for non-local image inpainting. In: Proceedings of the EMMCVPR, Bonn. Springer, Heidelberg (2009)

    Book  Google Scholar 

  3. Attneave, F.: Some informational aspects of visual perception. Psychol. Rev. 61(3), 183–193 (1954)

    Article  Google Scholar 

  4. Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations. Springer, New York (2006)

    Google Scholar 

  5. Bae, S., Paris, S., Durand, F.: Two-scale tone management for photographic look. ACM Trans. Graph. (TOG) 25(3), 645 (2006)

    Google Scholar 

  6. Barash, D.: A fundamental relationship between bilateral filtering, adaptive smoothing, and the nonlinear diffusion equation. IEEE Trans. Pattern Anal. Mach. Intell. 24, 844–847 (2002)

    Article  Google Scholar 

  7. Bennett, E.P., Mason, J.L., McMillan, L.: Multispectral bilateral video fusion. IEEE Trans. Image Process. 16(5), 1185 (2007)

    Article  MathSciNet  Google Scholar 

  8. Boulanger, J., Sibarita, J.B., Kervrann, C., Bouthemy, P.: Nonparametric regression for patch-based fluorescence microscopy image sequence denoising. In: 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2008 (ISBI 2008), Paris,pp. 748–751, 2008

    Google Scholar 

  9. Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary and region segmentation of objects in ND images. Int. Conf. Comput. Vis. 1, 105–112 (2001)

    Google Scholar 

  10. Brailean, J.C., Kleihorst, R.P., Efstratiadis, S., Katsaggelos, A.K., Lagendijk, R.L.: Noise reduction filters for dynamic image sequences: a review. Proc. IEEE 83(9), 1272–1292 (1995)

    Article  Google Scholar 

  11. Buades, A., Coll, B., Lisani, J., Sbert, C.: Conditional image diffusion. Inverse Probl. Imaging 1(4):593 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  12. Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model Simul. 4(2), 490–530 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  13. Buades, A., Coll, B., Morel, J.M.: Neighborhood filters and PDE’s. Numer. Math. 105(1), 1–34 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  14. Buades, A., Coll, B., Morel, J.M.: Nonlocal image and movie denoising. Int. J. Comput. Vision. 76(2), 123–139 (2008)

    Article  Google Scholar 

  15. Buades, A., Coll, B., Morel, J.M., Sbert, C.: Self-similarity driven color demosaicking. IEEE Trans. Image Process. 18(6), 1192–1202 (2009)

    Article  MathSciNet  Google Scholar 

  16. Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. Comput. Vis. 22(1), 61–79 (1997)

    Article  MATH  Google Scholar 

  17. Choudhury, P., Tumblin, J.: The trilateral filter for high contrast images and meshes. In: ACM SIGGRAPH 2005 Courses, Los Angeles, p. 5. ACM (2005)

    Google Scholar 

  18. Cleveland, W.S.: Robust locally weighted regression and smoothing scatterplots. J. Am. Stat. Assoc. 74(368), 829–836 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  19. Coifman, R.R., Donoho, D.L.: Translation-Invariant De-Noising. Lecture Notes in Statistics, pp. 125–125. Springer, New York (1995)

    Google Scholar 

  20. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  21. Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  22. Danielyan, A., Foi, A., Katkovnik, V., Egiazarian, K.: Image and video super-resolution via spatially adaptive block-matching filtering. In: Proceedings of International Workshop on Local and Non-local Approximation in Image Processing, Lausanne (2008)

    Google Scholar 

  23. De Bonet, J.S.: Multiresolution sampling procedure for analysis and synthesis of texture images. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, p. 368. ACM Press/Addison-Wesley (1997)

    Google Scholar 

  24. Delon, J., Desolneux, A.: Flicker stabilization in image sequences (2009). hal.archives-ouvertes.fr

    Google Scholar 

  25. Di Zenzo, S.: A note on the gradient of a multi-image. Comput. Vis. Graph. 33(1), 116–125 (1986)

    Article  MATH  Google Scholar 

  26. Dong, B., Ye, J., Osher, S., Dinov, I.: Level set based nonlocal surface restoration. Multiscale Model. Simul. 7, 589 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  27. Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  28. Durand, F., Dorsey, J.: Fast bilateral filtering for the display of highdynamic-range images. In: Proceedings of the 29th annual conference on Computer graphics and interactive techniques, ACM New York, pp. 257–266 (2002)

    Google Scholar 

  29. Ebrahimi, M., Vrscay, E.R.: Solving the Inverse Problem of Image Zooming Using “Self-Examples”. In: Kamel, M., Campilho, A. (eds.) Image Analysis and Recognition. Lecture Notes in Computer Science, vol. 4633, p. 117. Springer, Berlin/Heidelberg (2007)

    Google Scholar 

  30. Ebrahimi, M., Vrscay, E.R.: Multi-frame super-resolution with no explicit motion estimation. In: Proceedings of the 2008 International Conference on Image Processing, Computer Vision, and Pattern Recognition, Las Vegas, 2008

    Google Scholar 

  31. Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: International Conference on Computer Vision, Corful, vol. 2, pp. 1033–1038, 1999

    Google Scholar 

  32. Eisemann, E., Durand, F.: Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. (TOG) 23(3), 673–678 (2004)

    Google Scholar 

  33. Elad, M., Datsenko, D.: Example-based regularization deployed to superresolution reconstruction of a single image. Comput. J. 50, 1–16 (2007)

    Google Scholar 

  34. Elmoataz, A., Lezoray, O., Bougleux, S., Ta, V.T.: Unifying local and nonlocal processing with partial difference operators on weighted graphs. In: International Workshop on Local and Non-local Approximation in Image Processing, Lausanne (2008)

    Google Scholar 

  35. Fleishman, S., Drori, I., Cohen-Or, D.: Bilateral mesh denoising. ACM Trans. Graph. (TOG) 22(3), 950–953 (2003)

    Google Scholar 

  36. Gilboa, G., Osher, S.: Nonlocal linear image regularization and supervised segmentation. Multiscale Model. Simul. 6(2), 595–630 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  37. Grady, L.: Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1 (2006)

    Article  Google Scholar 

  38. Grady, L., Funka-Lea, G.: Multi-label image segmentation for medical applications based on graph-theoretic electrical potentials. In: Computer Vision and Mathematical Methods in Medical and Biomedical Image Analysis (ECCV), pp. 230–245 (2004)

    Google Scholar 

  39. Grady, L.J.: Space-variant computer vision: a graph-theoretic approach. PhD thesis, Boston University (2004)

    Google Scholar 

  40. Guichard, F., Morel, J.M., Ryan, R.: Contrast invariant image analysis and PDEs. Book in preparation

    Google Scholar 

  41. Harten, A., Engquist, B., Osher, S., Chakravarthy, S.R.: Uniformly high order accurate essentially non-oscillatory schemes, III. J. Comput. Phys. 71(2), 231–303 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  42. Huhle, B., Schairer, T., Jenke, P., Straßer, W.: Robust non-local denoising of colored depth data. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Anchorage, pp. 1–7 (2008)

    Google Scholar 

  43. Jones, T.R., Durand, F., Desbrun, M.: Non-iterative, feature-preserving mesh smoothing. ACM Trans. Graph. 22(3), 943–949 (2003)

    Article  Google Scholar 

  44. Jung, M., Vese, L.A.: Nonlocal variational image deblurring models in the presence of Gaussian or impulse noise. In: Tai, X.-C., Mørken, K., Lysaker, M., Lie, K.-A. (eds.) Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, vol. 5567. Springer, Berlin/Heidelberg (2009)

    Google Scholar 

  45. Kimia, B.B., Tannenbaum, A., Zucker, S.W.: On the evolution of curves via a function of curvature, I: the classical case. J. Math. Anal. Appl. 163(2), 438–458 (1992)

    MATH  MathSciNet  Google Scholar 

  46. Kimmel, R., Malladi, R., Sochen, N.: Images as embedded maps and minimal surfaces: movies, color, texture, and volumetric medical images. Int. J. Comput. Vis. 39(2), 111–129 (2000)

    Article  MATH  Google Scholar 

  47. Kindermann, S., Osher, S., Jones, P.W.: Deblurring and denoising of images by nonlocal functionals. Multiscale Model. Simul. 4(4), 1091–1115 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  48. Lee, J.S.: Digital image smoothing and the sigma filter. Comput. Vis. Graph. 24(2), 255–269 (1983)

    Article  Google Scholar 

  49. Lezoray, O., Ta, V.T., Elmoataz, A.: Nonlocal graph regularization for image colorization. In: International Conference on Pattern Recognition, Tampa (2008)

    Book  Google Scholar 

  50. Lou, Y., Zhang, X., Osher, S., Bertozzi, A.: Image recovery via nonlocal operators. J. Sci. Comput. 42(2), 185–197 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  51. Mairal, J., Elad, M., Sapiro, G., et al.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53 (2008)

    Article  MathSciNet  Google Scholar 

  52. Masnou, S.: Filtrage et désocclusion d’images par méthodes d’ensembles de niveau. PhD thesis, Ceremade, Universit e Paris-Dauphine (1998)

    Google Scholar 

  53. Mignotte, M.: A non-local regularization strategy for image deconvolution. Pattern Recognit. Lett. 29(16), 2206–2212 (2008)

    Article  Google Scholar 

  54. Osher, S., Rudin, L.I.: Feature-oriented image enhancement using shock filters. SIAM J Numer Anal 27(4), 919–940 (1990)

    Article  MATH  Google Scholar 

  55. Ozkan, M.K., Sezan, M.I., Tekalp, A.M.: Adaptive motion-compensated filtering of noisy image sequences. IEEE Trans. Circuits Syst. Video Technol. 3(4), 277–290 (1993)

    Article  Google Scholar 

  56. Peng, H., Rao, R., Messinger, D.W.: Spatio-spectral bilateral filters for hyperspectral imaging. In: SPIE Defense and Security Symposium. International Society for Optics and Photonics (2008)

    Google Scholar 

  57. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. PAMI 12(7), 629–639 (1990)

    Article  Google Scholar 

  58. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM Trans. Graph. (TOG) 23(2), 664–672 (2004)

    Google Scholar 

  59. Peyré, G.: Manifold models for signals and images. Comput. Vis. Image Underst. 113(2),249–260 (2009)

    Article  Google Scholar 

  60. Peyré, G.: Sparse modeling of textures. J. Math. Imaging Vis. 34(1), 17–31 (2009)

    Article  Google Scholar 

  61. Polzehl, J., Spokoiny, V.: Varying coefficient regression modeling by adaptive weights smoothing. WIAS preprint no. 818 (2003)

    Google Scholar 

  62. Protter, M., Elad, M., Takeda, H., Milanfar, P.: Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans. Image Process. 18(1), 35–51 (2009)

    MathSciNet  Google Scholar 

  63. Ramanath, R., Snyder, W.E.: Adaptive demosaicking. J. Electron. Imaging 12, 633 (2003)

    Article  Google Scholar 

  64. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1–4), 259–268 (1992)

    Article  MATH  Google Scholar 

  65. Sapiro, G., Ringach, D.L.: Anisotropic diffusion of multivalued images with applications to color filtering. IEEE Trans. Image Process. 5(11), 1582–1586 (1996)

    Article  Google Scholar 

  66. Sethian, J.A.: Curvature and the evolution of fronts. Commun. Math. Phys. 101(4), 487–499 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  67. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

    Article  Google Scholar 

  68. Smith, S.M., Brady, J.M.: SUSAN: a new approach to low level image processing. Int. J. Comput. Vis. 23(1), 45–78 (1997)

    Article  Google Scholar 

  69. Szlam, A.D., Maggioni, M., Coifman, R.R.: A general framework for adaptive regularization based on diffusion processes on graphs. Yale technical report (2006)

    Google Scholar 

  70. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the Sixth International Conference on Computer Vision, Bombay, pp. 839–846, 1998

    Google Scholar 

  71. van den Boomgaard, R., van de Weijer, J.: On the equivalence of localmode finding, robust estimation and mean-shift analysis as used in early vision tasks. In: International conference on pattern recognition, Quebec, vol. 16, Citeseer, pp. 927–930, 2002

    Google Scholar 

  72. Weickert, J.: Anisotropic Diffusion in Image Processing. B.G. Teubner, Stuttgart (1998). Citeseer

    Google Scholar 

  73. Winnemoller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. ACM Trans. Graph. (TOG) 25(3), 1226 (2006)

    Google Scholar 

  74. Wong, A., Orchard, J.: A nonlocal-means approach to exemplar-based inpainting. In: 15th IEEE International Conference on Image Processing, San Diego, 2008, pp. 2600–2603

    Google Scholar 

  75. Yaroslavsky, L.P.: Digital Picture Processing. Springer Secaucus. Springer, Berlin/New York (1985)

    Book  MATH  Google Scholar 

  76. Yaroslavsky, L.P.: Local adaptive image restoration and enhancement with the use of DFT and DCT in a running window. In: Unser, M.A., Aldroubi, A., Laine, A.F. (eds.) Wavelet Applications in Signal and Image Processing IV. Proceedings of SPIE, vol. 2825, p. 2. SPIE International Society for Optics and Photonics, Bellingham (1996)

    Google Scholar 

  77. Yaroslavsky, L.P., Egiazarian, K.O., Astola, J.T.: Transform domain image restoration methods: review, comparison, and interpretation. In: Dougherty, E.R., Astola, J.T. (eds.) Nonlinear Image Processing and Pattern Analysis XII. Proceedings of SPIE, vol. 4304, p. 155. SPIE International Society for Optics and Photonics, Bellingham (2001)

    Google Scholar 

  78. Yatziv, L., Sapiro, G.: Fast image and video colorization using chrominance blending. IEEE Trans. Image Process. 15(5), 1120–1129 (2006)

    Article  Google Scholar 

  79. Yoshizawa, S., Belyaev, A., Seidel, H.P.: Smoothing by example: Mesh denoising by averaging with similarity-based weights. In: IEEE International Conference on Shape Modeling and Applications, Matsushima, pp. 38–44, 2006

    Google Scholar 

  80. Zhang, D., Wang, Z.: Image information restoration based on longrange correlation. IEEE Trans. Circuits Syst. Video Technol. 12(5), 331–341 (2002)

    Article  Google Scholar 

  81. Zhang, X., Burger, M., Bresson, X., Osher, S.: Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. UCLA CAM report 09-03 (2009)

    Google Scholar 

  82. Zhu, S.C., Yuille, A.: Region competition: unifying snakes, region growing, and bayes/MDL for multiband image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 18(9), 884–900 (1996)

    Article  Google Scholar 

  83. Zhu, X., Lafferty, J., Ghahramani, Z.: Semi-supervised learning: from Gaussian fields to gaussian processes. School of Computer Science, Carnegie Mellon University (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. École Normale Supérieure de Cachan, Cachan, France

    Jean-Michel Morel

  2. Universitat Illes Balears, Palma de Mallorca, Spain

    Antoni Buades

  3. Universitat de les Illes Balears, Palma-Illes Balears, Spain

    Tomeu Coll

Authors
  1. Jean-Michel Morel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antoni Buades
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomeu Coll
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jean-Michel Morel .

Editor information

Editors and Affiliations

  1. Computational Science Center, University of Vienna, Vienna, Austria

    Otmar Scherzer

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media New York

About this entry

Cite this entry

Morel, JM., Buades, A., Coll, T. (2015). Local Smoothing Neighborhood Filters. In: Scherzer, O. (eds) Handbook of Mathematical Methods in Imaging. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-0790-8_26

Download citation

Publish with us

Policies and ethics

Search

Navigation