Skip to main content
SpringerLink
Log in

Orientation-Matching Minimization for Image Denoising and Inpainting

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper, we propose an orientation-matching functional minimization for image denoising and image inpainting. Following the two-step TV-Stokes algorithm (Rahman et al. in Scale space and variational methods in computer vision, pp. 473–482, Springer, Heidelberg, 2007; Tai et al. in Image processing based on partial differential equations, pp. 3–22, Springer, Heidelberg, 2006; Bertalmio et al. in Proc. conf. comp. vision pattern rec., pp. 355–362, 2001), a regularized tangential vector field with zero divergence condition is first obtained. Then a novel approach to reconstruct the image is proposed. Instead of finding an image that fits the regularized normal direction from the first step, we propose to minimize an orientation matching cost measuring the alignment between the image gradient and the regularized normal direction. This functional yields a new nonlinear partial differential equation (PDE) for reconstructing denoised and inpainted images. The equation has an adaptive diffusivity depending on the orientation of the regularized normal vector field, providing reconstructed images which have sharp edges and smooth regions. The additive operator splitting (AOS) scheme is used for discretizing Euler-Lagrange equations. We present the results of various numerical experiments that illustrate the improvements obtained with the new functional.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., & Verdera, J. (2001). Filling-in by joint interpolation of vector fields and gray levels. IEEE Transactions on Image Processing, 10(8), 1200–1211.

    Article  MathSciNet  MATH  Google Scholar 

  • Bertalmio, M. (2006). Strong-continuation contrast-invariant inpainting with a third-order optimal PDE. IEEE Transactions on Image Processing, 15(7), 1934–1938.

    Article  Google Scholar 

  • Bertalmio, M., Sapiro, G., Caselles, V., & Ballester, C. (2000). Image inpainting. In Computer graphics SIGGRAPH (pp. 417–424).

  • Bertalmio, M., Bertozzi, A. L., & Sapiro, G. (2001). Navier-Stokes, fluid dynamics, and image and video inpainting. In Proc. conf. comp. vision pattern rec. (pp. 355–362).

  • Bresson, X., & Chan, T. F. (2008). Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Problems and Imaging, 2(4), 455–484.

    Article  MathSciNet  MATH  Google Scholar 

  • Brox, T., Weickert, J., Burgeth, B., & Mrázek, P. (2006). Nonlinear structure tensors. Image Vision Computer, 24, 41–55.

    Article  Google Scholar 

  • Catté, F., Lions, P. L., Morel, J.-M., & Coll, T. (1992). Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 29(1), 182–193.

    Article  MathSciNet  MATH  Google Scholar 

  • Chan, T. F., & Shen, J. (2001). Nontexture inpainting by curvature driven diffusion (CDD). Journal of Visual Communication and Image Representation, 12, 436–449.

    Article  Google Scholar 

  • Chan, T. F., & Shen, J. (2002). Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics, 62(3), 1019–1043.

    Article  MathSciNet  MATH  Google Scholar 

  • Chan, T. F., & Shen, J. (2005). Variational image inpainting. Communications on Pure Applied Mathematics, 58, 579–619.

    Article  MathSciNet  MATH  Google Scholar 

  • Chan, T., & Vese, L. (2001). Active contours without edges. IEEE Transactions on Image Processing, 10(2), 266–277.

    Article  MATH  Google Scholar 

  • Chan, T. F., Kang, S.-H., & Shen, J. (2002). Euler’s elastica and curvature based inpaintings. SIAM Journal on Applied Mathematics, 63(2), 564–594.

    MathSciNet  MATH  Google Scholar 

  • Chan, R. H., Setzer, S., & Steidl, G. (2008). Inpainting by flexible Haar-wavelet shrinkage. SIAM Journal on Imaging. Science, 1, 273–293.

    Article  MathSciNet  MATH  Google Scholar 

  • Chessel, A., Cao, F., & Fablet, R. (2006). Interpolating orientation fields: An axiomatic approach. In ECCV06 (pp. IV, pp. 241–254).

  • Dong, F., Liu, Z., Kong, D., & Liu, K. (2009). An improved LOT model for image restoration. Journal of Mathematical Imaging and Vision, 34, 89–97.

    Article  MathSciNet  Google Scholar 

  • Hahn, J., & Lee, C.-O. (2009). A nonlinear structure tensor with the diffusivity matrix composed of the image gradient. Journal of Mathematical Imaging and Vision, 34, 137–151.

    Article  MathSciNet  Google Scholar 

  • Kimmel, R., & Bruckstein, A. M. (2001). Regularized Laplacian zero crossings as optimal edge integrators. International Journal of Computer Vision, 53, 225–243.

    Article  Google Scholar 

  • Kornprobst, P., & Aubert, G. (2006). Explicit reconstruction for image inpainting. In Research report N. 5905 INRIA.

  • Lindenbaum, M., Fischer, M., & Bruckstein, A. M. (1994). On Gabor’s contribution to image enhancement. Pattern Recognition, 27(1), 1–8.

    Article  Google Scholar 

  • Litvinov, W. G., Rahman, T., & Tai, X.-C. (2009). A modified TV-Stokes model for image processing. Technical report, Preprints—Herausgeber: Institut für Mathematik der Universität Augsburg 2009-25.

  • Lu, T., Neittaanmaki, P., & Tai, X.-C. (1992). A parallel splitting up method for partial differential equations and its application to Navier-Stokes equations. RAIRO Mathematical Modeling and Numerecal Analysis, 26(6), 673–708.

    MathSciNet  MATH  Google Scholar 

  • Lysaker, M., & Tai, X.-C. (2006). Iterative image restoration combining total variation minimization and a second-order functional. International Journal of Computer Vision, 66, 5–18.

    Article  Google Scholar 

  • Lysaker, M., Lundervold, A., & Tai, X.-C. (2003). Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Transaction on Image Processing, 12(12), 1579–1590.

    Article  Google Scholar 

  • Lysaker, M., Osher, S., & Tai, X.-C. (2004). Noise removal using smoothed normals and surface fitting. IEEE Transactions on Image Processing, 13(10), 1345–1357.

    Article  MathSciNet  Google Scholar 

  • Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th int’l conf. computer vision (Vol. 2, pp. 416–423), July 2001.

  • Masnou, S., & Morel, J.-M. (1998). Level lines based disocclusion. In Proc. IEEE int. conf. on image processing, Chicago, IL (pp. 259–263).

  • Perona, P. (1998). Orientation diffusion. IEEE Transactions on Image Processing, 7(3), 457–467.

    Article  Google Scholar 

  • Perona, P. & Malik, J. (1990). Scale space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7), 629–639.

    Article  Google Scholar 

  • Rahman, T., Tai, X.-C., & Osher, S. (2007). A TV-Stokes denoising algorithm. In Scale space and variational methods in computer vision (pp. 473–482). Heidelberg: Springer.

    Chapter  Google Scholar 

  • Rudin, L. I., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60, 259–268.

    Article  MATH  Google Scholar 

  • Sochen, N. A., Sagiv, C., & Kimmel, R. (2004). Stereographic combing a porcupine or studies on direction diffusion in image processing. SIAM Journal on Applied Mathematics, 64(5), 1477–1508.

    Article  MathSciNet  MATH  Google Scholar 

  • Spira, A., Kimmel, R., & Sochen, N. (2007). A short-time Beltrami kernel for smoothing images and manifolds. IEEE Transactions on Image Processing, 16, 1628–1636.

    Article  MathSciNet  Google Scholar 

  • Steidl, G., & Teuber, T. (2009). Anisotropic smoothing using double orientations. In SSVM ’09: proceedings of the second international conference on scale space and variational methods in computer vision (pp. 477–489). Berlin: Springer.

    Chapter  Google Scholar 

  • Tai, X.-C., Osher, S., & Holm, R. (2006). Image inpainting using TV-Stokes equation. In Image processing based on partial differential equations (pp. 3–22). Heidelberg: Springer.

    Google Scholar 

  • Tang, B., Sapiro, G., & Caselles, V. (2000). Diffusion of general data on non-flat manifolds via harmonic maps theory: The direction diffusion case. International Journal of Computer Vision, 36, 149–161.

    Article  Google Scholar 

  • Terzopoulos, D. (1988). The computation of visible-surface representations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4), 417–438.

    Article  MATH  Google Scholar 

  • Tschumperlé, D., & Deriche, R. (2002). Orthonormal vector sets regularization with PDE’s and applications. International Journal of Computer Vision, 50(3), 237–252.

    Article  MATH  Google Scholar 

  • Vese, L. A., & Osher, S. (2002). Numerical methods for p-harmonic flows and applications to image processing. SIAM Journal on Numerical Analysis, 40(6), 2085–2104.

    Article  MathSciNet  MATH  Google Scholar 

  • Weickert, J. (1999). Coherence-enhancing diffusion filtering. International Journal of Computer Vision, 31, 111–127.

    Article  Google Scholar 

  • Weickert, J., & Welk, M. (2006). Tensor field interpolation with PDEs. In Weickert, J., & Hagen, H. (Eds.) Visualization and processing of tensor fields (pp. 315–325). Berlin: Springer.

    Chapter  Google Scholar 

  • Weickert, J., ter Harr Romeny, B. M., & Viergever, M. A. (2001). Efficient and reliable schemes for nonlinear diffusion filtering. IEEE Transactions on Image Processing, 7, 398–410.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore, Singapore

    Jooyoung Hahn, Xue-Cheng Tai, Sofia Borok & Alfred Marcel Bruckstein

  2. Mathematics Institute, University of Bergen, Bergen, Norway

    Xue-Cheng Tai

  3. Department of Computer Science, Technion-Israel Institute of Technology, Haifa, Israel

    Alfred Marcel Bruckstein

Authors
  1. Jooyoung Hahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xue-Cheng Tai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sofia Borok
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alfred Marcel Bruckstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jooyoung Hahn.

Additional information

The research is supported by MOE (Ministry of Education) Tier II project T207N2202 and IDM project NRF2007IDMIDM002-010. In addition, the support from SUG 20/07 is also gratefully acknowledged. Professor A.M. Bruckstein’s work was supported in part by an NTU Joint visiting professorship at the School of Physical and Mathematical Sciences and the Institute for Media Innovations.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hahn, J., Tai, XC., Borok, S. et al. Orientation-Matching Minimization for Image Denoising and Inpainting. Int J Comput Vis 92, 308–324 (2011). https://doi.org/10.1007/s11263-010-0371-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-010-0371-5

Keywords

Search

Navigation