Skip to main content
SpringerLink
Log in

Multiplicative noise removal and blind inpainting of ultrasound images based on a new variational framework

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Image inpainting and denoising are two important preprocessing steps widely used in image and visual analysis. In this paper, by the maximum a posterior estimation, we present a new framework to remove multiplicative noise and artifacts simultaneously, when the locations of the artifacts/damaged pixels are unknown. By taking into account the statistical distribution of multiplicative noise as Gamma or Rayleigh noise, we give the special data fidelity term. To suppress the noise and repair the missing intensities, the proposed method applies spatial regularization to the desirable image, and \(\ell _{0}\) norm regularization to the artifacts. We introduce three typical spatial regularization: total variation, second-order total generalized variation (TGV) and fractional-order total variation (FOTV) for smoothing images. Due to the non-convexity and non-differentiability of the proposed minimization problem, we introduce additional auxiliary variables to simplify the original problem, and then use the alternating direction method of multipliers to solve it. A set of experiments on synthetic images and real medical ultrasound images show that the proposed method can efficiently remove the multiplicative noise, and more importantly fill in the missing pixels very well. Compared to other similar method, the TGV and FOTV regularization can not only preserve edges and texture details of the image but also avoid the staircase effect.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Lee, J.: Digital image enhancement and noise filtering by use of local statistics. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-2(2), 165–168 (1980). https://doi.org/10.1109/TPAMI.1980.4766994

  2. Frost, V., Stiles, J., Shanmugan, K., Holtzman, J.: A model for radar images and its application to adaptive digital filtering of multiplicative noise. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-4, 157–166 (1982). https://doi.org/10.1109/TPAMI.1982.4767223

  3. Kuan, D., Sawchuk, A., Strand, T., Chavel, P.: Adaptive noise smoothing filter for images with signal-dependent noise. IEEE Trans. Pattern Anal. Mach. Intell. 7, 165–177 (1985). https://doi.org/10.1109/TPAMI.1985.4767641

    Article  Google Scholar 

  4. Loupas, T., McDicken, W., Allan, P.: An adaptive weighted median filter for speckle suppression in medical ultrasonic images. IEEE Trans. Circuits Syst. 36, 129–135 (1989). https://doi.org/10.1109/31.16577

    Article  Google Scholar 

  5. Coupé, P., Hellier, P., Kervrann, C., Barillot, C.: Nonlocal means-based speckle filtering for ultrasound images. IEEE Trans. Image Process. 18, 2221–2229 (2009). https://doi.org/10.1109/TIP.2009.2024064

    Article  MathSciNet  MATH  Google Scholar 

  6. Deledalle, C., Denis, L., Tupin, F.: Iterative weighted maximum likelihood denoising with probabilistic patch-based weights. IEEE Trans. Image Process. 18, 2661–2672 (2010). https://doi.org/10.1109/TIP.2009.2029593

    Article  MathSciNet  MATH  Google Scholar 

  7. Aubert, G., Aujol, J.F.: A variational approach to removing multiplicative noise. SIAM J. Appl. Math. 68, 925–946 (2008). https://doi.org/10.1137/060671814

    Article  MathSciNet  MATH  Google Scholar 

  8. Shi, J., Osher, S.: A nonlinear inverse scale space method for a convex multiplicative noise model. SIAM J. Imaging Sci. 1, 294–321 (2008). https://doi.org/10.1137/070689954

    Article  MathSciNet  MATH  Google Scholar 

  9. Wen, Y.W., Ng, M., Wen, Y.W.: A new total variation method for multiplicative noise removal. SIAM J. Imaging Sci. 2, 20–40 (2009). https://doi.org/10.1137/080712593

    Article  MathSciNet  MATH  Google Scholar 

  10. Rudin, L., Lions, P., Osher, S.: Multiplicative denoising and deblurring: Theory and algorithms. Geom. Level Set Methods Imaging Vis. Graphics 4, 103–120 (2003). https://doi.org/10.1007/0-387-21810-6_6

    Article  MathSciNet  Google Scholar 

  11. Steidl, G., Teuber, T.: Removing multiplicative noise by Douglas–Rachford splitting methods. J. Math. Imaging Vis. 36, 168–184 (2010). https://doi.org/10.1007/s10851-009-0179-5

    Article  MathSciNet  MATH  Google Scholar 

  12. Shen, C., Pi, L., Peng, Y., Li, Z.: Variational-based speckle noise removal of SAR imagery. In: 2007 IEEE International Geoscience and Remote Sensing Symposium, pp. 532–535 (2007). https://doi.org/10.1109/IGARSS.2007.4422848

  13. Li, F., Ng, M., Shen, C.: Multiplicative noise removal with spatially varying regularization parameters. SIAM J. Imaging Sci. 3, 1–20 (2010). https://doi.org/10.1137/090748421

    Article  MathSciNet  MATH  Google Scholar 

  14. Seabra, J., Xavier, J.a., Sanches, J.a.: Convex ultrasound image reconstruction with Log–Euclidean priors. In: 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society 2008, 435–438 (2008). https://doi.org/10.1109/IEMBS.2008.4649183

  15. Lu, J., Yang, Z., Shen, L., Lu, Z., Yang, H., Xu, C.: A framelet algorithm for de-blurring images corrupted by multiplicative noise. Appl. Math. Model. 62, 51–61 (2018). https://doi.org/10.1016/j.apm.2018.05.007

    Article  MathSciNet  MATH  Google Scholar 

  16. Li, C., Ren, Z., Tang, L.: Multiplicative noise removal via using nonconvex regularizers based on total variation and wavelet frame. J. Comput. Appl. Math. 370, 112684 (2019). https://doi.org/10.1016/j.cam.2019.112684

    Article  MathSciNet  MATH  Google Scholar 

  17. Dong, F., Zhang, H., Kong, D.X.: Nonlocal total variation models for multiplicative noise removal using split Bregman iteration. Math. Comput. Model. 55, 939–954 (2012). https://doi.org/10.1016/j.mcm.2011.09.021

    Article  MathSciNet  MATH  Google Scholar 

  18. Liu, P.: Hybrid higher-order total variation model for multiplicative noise removal. IET Image Process. 14(5), 862–873 (2020). https://doi.org/10.1049/iet-ipr.2018.5930

    Article  Google Scholar 

  19. Shama, M., Huang, T., Liu, J., Wang, S.: A convex total generalized variation regularized model for multiplicative noise and blur removal. Appl. Math. Comput. 276, 109–121 (2016). https://doi.org/10.1016/j.amc.2015.12.005

    Article  MATH  Google Scholar 

  20. Tian, D., Du, Y., Chen, D.: An adaptive fractional-order variation method for multiplicative noise removal. J. Inf. Sci. Eng. 32, 747–762 (2016)

    MathSciNet  Google Scholar 

  21. Chen, G., Li, G., Liu, Y., Zhang, X.P., Zhang, L.: Sar image despeckling based on combination of fractional-order total variation and nonlocal low rank regularization. IEEE Trans. Geosci. Remote Sens. 58, 483–491 (2020)

    Google Scholar 

  22. Gao, Y., Yang, X.: Tgv-based multiplicative noise removal approach: Models and algorithms. J. Inverse Ill-posed Probl. 26, 703–727 (2018). https://doi.org/10.1515/jiip-2016-0051

    Article  MathSciNet  MATH  Google Scholar 

  23. Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3, 492–526 (2010). https://doi.org/10.1137/090769521

    Article  MathSciNet  MATH  Google Scholar 

  24. Na, H., Kang, M., Jung, M., Kang, M.: Nonconvex tgv regularization model for multiplicative noise removal with spatially varying parameters. Inverse Probl. Imaging 13, 117–147 (2019). https://doi.org/10.3934/ipi.2019007

    Article  MathSciNet  MATH  Google Scholar 

  25. Milici, C., Draganescu, G., Tenreiro Machado, J.: Fractional Differential Equations, pp. 47–86 (2019). https://doi.org/10.1007/978-3-030-00895-6_4

  26. Barcelos, C., Batista, M.: Image restoration using digital inpainting and noise removal. Image Vis. Comput. 25, 61–69 (2007). https://doi.org/10.1016/j.imavis.2005.12.008

    Article  Google Scholar 

  27. Rodríguez, P., Rojas, R., Wohlberg, B.: Mixed gaussian-impulse noise image restoration via total variation. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1077–1080 (2012). https://doi.org/10.1109/ICASSP.2012.6288073

  28. Thai, D., Gottschlich, C.: Simultaneous inpainting and denoising by directional global three-part decomposition: connecting variational and fourier domain based image processing. R. Soc. Open Sci. 5, 171–176 (2016). https://doi.org/10.1098/rsos.171176

    Article  Google Scholar 

  29. Dong, B., Ji, H., Li, J., Shen, Z., Xu, Y.: Wavelet frame based blind image inpainting. Appl. Comput. Harmon. Anal. 32, 268–279 (2012). https://doi.org/10.1016/j.acha.2011.06.001

  30. Yang, J.: A tv-based approach to blind image inpainting. In: 2011 4th International Congress on Image and Signal Processing 2, 779–781 (2011). https://doi.org/10.1109/CISP.2011.6100364

  31. Yan, M.: Restoration of images corrupted by impulse noise and mixed gaussian impulse noise using blind inpainting. SIAM J. Imaging Sci. 6, 1227–1245 (2013). https://doi.org/10.1137/12087178X

    Article  MathSciNet  MATH  Google Scholar 

  32. Wang, Y., Szlam, A., Lerman, G.: Robust locally linear analysis with applications to image denoising and blind inpainting. SIAM J. Imaging Sci. 6, 526–562 (2013). https://doi.org/10.1137/110843642

    Article  MathSciNet  MATH  Google Scholar 

  33. Shen, Y., Han, B., Braverman, E.: Removal of mixed gaussian and impulse noise using directional tensor product complex tight framelets. J. Math. Imaging Vis. 54, 64–77 (2015). https://doi.org/10.1007/s10851-015-0589-5

    Article  MathSciNet  MATH  Google Scholar 

  34. Setzer, S.: Split Bregman algorithm, Douglas–Rachford splitting and frame shrinkage. In: Proceeding of Scale Space and Variational Methods in Computer Vision, Second International Conference, vol. 5567, pp. 464–476 (2009). https://doi.org/10.1007/978-3-642-02256-2_39

  35. Afonso, M.V., Sanches, J.M.R.: Blind inpainting using \(\ell \_{0}\) and total variation regularization. IEEE Trans. Image Process. 24(7), 2239–2253 (2015). https://doi.org/10.1109/TIP.2015.2417505

    Article  MathSciNet  MATH  Google Scholar 

  36. Eckstein, J., Bertsekas, D., Systems, M.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992). https://doi.org/10.1007/BF01581204

    Article  MathSciNet  MATH  Google Scholar 

  37. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. (2011). https://doi.org/10.1007/s10851-010-0251-1

    Article  MathSciNet  MATH  Google Scholar 

  38. Blumensath, T., Davies, M.E.: Iterative thresholding for sparse approximations. J. Fourier Anal. Appl. 14, 629–654 (2008). https://doi.org/10.1007/s00041-008-9035-z

    Article  MathSciNet  MATH  Google Scholar 

  39. Knoll, F., Bredies, K., Pock, T., Stollberger, R.: Second order total generalized variation (tgv) for mri. Magnetic resonance in medicine. Off. J. Soc. Magn. Reson. Med. 65, 480–91 (2011). https://doi.org/10.1002/mrm.22595

  40. Chen, D., Chen, Y., Xue, D.: Fractional-order total variation image restoration based on primal-dual algorithm. Abstr. Appl. Anal. (2013). https://doi.org/10.1155/2013/585310

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Statistics and Mathematics, Zhejiang Gongshang University, Hangzhou, 310018, China

    Fangfang Dong & Nannan Li

Authors
  1. Fangfang Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nannan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fangfang Dong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by Natural Science Foundation of Zhejiang Province, China (Grant No. LY20A010001)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, F., Li, N. Multiplicative noise removal and blind inpainting of ultrasound images based on a new variational framework. Machine Vision and Applications 32, 86 (2021). https://doi.org/10.1007/s00138-021-01214-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01214-5

Keywords

Search

Navigation