Skip to main content

Advertisement

SpringerLink
Log in

Patch-based models and algorithms for image processing: a review of the basic principles and methods, and their application in computed tomography

  • Review Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Image models are central to all image processing tasks. The great advancements in digital image processing would not have been made possible without powerful models which, themselves, have evolved over time. In the past decade, “patch-based” models have emerged as one of the most effective models for natural images. Patch-based methods have outperformed other competing methods in many image processing tasks. These developments have come at a time when greater availability of powerful computational resources and growing concerns over the health risks of the ionizing radiation encourage research on image processing algorithms for computed tomography (CT). The goal of this paper is to explain the principles of patch-based methods and to review some of their recent applications in CT.

Methods

We first review the central concepts in patch-based image processing and explain some of the state-of-the-art algorithms, with a focus on aspects that are more relevant to CT. Then, we review some of the recent application of patch-based methods in CT.

Results

Patch-based methods have already transformed the field of image processing, leading to state-of-the-art results in many applications. More recently, several studies have proposed patch-based algorithms for various image processing tasks in CT, from denoising and restoration to iterative reconstruction. Although these studies have reported good results, the true potential of patch-based methods for CT has not been yet appreciated.

Conclusions

Patch-based methods can play a central role in image reconstruction and processing for CT. They have the potential to lead to substantial improvements in the current state of the art.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aharon M, Elad M (2008) Sparse and redundant modeling of image content using an image-signature-dictionary. SIAM J Imaging Sci 1(3):228–247

    Article  Google Scholar 

  2. Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    Article  Google Scholar 

  3. AlAfeef A, Cockshott P, MacLaren I, McVitie S (2014) Compressed sensing electron tomography using adaptive dictionaries: a simulation study. J Phys: Conf Ser 522(1):012021

    Google Scholar 

  4. Bach F, Jenatton R, Mairal J, Obozinski G (2012) Optimization with sparsity-inducing penalties. Found Trends Mach Learn 4(1):1–106

    Article  Google Scholar 

  5. Barnes C, Shechtman E, Goldman D, Finkelstein A (2010) The generalized patchmatch correspondence algorithm. In: Daniilidis K, Maragos P, Paragios N (eds) Computer vision ECCV 2010, vol 6313, lecture notes in computer ScienceSpringer, Berlin, pp 29–43

  6. Borsdorf A, Köstler H, Rubinstein R, Bartuschat D, Strmer M (2009) A parallel K-SVD implementation for CT image denoising. Technical Report CS 10, University of Erlangen-Nurnberg

  7. Bian Z, Ma J, Huang J, Zhang H, Niu S, Feng Q, Liang Z, Chen W (2013) Sr-nlm: a sinogram restoration induced non-local means image filtering for low-dose computed tomography. Comput Med Imaging Graph 37(4):293–303

    Article  PubMed  PubMed Central  Google Scholar 

  8. Buades A, Coll B, Morel J-M (2005) A review of image denoising algorithms, with a new one. Multiscale Model Simul 4(2):490–530

    Article  Google Scholar 

  9. Buades A, Coll B, Morel J-M (2006) Image enhancement by non-local reverse heat equation. Preprint CMLA 22:2006

    Google Scholar 

  10. Chainais P (Sept 2012) Towards dictionary learning from images with non gaussian noise. In: 2012 IEEE international workshop on machine learning for signal processing (MLSP), pp 1–6

  11. Chatterjee P, Milanfar P (2010) Is denoising dead? IEEE Trans Image Process 19(4):895–911

    Article  PubMed  Google Scholar 

  12. Chatterjee P, Milanfar P (2012) Patch-based near-optimal image denoising. IEEE Trans Image Process 21(4):1635–1649

    Article  PubMed  Google Scholar 

  13. Chen Y, Chen W, Yin X, Ye X, Bao X, Luo L, Feng Q, li Y, Yu X (2011) Improving low-dose abdominal CT images by weighted intensity averaging over large-scale neighborhoods. Eur J Radiol 80(2):e42–e49

    Article  PubMed  Google Scholar 

  14. Chen Y, Gao D, Nie C, Luo L, Chen W, Yin X, Lin Y (2009) Bayesian statistical reconstruction for low-dose X-ray computed tomography using an adaptive-weighting nonlocal prior. Comput Med Imaging Graph 33(7):495–500

    Article  PubMed  Google Scholar 

  15. Chen Y, Shi L, Feng Q, Yang J, Shu H, Luo L, Coatrieux J-L, Chen W (2014) Artifact suppressed dictionary learning for low-dose CT image processing. IEEE Trans Med Imaging 33(12):2271–2292

    Article  PubMed  Google Scholar 

  16. Chen Y, Shi L, Yang J, Hu Y, Luo L, Yin X, Coatrieux J-L (2014) Radiation dose reduction with dictionary learning based processing for head CT. Australas Phys Eng Sci Med 37(3):483–493

    Article  PubMed  Google Scholar 

  17. Chen Y, Yang Z, Hu Y, Yang G, Zhu Y, Li Y, luo L, Chen W, Toumoulin C (2012) Thoracic low-dose CT image processing using an artifact suppressed large-scale nonlocal means. Phys Med Biol 57(9):2667

    Article  PubMed  Google Scholar 

  18. Chen Y, Yin X, Shi L, Shu H, Luo L, Coatrieux J-L, Toumoulin C (2013) Improving abdomen tumor low-dose CT images using a fast dictionary learning based processing. Phys Med Biol 58(16):5803

    Article  PubMed  Google Scholar 

  19. Couzinie-Devy F, Mairal J, Bach F, Ponce J (2011) Dictionary learning for deblurring and digital zoom. Preprint. arXiv:1110.0957

  20. Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans Image Process 16(8):2080–2095

    Article  PubMed  Google Scholar 

  21. Darbon J, Cunha A, Chan T, Osher S, Jensen G (2008) Fast nonlocal filtering applied to electron cryomicroscopy. In: 5th IEEE international symposium on biomedical imaging: from Nano to Macro, 2008. ISBI 2008, pp 1331–1334, May

  22. Deledalle C-A, Denis L, Tupin F (2012) How to compare noisy patches? patch similarity beyond gaussian noise. Int J Comput Vis 99(1):86–102

    Article  Google Scholar 

  23. Deledalle C-A, Duval V, Salmon J (2012) Non-local methods with shape-adaptive patches (NLM-SAP). J Math Imaging Vis 43(2):103–120

    Article  Google Scholar 

  24. Deledalle C-A, Salmon J, Dalalyan A (2011) Image denoising with patch based PCA: local versus global. BMVC 81:425–455

    Google Scholar 

  25. Deledalle C-A, Tupin F, Denis L (Sept 2010) Poisson NL means: unsupervised non local means for poisson noise. In: 17th IEEE international conference on image processing (ICIP), 2010, pp 801–804

  26. Deledalle C-A, Tupin F, Denis L (Sept 2011) Patch similarity under non Gaussian noise. In: International conference on image processing. Brussels, pp 1845–1848

  27. Dong W, Zhang D, Shi G, Wu X (2011) Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans Image Process 20(7):1838–1857

    Article  PubMed  Google Scholar 

  28. Dore V, Cheriet M (2009) Robust NL-means filter with optimal pixel-wise smoothing parameter for statistical image denoising. IEEE Trans Signal Process 57(5):1703–1716

    Article  Google Scholar 

  29. Dupe F-X, Anthoine S (Sept 2013) A greedy approach to sparse poisson denoising. In: 2013 IEEE international workshop on machine learning for signal processing (MLSP), pp 1–6

  30. Elad M (2010) Sparse and redundant representations: from theory to applications in signal and image processing. Springer, Berlin

    Book  Google Scholar 

  31. Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745

    Article  PubMed  Google Scholar 

  32. Engan K, Aase S, Hakon Husoy J (1999) Method of optimal directions for frame design. In 1999. Proceedings, 1999 IEEE international conference on acoustics, speech, and signal processing, vol 5, pp 2443–2446

  33. Etter V, Jovanovic I, Vetterli M (2011) Use of learned dictionaries in tomographic reconstruction. In: SPIE optical engineering+ applications, International Society for Optics and Photonics, pp 81381C–81381C-11

  34. Giryes R, Elad M (2013) Sparsity based poisson denoising with dictionary learning. Preprint. arXiv:1309.4306

  35. Gregor K, Lecun Y (2010) Learning fast approximations of sparse coding. In: 2010 international conference on machine learning (ICML), Omnipress, pp 1–8

  36. Hawe S, Seibert M, Kleinsteuber M (2013) Separable dictionary learning. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR), IEEE, pp 438–445

  37. Huang D-A, Wang Y-CF (2013) Coupled dictionary and feature space learning with applications to cross-domain image synthesis and recognition. In: 2013 IEEE international conference on computer vision (ICCV), IEEE, pp 2496–2503

  38. Huang J, Ma J, Liu N, Zhang H, Bian Z, Feng Y, Feng Q, Chen W (2011) Sparse angular CT reconstruction using non-local means based iterative-correction POCS. Comput Biol Med 41(4):195–205

    Article  PubMed  Google Scholar 

  39. Jenatton R, Mairal J, Bach FR, Obozinski GR (2010) Proximal methods for sparse hierarchical dictionary learning. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 487–494

  40. Jia X, Tian Z, Lou Y, Sonke J-J, Jiang SB (2012) Four-dimensional cone beam CT reconstruction and enhancement using a temporal nonlocal means method. Med Phys 39(9):5592–5602

    Article  PubMed  PubMed Central  Google Scholar 

  41. Kang L-W, Lin C-W, Fu Y-H (2012) Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans Image Process 21(4):1742–1755

    Article  PubMed  Google Scholar 

  42. Karimi D, Ward RK (2016) A novel structured dictionary for fast processing of 3D medical images, with application to computed tomography restoration and denoising. In: SPIE medical imaging, International Society for Optics and Photonics, pp 97840N–97840N-7

  43. Karimi D, Ward R (2016) Reducing streak artifacts in computed tomography via sparse representation in coupled dictionaries. Med Phys 43(3):1473–1486

    Article  PubMed  Google Scholar 

  44. Karimi D, Ward R, Ford N (Sept 2015) Angular upsampling of projection measurements in 3D computed tomography using a sparsity prior. In: 2015 IEEE international conference on image processing (ICIP), pp 3363–3367

  45. Karimi D, Ward RK (2016) Sinogram denoising via simultaneous sparse representation in learned dictionaries. Phys Med Biol 61(9):3536–3553

    Article  PubMed  Google Scholar 

  46. Kavukcuoglu K, Ranzato M, Fergus R, Le-Cun Y (June 2009) Learning invariant features through topographic filter maps. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009, pp 1605–1612

  47. Kazantsev D, Thompson WM, Van Eyndhoven G, Dobson K, Kaestner AP, Lionheart W, Withers PJ, Lee PD (2015) 4D-CT reconstruction with unified spatial-temporal patch-based regularization. Inverse Probl. Imaging 9:447–467

    Article  Google Scholar 

  48. Kazantsev D, Van Eyndhoven G, Lionheart W, Withers P, Dobson K, McDonald S, Atwood R, Lee P (2015) Employing temporal self-similarity across the entire time domain in computed tomography reconstruction. Philos Trans R Soc Lond A 373(2043):20140389

    Article  Google Scholar 

  49. Kelm Z, Blezek D, Bartholmai B, Erickson B (June 2009) Optimizing non-local means for denoising low dose CT. In: IEEE international symposium on biomedical imaging: from Nano to Macro, 2009. ISBI ’09, pp 662–665

  50. Kindermann S, Osher S, Jones PW (2005) Deblurring and denoising of images by nonlocal functionals. Multiscale Model Simul 4(4):1091–1115

    Article  Google Scholar 

  51. Kumar N, Zhang L, Nayar SK (Oct 2008) What is a good nearest neighbors algorithm for finding similar patches in images? In: European conference on computer vision (ECCV), pp 364–378

  52. Lebrun M, Buades A, Morel JM (2013) A nonlocal Bayesian image denoising algorithm. SIAM J Imaging Sci 6(3):1665–1688

    Article  Google Scholar 

  53. Lee H, Battle A, Raina R, Ng AY (2006) Efficient sparse coding algorithms. In: Advances in neural information processing systems, pp 801–808

  54. Lewicki MS, Olshausen BA (1999) Probabilistic framework for the adaptation and comparison of image codes. JOSA A 16(7):1587–1601

    Article  Google Scholar 

  55. Li S, Cao Q, Chen Y, Hu Y, Luo L, Toumoulin C (2014) Dictionary learning based sinogram inpainting for CT sparse reconstruction. Optik Int J Light Electron Optics 125(12):2862–2867

    Article  Google Scholar 

  56. Li S, Fang L, Yin H (2012) An efficient dictionary learning algorithm and its application to 3-D medical image denoising. IEEE Trans Biomed Eng 59(2):417–427

    Article  PubMed  Google Scholar 

  57. Li Z, Yu L, Trzasko JD, Lake DS, Blezek DJ, Fletcher JG, McCollough CH, Manduca A (2014) Adaptive nonlocal means filtering based on local noise level for CT denoising. Med Phys 41(1):011908

    Article  PubMed  Google Scholar 

  58. Liu B, Yu H, Verbridge SS, Sun L, Wang G (2014) Dictionary-learning-based reconstruction method for electron tomography. Scanning 36(4):377–383

    Article  PubMed  PubMed Central  Google Scholar 

  59. Lou Y, Zhang X, Osher S, Bertozzi A (2010) Image recovery via nonlocal operators. J Sci Comput 42(2):185–197

    Article  Google Scholar 

  60. Lu Y, Zhao J, Wang G (2012) Few-view image reconstruction with dual dictionaries. Phys Med Biol 57(1):173

    Article  PubMed  PubMed Central  Google Scholar 

  61. Ma J, Huang J, Feng Q, Zhang H, Lu H, Liang Z, Chen W (2011) Low-dose computed tomography image restoration using previous normal-dose scan. Med Phys 38(10):5713–5731

    Article  PubMed  PubMed Central  Google Scholar 

  62. Ma J, Zhang H, Gao Y, Huang J, Liang Z, Feng Q, Chen W (2012) Iterative image reconstruction for cerebral perfusion CT using a pre-contrast scan induced edge-preserving prior. Phys Med Biol 57(22):7519

    Article  PubMed  PubMed Central  Google Scholar 

  63. Maggioni M, Katkovnik V, Egiazarian K, Foi A (2013) A nonlocal transform-domain filter for volumetric data denoising and reconstruction. IEEE Trans Image Process 22(1):1057–7149

    Article  Google Scholar 

  64. Mairal J, Bach F, Ponce J (2012) Task-driven dictionary learning. IEEE Trans Pattern Anal Mach Intell (PAMI) 34(4):791–804

    Article  Google Scholar 

  65. Mairal J, Bach F, Ponce J (2014) Sparse modeling for image and vision processing. Found Trends Comput Graph Vis 8(2–3):85–283

    Article  Google Scholar 

  66. Mairal J, Bach F, Ponce J, Sapiro G (2010) Online learning for matrix factorization and sparse coding. J Mach Learn Res (JMLR) 11:19–60

    Google Scholar 

  67. Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2009) Non-local sparse models for image restoration. In: International conference on computer vision (ICCV). I

  68. Mairal J, Sapiro G, Elad M (2008) Learning multiscale sparse representations for image and video restoration. Multiscale Model Simul 7(1):214–241

    Article  Google Scholar 

  69. Peyr G, Bougleux S, Cohen L (2008) Non-local regularization of inverse problems. In: Forsyth D, Torr P, Zisserman A (eds) Computer vision ECCV 2008, vol 5304, Lecture notes in computer ScienceSpringer, Berlin, pp 57–68

  70. Pfister L, Bresler Y (2014) Model-based iterative tomographic reconstruction with adaptive sparsifying transforms . In: IS&T/SPIE electronic imaging, International Society for Optics and Photonics, pp 90200H–90200H

  71. Protter M, Elad M, Takeda H, Milanfar P (2009) Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans Image Process 18(1):36–51

    Article  PubMed  Google Scholar 

  72. Ramirez I, Sprechmann P, Sapiro G (June 2010) Classification and clustering via dictionary learning with structured incoherence and shared features. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR), pp 3501–3508

  73. Rubinstein R, Zibulevsky M, Elad M (2010) Double sparsity: learning sparse dictionaries for sparse signal approximation. IEEE Trans Signal Process 58(3):1553–1564

    Article  Google Scholar 

  74. Sakhaee E, Entezari A (2014) Learning splines for sparse tomographic reconstruction. In: Bebis G, Boyle R, Parvin B, Koracin D, McMahan R, Jerald J, Zhang H, Drucker S, Kambhamettu C, El Choubassi M, Deng Z, Carlson M (eds) Advances in visual computing, vol 8887. Lecture notes in computer science. Springer International Publishing, pp 1–10

  75. Salmon J, Harmany Z, Deledalle C-A, Willett R (2014) Poisson noise reduction with non-local PCA. J Math Imaging Vis 48(2):279–294

    Article  Google Scholar 

  76. Shtok J, Elad M, Zibulevsky M (2013) Learned shrinkage approach for low-dose reconstruction in computed tomography. J Biomed Imaging 2013:7

    Google Scholar 

  77. Shtok J, Elad P, Zibulevsky M (May 2011) Sparsity-based sinogram denoising for low-dose computed tomography. In: 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 569–572

  78. Soltani S (2015) Studies of sensitivity in the dictionary learning approach to computed tomography: simplifying the reconstruction problem, rotation, and scale. DTU Compute-Technical Report-2015. Technical University of Denmark

  79. Soltani S, Andersen MS, Hansen PC (2015) Tomographic image reconstruction using dictionary priors. CoRR, abs/1503.01993

  80. Stojanovic I, Pien H, Do S, Karl W (May 2012) Low-dose X-ray CT reconstruction based on joint sinogram smoothing and learned dictionary-based representation. In: 2012 9th IEEE international symposium on biomedical imaging (ISBI), pp 1012–1015

  81. Tan S, Zhang Y, Wang G, Mou X, Cao G, Wu Z, Yu H (2015) Tensor-based dictionary learning for dynamic tomographic reconstruction. Phys Med Biol 60(7):2803

    Article  PubMed  PubMed Central  Google Scholar 

  82. Tian Z, Jia X, Dong B, Lou Y, Jiang SB (2011) Low-dose 4DCT reconstruction via temporal nonlocal means. Med Phys 38(3):1359–1365

    Article  PubMed  Google Scholar 

  83. Wang S, Zhang L, Liang Y, Pan Q (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In 2012 IEEE conference on computer vision and pattern recognition (CVPR), IEEE, pp 2216–2223

  84. Xu Q, Yu H, Mou X, Zhang L, Hsieh J, Wang G (2012) Low-dose X-ray CT reconstruction via dictionary learning. IEEE Trans Med Imaging 31(9):1682–1697

    Article  PubMed  PubMed Central  Google Scholar 

  85. Xu W, Ha S, Mueller K (2013) Database-assisted low-dose CT image restoration. Med Phys 40(3):031109

    Article  PubMed  Google Scholar 

  86. Xu W, Mueller K (2012) Efficient low-dose CT artifact mitigation using an artifact-matched prior scan. Med Phys 39(8):4748–4760

    Article  PubMed  Google Scholar 

  87. Yan H, Zhen X, Cervio L, Jiang SB, Jia X (2013) Progressive cone beam CT dose control in image-guided radiation therapy. Med Phys 40(6):060701

    Article  PubMed  PubMed Central  Google Scholar 

  88. Yang J, Wright J, Huang T, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  PubMed  Google Scholar 

  89. Zhang H, Huang J, Ma J, Bian Z, Feng Q, Lu H, Liang Z, Chen W (2014) Iterative reconstruction for X-ray computed tomography using prior-image induced nonlocal regularization. IEEE Trans Biomed Eng 61(9):2367–2378

    Article  PubMed  Google Scholar 

  90. Zhang H, Ma J, Wang J, Liu Y, Lu H, Liang Z (2014) Statistical image reconstruction for low-dose CT using nonlocal means-based regularization. Comput Med Imaging Graph 38(6):423–435

    Article  PubMed  PubMed Central  Google Scholar 

  91. Zhang X, Burger M, Bresson X, Osher S (2010) Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J Imaging Sci 3(3):253–276

    Article  Google Scholar 

  92. Zhao B, Ding H, Lu Y, Wang G, Zhao J, Molloi S (2012) Dual-dictionary learning-based iterative image reconstruction for spectral computed tomography application. Phys Med Biol 57(24):8217

    Article  PubMed  PubMed Central  Google Scholar 

  93. Zhou M, Chen H, Paisley J, Ren L, Sapiro G, Carin L (2009) Non-parametric Bayesian dictionary learning for sparse image representations 1. In: NIPS 2009

Download references

Author information

Authors and Affiliations

  1. 2366 Main Mall, Vancouver, BC, V6T 1Z4, Canada

    Davood Karimi & Rabab K. Ward

Authors
  1. Davood Karimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rabab K. Ward
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Davood Karimi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This articles does not contain patient data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karimi, D., Ward, R.K. Patch-based models and algorithms for image processing: a review of the basic principles and methods, and their application in computed tomography. Int J CARS 11, 1765–1777 (2016). https://doi.org/10.1007/s11548-016-1434-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-016-1434-z

Keywords

Search

Navigation