Skip to main content

Microscopy Image Restoration with Deep Wiener-Kolmogorov Filters

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12365))

Included in the following conference series:

  • 4184 Accesses

Abstract

Microscopy is a powerful visualization tool in biology, enabling the study of cells, tissues, and the fundamental biological processes; yet, the observed images typically suffer from blur and background noise. In this work, we propose a unifying framework of algorithms for Gaussian image deblurring and denoising. These algorithms are based on deep learning techniques for the design of learnable regularizers integrated into the Wiener-Kolmogorov filter. Our extensive experimentation line showcases that the proposed approach achieves a superior quality of image reconstruction and surpasses the solutions that rely either on deep learning or on optimization schemes alone. Augmented with the variance stabilizing transformation, the proposed reconstruction pipeline can also be successfully applied to the problem of Poisson image deblurring, surpassing the state-of-the-art methods. Moreover, several variants of the proposed framework demonstrate competitive performance at low computational complexity, which is of high importance for real-time imaging applications .

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. PSF Generator. http://bigwww.epfl.ch/algorithms/psfgenerator/#ref. Accessed 25 Feb 2020

  2. Diffraction PSF 3D. https://www.optinav.info/Diffraction-PSF-3D.htm. Accessed 30 May 2019

  3. Al-Kofahi, Y., Zaltsman, A.B., Graves, R.M., Marshall, W., Rusu, M.: A deep learning-based algorithm for 2-D cell segmentation in microscopy images. BMC Bioinform. 19 (2018)

    Google Scholar 

  4. AL-Qinani, I.H.: Deblurring image and removing noise from medical images for cancerous diseases using a wiener filter. IRJET 8(4), 2354–2365 (2017)

    Google Scholar 

  5. Anscombe, F.J.: The transformation of Poisson, binomial and negative-binomial data. Biometrika 35(3–4), 246–254 (1948)

    Article  MathSciNet  Google Scholar 

  6. Arjomand Bigdeli, S., Zwicker, M., Favaro, P., Jin, M.: Deep mean-shift priors for image restoration. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 763–772. Curran Associates, Inc. (2017)

    Google Scholar 

  7. van Beek, P., Yang, J., Yamamoto, S., Ueda, Y.: Image deblurring and denoising with non-local regularization constraint. In: Information Processing and Communications, vol. 7543, January 2010

    Google Scholar 

  8. Bertsekas, D.P.: Nonlinear Programming, 2nd edn. (1999)

    Google Scholar 

  9. Boyat, A.K., Joshi, B.K.: Image denoising using wavelet transform and Wiener filter based on log energy distribution over Poisson-Gaussian noise model, pp. 1–6 (2014)

    Google Scholar 

  10. Buades, A., Coll, B., Morel, J.: A non-local algorithm for image denoising. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 60–65 (2005)

    Google Scholar 

  11. Chen, D.Q.: Regularized generalized inverse accelerating linearized alternating minimization algorithm for frame-based Poissonian image deblurring. SIAM J. Imaging Sci. 7, 716–739 (2014)

    Article  MathSciNet  Google Scholar 

  12. Chen, K.: Introduction to variational image-processing models and applications. Int. J. Comput. Math. 90, 1–8 (2013)

    Article  MathSciNet  Google Scholar 

  13. Chowdhury, A., et al.: Blood vessel characterization using virtual 3D models and convolutional neural networks in fluorescence microscopy. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI), pp. 629–632. IEEE (2017)

    Google Scholar 

  14. Conchello, J.A., Lichtman, J.W.: Fluorescence microscopy. Nat. Methods 2(12), 910–919 (2005)

    Article  Google Scholar 

  15. Dey, N., et al.: Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution. Microsc. Res. Tech. 69, 4 (2006)

    Article  Google Scholar 

  16. Eigen, D., Krishnan, D., Fergus, R.: Restoring an image taken through a window covered with dirt or rain. In: 2013 IEEE International Conference on Computer Vision, pp. 633–640 (2013)

    Google Scholar 

  17. Evangelista, V., Barsanti, L., Passarelli, V., Gualtieri, P.: From cells to proteins: imaging nature across dimensions. In: Proceedings of the NATO Advanced Study Institute, Pisa, Italy (2005)

    Google Scholar 

  18. Foi, A., Trimeche, M., Katkovnik, V., Egiazarian, K.: Practical Poissonian-Gaussian noise modeling and fitting for single-image raw-data. IEEE Trans. Image Process. 17(10), 1737–1754 (2008)

    Article  MathSciNet  Google Scholar 

  19. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)

    Google Scholar 

  20. Kokkinos, F., Lefkimmiatis, S.: Deep image demosaicking using a cascade of convolutional residual denoising networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 317–333. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_19

    Chapter  Google Scholar 

  21. Kokkinos, F., Lefkimmiatis, S.: Iterative joint image demosaicking and denoising using a residual denoising network. IEEE Trans. Image Process. PP, 1 (2019)

    Google Scholar 

  22. Krishnan, D., Fergus, R.: Fast image deconvolution using hyper-laplacian priors. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) Advances in Neural Information Processing Systems 22, pp. 1033–1041. Curran Associates, Inc. (2009)

    Google Scholar 

  23. Kruse, J., Rother, C., Schmidt, U.: Learning to push the limits of efficient FFT-based image deconvolution. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4596–4604 (2017)

    Google Scholar 

  24. Lefkimmiatis, S.: Universal denoising networks: a novel CNN architecture for image denoising. In: Proceedings of the CVPR, June 2018

    Google Scholar 

  25. Lefkimmiatis, S.: Non-local color image denoising with convolutional neural networks, pp. 5882–5891 (2017)

    Google Scholar 

  26. Lefkimmiatis, S., Unser, M.: Poisson image reconstruction with Hessian Schatten-norm regularization. IEEE Trans. Image Process. 22, 4314–4327 (2013)

    Article  MathSciNet  Google Scholar 

  27. Li, J., Luisier, F., Blu, T.: PURE-LET image deconvolution. IEEE Trans. Image Process. 27(1), 92–105 (2018)

    Article  MathSciNet  Google Scholar 

  28. Lu, H., Cheng, J.H., Han, G., Li, L., Liang, Z.: 3D distance-weighted Wiener filter for Poisson noise reduction in sinogram space for SPECT imaging. In: Antonuk, L.E., Yaffe, M.J. (eds.) Medical Imaging 2001: Physics of Medical Imaging, vol. 4320, pp. 905–913. International Society for Optics and Photonics, SPIE (2001)

    Google Scholar 

  29. Lucy, L.B.: An iterative technique for the rectification of observed distributions. Astron. J. 79, 745–754 (1974)

    Article  Google Scholar 

  30. Makitalo, M., Foi, A.: A closed-form approximation of the exact unbiased inverse of the Anscombe variance-stabilizing transformation. IEEE Trans. Image Process. 20(9), 2697–2698 (2011)

    Article  MathSciNet  Google Scholar 

  31. Makitalo, M., Foi, A.: Optimal inversion of the generalized Anscombe transformation for Poisson-Gaussian noise. IEEE Trans. Image Process. 22(1), 91–103 (2013)

    Article  MathSciNet  Google Scholar 

  32. Mildenhall, B., Barron, J.T., Chen, J., Sharlet, D., Ng, R., Carroll, R.: Burst denoising with kernel prediction networks, pp. 2502–2510 (2018)

    Google Scholar 

  33. de Monvel, J.B., Calvez, S.L., Ulfendahl, M.: Image restoration for confocal microscopy: improving the limits of deconvolution, with application to the visualization of the mammalian hearing organ. Biophys. J. 80(5), 2455–70 (2001)

    Article  Google Scholar 

  34. Reeves, S.J.: Fast image restoration without boundary artifacts. IEEE Trans. Image Process. 14(10), 1448–1453 (2005)

    Article  Google Scholar 

  35. Richardson, W.H.: Bayesian-based iterative method of image restoration\(\ast \). J. Opt. Soc. Am. 62(1), 55–59 (1972)

    Article  MathSciNet  Google Scholar 

  36. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear Total Variation Based Noise Removal Algorithms. Elsevier North-Holland Inc., USA (1992)

    Book  Google Scholar 

  37. Schmidt, U., Roth, S.: Shrinkage fields for effective image restoration. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2774–2781 (2014)

    Google Scholar 

  38. Shaw, P.J., Rawlins, D.J.: The point-spread function of a confocal microscope: its measurement and use in deconvolution of 3-D data. J. Microsc. 163(2), 151–165 (1991)

    Article  Google Scholar 

  39. Sheppard, C., Wilson, T.: Image formation in confocal scanning microscopes. Optik - Int. J. Light Electron Opt. 55, 331–342 (1980)

    Google Scholar 

  40. Shewchuk, J.R.: An introduction to the conjugate gradient method without the agonizing pain. Technical report, USA (1994)

    Google Scholar 

  41. Tao, X., Gao, H., Liao, R., Wang, J., Jia, J.: Detail-revealing deep video super-resolution. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4482–4490 (2017)

    Google Scholar 

  42. Tikhonov, A.N.: Solution of incorrectly formulated problems and the regularization method. Soviet Math. Dokl. 4, 1035–1038 (1963)

    MATH  Google Scholar 

  43. Tintner, G., Kailath, T.: Linear least-squares estimation (1980)

    Google Scholar 

  44. Ulyanov, D., Vedaldi, A., Lempitsky, S.V.: Instance normalization: the missing ingredient for fast stylization. arXiv: 1607.08022. Computer Vision and Pattern Recognition (2016)

  45. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)

    Article  Google Scholar 

  46. Wiener, N.: The Extrapolation, Interpolation and Smoothing of Stationary Time Series, with Engineering Applications. Wiley, New York (1949)

    Book  Google Scholar 

  47. Wu, Q., Merchant, F., Castleman, K.: Microscope Image Processing. Elsevier (2010)

    Google Scholar 

  48. Xu, L., Ren, J.S., Liu, C., Jia, J.: Deep convolutional neural network for image deconvolution. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 1790–1798. Curran Associates, Inc. (2014)

    Google Scholar 

  49. Zhang, J., Pan, J., Lai, W., Lau, R.W.H., Yang, M.: Learning fully convolutional networks for iterative non-blind deconvolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6969–6977 (2017)

    Google Scholar 

  50. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2808–2817 (2017)

    Google Scholar 

  51. Zhang, X.: An effective SURE-based Wiener filter for image denoising. In: Liang, Q., Mu, J., Wang, W., Zhang, B. (eds.) CSPS 2016. LNEE, vol. 423, pp. 889–895. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-3229-5_96

    Chapter  Google Scholar 

  52. Zhang, Y., et al.: A Poisson-Gaussian denoising dataset with real fluorescence microscopy images. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  53. Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for image restoration with neural networks. IEEE Trans. Comput. Imaging 3, 47–57 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Valeriya Pronina .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 27745 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pronina, V., Kokkinos, F., Dylov, D.V., Lefkimmiatis, S. (2020). Microscopy Image Restoration with Deep Wiener-Kolmogorov Filters. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12365. Springer, Cham. https://doi.org/10.1007/978-3-030-58565-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58565-5_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58564-8

  • Online ISBN: 978-3-030-58565-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics