Abstract
Microscopy is a powerful visualization tool in biology, enabling the study of cells, tissues, and the fundamental biological processes; yet, the observed images typically suffer from blur and background noise. In this work, we propose a unifying framework of algorithms for Gaussian image deblurring and denoising. These algorithms are based on deep learning techniques for the design of learnable regularizers integrated into the Wiener-Kolmogorov filter. Our extensive experimentation line showcases that the proposed approach achieves a superior quality of image reconstruction and surpasses the solutions that rely either on deep learning or on optimization schemes alone. Augmented with the variance stabilizing transformation, the proposed reconstruction pipeline can also be successfully applied to the problem of Poisson image deblurring, surpassing the state-of-the-art methods. Moreover, several variants of the proposed framework demonstrate competitive performance at low computational complexity, which is of high importance for real-time imaging applications .
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
PSF Generator. http://bigwww.epfl.ch/algorithms/psfgenerator/#ref. Accessed 25 Feb 2020
Diffraction PSF 3D. https://www.optinav.info/Diffraction-PSF-3D.htm. Accessed 30 May 2019
Al-Kofahi, Y., Zaltsman, A.B., Graves, R.M., Marshall, W., Rusu, M.: A deep learning-based algorithm for 2-D cell segmentation in microscopy images. BMC Bioinform. 19 (2018)
AL-Qinani, I.H.: Deblurring image and removing noise from medical images for cancerous diseases using a wiener filter. IRJET 8(4), 2354–2365 (2017)
Anscombe, F.J.: The transformation of Poisson, binomial and negative-binomial data. Biometrika 35(3–4), 246–254 (1948)
Arjomand Bigdeli, S., Zwicker, M., Favaro, P., Jin, M.: Deep mean-shift priors for image restoration. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 763–772. Curran Associates, Inc. (2017)
van Beek, P., Yang, J., Yamamoto, S., Ueda, Y.: Image deblurring and denoising with non-local regularization constraint. In: Information Processing and Communications, vol. 7543, January 2010
Bertsekas, D.P.: Nonlinear Programming, 2nd edn. (1999)
Boyat, A.K., Joshi, B.K.: Image denoising using wavelet transform and Wiener filter based on log energy distribution over Poisson-Gaussian noise model, pp. 1–6 (2014)
Buades, A., Coll, B., Morel, J.: A non-local algorithm for image denoising. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 60–65 (2005)
Chen, D.Q.: Regularized generalized inverse accelerating linearized alternating minimization algorithm for frame-based Poissonian image deblurring. SIAM J. Imaging Sci. 7, 716–739 (2014)
Chen, K.: Introduction to variational image-processing models and applications. Int. J. Comput. Math. 90, 1–8 (2013)
Chowdhury, A., et al.: Blood vessel characterization using virtual 3D models and convolutional neural networks in fluorescence microscopy. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI), pp. 629–632. IEEE (2017)
Conchello, J.A., Lichtman, J.W.: Fluorescence microscopy. Nat. Methods 2(12), 910–919 (2005)
Dey, N., et al.: Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution. Microsc. Res. Tech. 69, 4 (2006)
Eigen, D., Krishnan, D., Fergus, R.: Restoring an image taken through a window covered with dirt or rain. In: 2013 IEEE International Conference on Computer Vision, pp. 633–640 (2013)
Evangelista, V., Barsanti, L., Passarelli, V., Gualtieri, P.: From cells to proteins: imaging nature across dimensions. In: Proceedings of the NATO Advanced Study Institute, Pisa, Italy (2005)
Foi, A., Trimeche, M., Katkovnik, V., Egiazarian, K.: Practical Poissonian-Gaussian noise modeling and fitting for single-image raw-data. IEEE Trans. Image Process. 17(10), 1737–1754 (2008)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)
Kokkinos, F., Lefkimmiatis, S.: Deep image demosaicking using a cascade of convolutional residual denoising networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 317–333. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_19
Kokkinos, F., Lefkimmiatis, S.: Iterative joint image demosaicking and denoising using a residual denoising network. IEEE Trans. Image Process. PP, 1 (2019)
Krishnan, D., Fergus, R.: Fast image deconvolution using hyper-laplacian priors. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) Advances in Neural Information Processing Systems 22, pp. 1033–1041. Curran Associates, Inc. (2009)
Kruse, J., Rother, C., Schmidt, U.: Learning to push the limits of efficient FFT-based image deconvolution. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4596–4604 (2017)
Lefkimmiatis, S.: Universal denoising networks: a novel CNN architecture for image denoising. In: Proceedings of the CVPR, June 2018
Lefkimmiatis, S.: Non-local color image denoising with convolutional neural networks, pp. 5882–5891 (2017)
Lefkimmiatis, S., Unser, M.: Poisson image reconstruction with Hessian Schatten-norm regularization. IEEE Trans. Image Process. 22, 4314–4327 (2013)
Li, J., Luisier, F., Blu, T.: PURE-LET image deconvolution. IEEE Trans. Image Process. 27(1), 92–105 (2018)
Lu, H., Cheng, J.H., Han, G., Li, L., Liang, Z.: 3D distance-weighted Wiener filter for Poisson noise reduction in sinogram space for SPECT imaging. In: Antonuk, L.E., Yaffe, M.J. (eds.) Medical Imaging 2001: Physics of Medical Imaging, vol. 4320, pp. 905–913. International Society for Optics and Photonics, SPIE (2001)
Lucy, L.B.: An iterative technique for the rectification of observed distributions. Astron. J. 79, 745–754 (1974)
Makitalo, M., Foi, A.: A closed-form approximation of the exact unbiased inverse of the Anscombe variance-stabilizing transformation. IEEE Trans. Image Process. 20(9), 2697–2698 (2011)
Makitalo, M., Foi, A.: Optimal inversion of the generalized Anscombe transformation for Poisson-Gaussian noise. IEEE Trans. Image Process. 22(1), 91–103 (2013)
Mildenhall, B., Barron, J.T., Chen, J., Sharlet, D., Ng, R., Carroll, R.: Burst denoising with kernel prediction networks, pp. 2502–2510 (2018)
de Monvel, J.B., Calvez, S.L., Ulfendahl, M.: Image restoration for confocal microscopy: improving the limits of deconvolution, with application to the visualization of the mammalian hearing organ. Biophys. J. 80(5), 2455–70 (2001)
Reeves, S.J.: Fast image restoration without boundary artifacts. IEEE Trans. Image Process. 14(10), 1448–1453 (2005)
Richardson, W.H.: Bayesian-based iterative method of image restoration\(\ast \). J. Opt. Soc. Am. 62(1), 55–59 (1972)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear Total Variation Based Noise Removal Algorithms. Elsevier North-Holland Inc., USA (1992)
Schmidt, U., Roth, S.: Shrinkage fields for effective image restoration. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2774–2781 (2014)
Shaw, P.J., Rawlins, D.J.: The point-spread function of a confocal microscope: its measurement and use in deconvolution of 3-D data. J. Microsc. 163(2), 151–165 (1991)
Sheppard, C., Wilson, T.: Image formation in confocal scanning microscopes. Optik - Int. J. Light Electron Opt. 55, 331–342 (1980)
Shewchuk, J.R.: An introduction to the conjugate gradient method without the agonizing pain. Technical report, USA (1994)
Tao, X., Gao, H., Liao, R., Wang, J., Jia, J.: Detail-revealing deep video super-resolution. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4482–4490 (2017)
Tikhonov, A.N.: Solution of incorrectly formulated problems and the regularization method. Soviet Math. Dokl. 4, 1035–1038 (1963)
Tintner, G., Kailath, T.: Linear least-squares estimation (1980)
Ulyanov, D., Vedaldi, A., Lempitsky, S.V.: Instance normalization: the missing ingredient for fast stylization. arXiv: 1607.08022. Computer Vision and Pattern Recognition (2016)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)
Wiener, N.: The Extrapolation, Interpolation and Smoothing of Stationary Time Series, with Engineering Applications. Wiley, New York (1949)
Wu, Q., Merchant, F., Castleman, K.: Microscope Image Processing. Elsevier (2010)
Xu, L., Ren, J.S., Liu, C., Jia, J.: Deep convolutional neural network for image deconvolution. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 1790–1798. Curran Associates, Inc. (2014)
Zhang, J., Pan, J., Lai, W., Lau, R.W.H., Yang, M.: Learning fully convolutional networks for iterative non-blind deconvolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6969–6977 (2017)
Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2808–2817 (2017)
Zhang, X.: An effective SURE-based Wiener filter for image denoising. In: Liang, Q., Mu, J., Wang, W., Zhang, B. (eds.) CSPS 2016. LNEE, vol. 423, pp. 889–895. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-3229-5_96
Zhang, Y., et al.: A Poisson-Gaussian denoising dataset with real fluorescence microscopy images. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition (2019)
Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for image restoration with neural networks. IEEE Trans. Comput. Imaging 3, 47–57 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pronina, V., Kokkinos, F., Dylov, D.V., Lefkimmiatis, S. (2020). Microscopy Image Restoration with Deep Wiener-Kolmogorov Filters. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12365. Springer, Cham. https://doi.org/10.1007/978-3-030-58565-5_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-58565-5_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58564-8
Online ISBN: 978-3-030-58565-5
eBook Packages: Computer ScienceComputer Science (R0)