Abstract
Uncertainty quantification in inverse medical imaging tasks with deep learning has received little attention. However, deep models trained on large data sets tend to hallucinate and create artifacts in the reconstructed output that are not anatomically present. We use a randomly initialized convolutional network as parameterization of the reconstructed image and perform gradient descent to match the observation, which is known as deep image prior. In this case, the reconstruction does not suffer from hallucinations as no prior training is performed. We extend this to a Bayesian approach with Monte Carlo dropout to quantify both aleatoric and epistemic uncertainty. The presented method is evaluated on the task of denoising different medical imaging modalities. The experimental results show that our approach yields well-calibrated uncertainty. That is, the predictive uncertainty correlates with the predictive error. This allows for reliable uncertainty estimates and can tackle the problem of hallucinations and artifacts in inverse medical imaging tasks.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Agostinelli, F., Anderson, M.R., Lee, H.: Adaptive multi-column deep neural networks with application to robust image denoising. In: Advances in Neural Information Processing Systems, pp. 1493–1501 (2013)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Boston (2006). https://doi.org/10.1007/978-1-4615-7566-5
Chang, S.G., Yu, B., Vetterli, M.: Adaptive wavelet thresholding for image denoising and compression. IEEE Trans. Image Process. 9(9), 1532–1546 (2000). https://doi.org/10.1109/83.862633
Cheng, Z., Gadelha, M., Maji, S., Sheldon, D.: A Bayesian perspective on the deep image prior. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5443–5451 (2019)
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. Trans. Image Process. 16(8), 2080–2095 (2007). https://doi.org/10.1109/TIP.2007.901238
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In: ICML, pp. 1050–1059 (2016)
Gondara, L.: Medical image denoising using convolutional denoising autoencoders. In: International Conference on Data Mining Workshops, pp. 241–246 (2016). https://doi.org/10.1109/ICDMW.2016.0041
Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: ICML, pp. 1321–1330 (2017)
van den Heuvel, T.L., de Bruijn, D., de Korte, C.L., Ginneken, B.v.: Automated measurement of fetal head circumference using 2D ultrasound images. PloS One 13(8), e0200412 (2018). https://doi.org/10.1371/journal.pone.0200412. US dataset source
Hogg, R.V., McKean, J., Craig, A.T.: Introduction to Mathematical Statistics, 8th edn. Pearson, New York (2018)
Jain, V., Seung, S.: Natural image denoising with convolutional networks. In: Advances in Neural Information Processing Systems, pp. 769–776 (2009)
Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: NeurIPS, pp. 5574–5584 (2017)
Kermany, D.S., et al.: Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5), 1122–1131 (2018). https://doi.org/10.1016/j.cell.2018.02.010
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: ICLR (2014)
Laves, M.H., Ihler, S., Fast, J.F., Kahrs, L.A., Ortmaier, T.: Well-calibrated regression uncertainty in medical imaging with deep learning. In: Medical Imaging with Deep Learning (2020)
Laves, M.H., Ihler, S., Kahrs, L.A., Ortmaier, T.: Semantic denoising autoencoders for retinal optical coherence tomography. In: SPIE/OSA European Conference on Biomedical Optics, vol. 11078, pp. 86–89 (2019). https://doi.org/10.1117/12.2526936
Lee, S., Lee, M.S., Kang, M.G.: Poisson-gaussian noise analysis and estimation for low-dose x-ray images in the NSCT domain. Sensors 18(4), 1019 (2018)
Lempitsky, V., Vedaldi, A., Ulyanov, D.: Deep Image Prior. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9446–9454 (2018). https://doi.org/10.1109/CVPR.2018.00984
Levi, D., Gispan, L., Giladi, N., Fetaya, E.: Evaluating and calibrating uncertainty prediction in regression tasks. arXiv arXiv:1905.11659 (2019)
Li, C., Chen, C., Carlson, D., Carin, L.: Preconditioned stochastic gradient Langevin dynamics for deep neural networks. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 1788–1794 (2016)
Michailovich, O.V., Tannenbaum, A.: Despeckling of medical ultrasound images. Trans. Ultrason. Ferroelectr. Freq. Control 53(1), 64–78 (2006). https://doi.org/10.1109/TUFFC.2006.1588392
Rabbani, H., Nezafat, R., Gazor, S.: Wavelet-domain medical image denoising using bivariate Laplacian mixture model. Trans. Biomed. Eng. 56(12), 2826–2837 (2009). https://doi.org/10.1109/TBME.2009.2028876
Salinas, H.M., Fernandez, D.C.: Comparison of PDE-based nonlinear diffusion approaches for image enhancement and denoising in optical coherence tomography. IEEE Trans. Med. Imaging 26(6), 761–771 (2007). https://doi.org/10.1109/TMI.2006.887375
Sotiras, A., Davatzikos, C., Paragios, N.: Deformable medical image registration: a survey. IEEE Trans. Med. Imaging 32(7), 1153–1190 (2013). https://doi.org/10.1109/TMI.2013.2265603
Wang, N., Tao, D., Gao, X., Li, X., Li, J.: A comprehensive survey to face hallucination. Int. J. Comput. Vis. 106(1), 9–30 (2014). https://doi.org/10.1007/s11263-013-0645-9
Welling, M., Teh, Y.W.: Bayesian learning via stochastic gradient Langevin dynamics. In: ICML, pp. 681–688 (2011)
Žabić, S., Wang, Q., Morton, T., Brown, K.M.: A low dose simulation tool for CT systems with energy integrating detectors. Med. Phys. 40(3), 031102 (2013). https://doi.org/10.1118/1.4789628
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017). https://doi.org/10.1109/TIP.2017.2662206
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
A Appendix
A Appendix
1.1 A.1 Additional Figures
MSE (top row) between denoised \( \hat{\textit{\textbf{x}}} \) image and noisy image \( \tilde{\textit{\textbf{x}}} \) and SSIM (bottom row) between denoised \( \hat{\textit{\textbf{x}}} \) image and ground truth \( \textit{\textbf{x}} \) vs. iteration. Only MCDIP does not overfit the noisy image and converges with highest similarity to the ground truth. Despite the claim of the authors, SGLD suffers from overfitting and creates the need for carefully applied early stopping [4]. Note: We compared both our own implementation of SGLD and the original code provided by the authors. The plots show means from 3 runs with different random initialization.
1.2 A.2 Additional Tables
Calibration diagrams and uncertainty maps for SGLD+NLL and MCDIP after convergence (best viewed with digital zoom). (Left) The calibration diagrams show MSE vs. uncertainty and provide mean uncertainty (U) and UCE values. (Right) Uncertainty maps show per-pixel uncertainty. Due to overfitting, the MSE and uncertainty from SGLD+NLL concentrates around 0.0.
1.3 A.3 SGLD with Step Size Decay
Additionall, we implement SGLD with step size decay as described by Welling et al. [26]. The step size \( \epsilon \) is used to scale the parameter update in the SGD step (i.e. the learning rate) and defines the variance of the noise that is injected into the gradients. Here, we reduce the step size at each step t exponentially with \( \epsilon _{t} = 0.999^{t} \epsilon _{0} \). To satisfy the step size property (Eq. (2) in [26]), we fix the step size once it decreases below 1e-8. We observe no overfitting of the noisy image with step size decay (see Fig. 11). However, the quality of the resulting denoised image is very sensitive to the decay scheme. Choosing a decrease that is too low (i.e. \( \epsilon _{t} = 0.9999^{t} \epsilon _{0} \)) results in overfitting; a decrease that is too high (i.e. \( \epsilon _{t} = 0.99^{t} \epsilon _{0} \)) results in convergence to a subpar reconstruction. This is equivalent to carefully applied early stopping and therefore nullifies the advantage of SGLD for denoising of medical images.
1.4 A.4 Downsampling
Here, we provide justification why downsampling of an image by averaging neighboring pixels reduces the noise level and can be used as an approximation to a ground truth noise-free image (by sacrificing image resolution).
Proposition 1
Downsampling of an image reduces the observation noise.
Proof
Let \( X = \mu _{x} + \varepsilon _{x} \) and \( Y = \mu _{y} + \varepsilon _{y} \) be two neighboring pixels affected by additive i.i.d. noise \( \varepsilon _{x} , \varepsilon _{y} \sim \mathcal {N}(0, \sigma ^{2}) \). The pixels are assumed to be uncorrelated to noise. Pixels in a local neighborhood are highly correlated and assumed to be of high similarity \( \mu _{x} \approx \mu _{y} = \mu \). Let \( Z = \tfrac{1}{2} \left( X + Y \right) \) be the average of two neighboring pixels (i.e. the result of downsampling). The expectation is given by
and the variance is given by
Thus, if the similarity of neighboring pixels is sufficiently high, downsampling reduces the variance of average pixel Z by a factor of 2. \(\square \)
Naturally, two neighboring pixels are not exactly equal. However, downsampling can also be viewed as superposing two signals, each with a highly correlated and an uncorrelated part. Without providing proof, the amplitude of the addition of two signals can be viewed as vector addition. In the uncorrelated case, the two signals are perpendicular to each other and in the correlated case, the angle between the two signals is acute. Thus, the correlated parts of the two signals have a higher impact on the resulting addition than the uncorrelated (noise) parts. In the ideal case, where the noise is uncorrelated and the signals are in parallel, the same noise reduction as above follows.
1.5 A.5 Link Between Poisson Distribution and Normal Distribution
We approximate the Poisson noise to simulate a low-dose X-ray image with a Normal distribution. It is well-known that the limiting distribution of \( \mathsf {Poisson}(\lambda ) \) is Normal as \( \lambda \rightarrow \infty \) [10]. For completeness, we list a common proof using the moment generating function of a standardized Poisson random variable:
Theorem 1
The Poisson(\(\lambda \)) distribution can be approximated with a Normal distribution as \( \lambda \rightarrow \infty \).
Proof
Let \( X_{\lambda } \sim \mathsf {Poisson}(\lambda ), ~ \lambda \in \{ 1, 2, \ldots \} \). The probability mass function of \( X_{\lambda } \) is given by
The moment generating function is given by [10]
The standardized Poisson random variable
has the limiting moment generating function
which is the moment generating function of a standard normal random variable. \(\square \)
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Laves, MH., Tölle, M., Ortmaier, T. (2020). Uncertainty Estimation in Medical Image Denoising with Bayesian Deep Image Prior. In: Sudre, C.H., et al. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis. UNSURE GRAIL 2020 2020. Lecture Notes in Computer Science(), vol 12443. Springer, Cham. https://doi.org/10.1007/978-3-030-60365-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-60365-6_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60364-9
Online ISBN: 978-3-030-60365-6
eBook Packages: Computer ScienceComputer Science (R0)