Abstract
Ultrasound visual imaging is currently one of three mainstream image diagnosis technologies in the medical industry, but due to the limitations of sensors, transmission media and ultrasound characteristics, the quality of ultrasound imaging may be poor, especially its low spatial resolution. We incorporate, in this paper, a new multi-scale deep encoder-decoder structure into a PatchGAN (patch generative adversarial network) based framework for fast perceptual ultrasound image super-resolution (SR). Specifically, the entire algorithm is carried out in two stages: ultrasound SR image generation and image refinement. In the first stage, a multi-scale deep encoder-decoder generator is employed to accurately super-resolve the LR ultrasound images. In the second stage, we advocate the confrontational characteristics of the discriminator to impel the generator such that more realistic high-resolution (HR) ultrasound images can be produced. The assessments in terms of PSNR/IFC/SSIM, inference efficiency and visual effects demonstrate its effectiveness and superiority, when compared to the most state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Blau, Y., Michaeli, T.: The perception-distortion tradeoff. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6228–6237 (2018)
Chen, J., Chen, J., Chao, H., Yang, M.: Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3155–3164 (2018)
Choi, W., Kim, M., HakLee, J., Kim, J., BeomRa, J.: Deep CNN-based ultrasound super-resolution for high-speed high-resolution B-mode imaging. In: Proceedings of the IEEE International Ultrasonics Symposium, pp. 1–4, October 2018. https://doi.org/10.1109/ULTSYM.2018.8580032
Diamantis, K., Greenaway, A.H., Anderson, T., Jensen, J.A., Dalgarno, P.A., Sboros, V.: Super-resolution axial localization of ultrasound scatter using multi-focal imaging. IEEE Trans. Biomed. Eng. 65(8), 1840–1851 (2018)
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)
Hoskins, P.R., Martin, K., Thrush, A.: Diagnostic Ultrasound: Physics and Equipment. Cambridge University Press, Cambridge (2010)
Hudson, J.M.: Dynamic contrast enhanced ultrasound for therapy monitoring. Eur. J. Radiol. 84(9), 1650–1657 (2015)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)
Liang, Y., Wang, J., Zhou, S., Gong, Y., Zheng, N.: Incorporating image priors with deep convolutional neural networks for image super-resolution. Neurocomputing 194, 340–347 (2016)
Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)
Liu, H., Fu, Z., Han, J., Shao, L., Hou, S., Chu, Y.: Single image super-resolution using multi-scale deep encoder-decoder with phase congruency edge map guidance. Inf. Sci. 473, 44–58 (2019)
Lu, J., Liu, W.: Unsupervised super-resolution framework for medical ultrasound images using dilated convolutional neural networks. In: Proceedings of the IEEE 3rd International Conference on Image, Vision and Computing, pp. 739–744. IEEE (2018)
Mallat, S.: A Wavelet Tour of Signal Processing. Academic Press, San Diego (1999)
Morin, R., Bidon, S., Basarab, A., Kouamé, D.: Semi-blind deconvolution for resolution enhancement in ultrasound imaging. In: Proceedings of the IEEE International Conference on Image Processing, pp. 1413–1417. IEEE (2013)
Odena, A., Dumoulin, V., Olah, C.: Deconvolution and checkerboard artifacts. Distill 1(10), e3 (2016)
Park, S.J., Son, H., Cho, S., Hong, K.S., Lee, S.: SRFeat: single image super-resolution with feature discrimination. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 439–455 (2018)
Rahaman, N., et al.: On the spectral bias of neural networks. In: Proceedings of the 36th International Conference on Machine Learning, pp. 5301–5310. PMLR (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sakkos, D., Liu, H., Han, J., Shao, L.: End-to-end video background subtraction with 3D convolutional neural networks. Multimedia Tools Appl. 77(17), 23023–23041 (2017). https://doi.org/10.1007/s11042-017-5460-9
Sheikh, H.R., Bovik, A.C., De Veciana, G.: An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans. Image Process. 14(12), 2117–2128 (2005)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Skourt, B.A., El Hassani, A., Majda, A.: Lung CT image segmentation using deep neural networks. Procedia Comput. Sci. 127, 109–113 (2018)
van Sloun, R.J., Solomon, O., Bruce, M., Khaing, Z.Z., Eldar, Y.C., Mischi, M.: Deep learning for super-resolution vascular ultrasound imaging. In: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1055–1059. IEEE (2019)
Umehara, K., Ota, J., Ishida, T.: Application of super-resolution convolutional neural network for enhancing image resolution in chest CT. J. Digit. Imaging 31(4), 441–450 (2018)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Xu, Z.-Q.J., Zhang, Y., Xiao, Y.: Training behavior of deep neural network in frequency domain. In: Gedeon, T., Wong, K.W., Lee, M. (eds.) ICONIP 2019. LNCS, vol. 11953, pp. 264–274. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-36708-4_22
Zhao, N., Wei, Q., Basarab, A., Kouamé, D., Tourneret, J.Y.: Single image super-resolution of medical ultrasound images using a fast algorithm. In: Proceedings of the IEEE 13th International Symposium on Biomedical Imaging, pp. 473–476. IEEE (2016)
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant No. 61971004 and by the Key Project of Natural Science of Anhui Provincial Department of Education under Grant No. KJ2019A0083.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, J., Liu, H., Zheng, X., Han, J. (2020). Exploring Multi-scale Deep Encoder-Decoder and PatchGAN for Perceptual Ultrasound Image Super-Resolution. In: Zhang, H., Zhang, Z., Wu, Z., Hao, T. (eds) Neural Computing for Advanced Applications. NCAA 2020. Communications in Computer and Information Science, vol 1265. Springer, Singapore. https://doi.org/10.1007/978-981-15-7670-6_5
Download citation
DOI: https://doi.org/10.1007/978-981-15-7670-6_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-7669-0
Online ISBN: 978-981-15-7670-6
eBook Packages: Computer ScienceComputer Science (R0)