Abstract
Super-Resolution (SR) techniques can compensate for the missing information of low-resolution images and further promote experts and algorithms to make accurate diagnosis decisions. Although the existing pixel-loss based SR works produce high-resolution images with impressive objective metrics, the over-smoothed contents that lose high-frequency information would disturb the visual experience and the subsequent diagnosis. To address this issue, we propose a joint Spatial-Wavelet super-resolution Network (SWD-Net) with collaborative Dual-stream. In the spatial stage, a Refined Context Fusion (RCF) is proposed to iteratively rectify the features by a counterpart stream with compensative receptive fields. After that, the wavelet stage enhances the reconstructed images, especially the structural boundaries. Specifically, we design the tailor-made Wavelet Features Adaptation (WFA) to adjust the wavelet coefficients for better compatibility with networks and Wavelet-Aware Convolutional blocks (WAC) to exploit features in the wavelet domain efficiently. We further introduce the wavelet coefficients supervision together with the traditional spatial loss to jointly optimize the network and obtain the high-frequency enhanced SR images. To evaluate the SR for medical images, we build a benchmark dataset with histopathology images and evaluate the proposed SWD-Net under different settings. The comprehensive experiments demonstrate our SWD-Net outperforms state-of-the-art methods. Furthermore, SWD-Net is proven to promote medical image diagnosis with a large margin. The source code and dataset are available at https://github.com/franciszchen/SWD-Net.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Aeffner, F., et al.: Introduction to digital image analysis in whole-slide imaging: a white paper from the digital pathology association. J. Pathol. Inform. 10, 9 (2019)
Ahn, N., Kang, B., Sohn, K.A.: Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 252–268 (2018)
Bejnordi, B.E., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017)
Cao, Y., Xu, J., Lin, S., Wei, F., Hu, H.: Gcnet: Non-local networks meet squeeze-excitation networks and beyond. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)
Du, C., et al.: Orientation-aware deep neural network for real image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Guo, T., Seyed Mousavi, H., Huu Vu, T., Monga, V.: Deep wavelet prediction for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 104–113 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, H., He, R., Sun, Z., Tan, T.: Wavelet-srnet: A wavelet-based CNN for multi-scale face super resolution. In: ICCV, pp. 1689–1697 (2017)
Khan, S., Huh, J., Ye, J.C.: Deep learning-based universal beamformer for ultrasound imaging. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 619–627. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_69
Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR) (2015)
Li, Z., et al.: A two-stage multi-loss super-resolution network for arterial spin labeling magnetic resonance imaging. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 12–20. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_2
Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and pattern recognition workshops, pp. 136–144 (2017)
Liu, P., Zhang, H., Zhang, K., Lin, L., Zuo, W.: Multi-level wavelet-cnn for image restoration. In: CVPR Workshops. pp. 773–782 (2018)
Mukherjee, L., Bui, H.D., Keikhosravi, A., Loeffler, A., Eliceiri, K.W.: Super-resolution recurrent convolutional neural networks for learning with multi-resolution whole slide images. J. Biomed. Optics 24(12), 126003 (2019)
Peyré, G.: A Wavelet Tour of Signal Processing: The Sparse Way. Academic Press, USA (2009)
Sajjadi, M.S., Scholkopf, B., Hirsch, M.: Enhancenet: Single image super-resolution through automated texture synthesis. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4491–4500 (2017)
Shermeyer, J., Van Etten, A.: The effects of super-resolution on object detection performance in satellite imagery. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883 (2016)
Srivastav, V., Gangi, A., Padoy, N.: Human pose estimation on privacy-preserving low-resolution depth images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 583–591. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_65
Sun, K., Xiao, B., Liu, D., Wang, J.: Deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5693–5703 (2019)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
Upadhyay, U., Awate, S.P.: A mixed-supervision multilevel GAN framework for image quality enhancement. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 556–564. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_62
Veeling, B.S., Linmans, J., Winkens, J., Cohen, T., Welling, M.: Rotation equivariant CNNs for digital pathology. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 210–218. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_24
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 286–301 (2018)
Zhao, X., Zhang, Y., Zhang, T., Zou, X.: Channel splitting network for single MR image super-resolution. IEEE Trans. Image Process. 28(11), 5649–5662 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, Z., Guo, X., Yang, C., Ibragimov, B., Yuan, Y. (2020). Joint Spatial-Wavelet Dual-Stream Network for Super-Resolution. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12265. Springer, Cham. https://doi.org/10.1007/978-3-030-59722-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-59722-1_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59721-4
Online ISBN: 978-3-030-59722-1
eBook Packages: Computer ScienceComputer Science (R0)
