Skip to main content
Springer Nature Link
Log in

Whole Image Synthesis Using a Deep Encoder-Decoder Network

  • Conference paper
  • First Online:
Simulation and Synthesis in Medical Imaging (SASHIMI 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9968))

Included in the following conference series:

Abstract

The synthesis of medical images is an intensity transformation of a given modality in a way that represents an acquisition with a different modality (in the context of MRI this represents the synthesis of images originating from different MR sequences). Most methods follow a patch-based approach, which is computationally inefficient during synthesis and requires some sort of ‘fusion’ to synthesize a whole image from patch-level results. In this paper, we present a whole image synthesis approach that relies on deep neural networks. Our architecture resembles those of encoder-decoder networks, which aims to synthesize a source MRI modality to an other target MRI modality. The proposed method is computationally fast, it doesn’t require extensive amounts of memory, and produces comparable results to recent patch-based approaches.

V. Sevetlidis and M.V. Giuffrida—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    There is also the recent exciting unsupervised work by [24], however for ease of introducing the reader to the topic this is not discussed here.

  2. 2.

    Freely available at https://github.com/rasmusbergpalm/DeepLearnToolbox [18]. We modified the current implementation to enable also GPU (CUDA) processing.

  3. 3.

    We use only the CPU and not GPU to permit fair comparison with our MP implementation which does not use GPU.

References

  1. Alexander, D.C., Zikic, D., Zhang, J., Zhang, H., Criminisi, A.: Image quality transfer via random forest regression: applications in diffusion MRI. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part III. LNCS, vol. 8675, pp. 225–232. Springer, Heidelberg (2014)

    Google Scholar 

  2. Artaechevarria, X., Munoz-Barrutia, A., Ortiz-de-Solórzano, C.: Combination strategies in multi-atlas image segmentation: application to brain MR data. IEEE Trans. Med. Imaging 28(8), 1266–1277 (2009)

    Article  Google Scholar 

  3. Burgos, N., Cardoso, M.J., Thielemans, K., Modat, M., Pedemonte, S., Dickson, J., Barnes, A., Ahmed, R., Mahoney, C.J., Schott, J.M., et al.: Attenuation correction synthesis for hybrid PET-MR scanners: application to brain studies. IEEE Trans. Med. Imaging 33(12), 2332–2341 (2014)

    Article  Google Scholar 

  4. Cardoso, M.J., Sudre, C.H., Modat, M., Ourselin, S.: Template-based multimodal joint generative model of brain data. In: Ourselin, S., Alexander, D.C., Westin, C.-F., Cardoso, M.J. (eds.) IPMI 2015. LNCS, vol. 9123, pp. 17–29. Springer, Heidelberg (2015). doi:10.1007/978-3-319-19992-4_2

    Chapter  Google Scholar 

  5. Cho, K.H., Ilin, A., Raiko, T.: Improved learning of Gaussian-Bernoulli restricted Boltzmann machines. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds.) ICANN 2011. LNCS, vol. 6791, pp. 10–17. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21735-7_2

    Chapter  Google Scholar 

  6. Fischl, B., Salat, D.H., van der Kouwe, A.J., Makris, N., Ségonne, F., Quinn, B.T., Dale, A.M.: Sequence-independent segmentation of magnetic resonance images. Neuroimage 23, S69–S84 (2004)

    Article  Google Scholar 

  7. Guimond, A., Roche, A., Ayache, N., Meunier, J.: Three-dimensional multimodal brain warping using the demons algorithm and adaptive intensity corrections. IEEE Trans. Med. Imaging 20(1), 58–69 (2001)

    Article  Google Scholar 

  8. Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image analogies. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 327–340. ACM (2001)

    Google Scholar 

  9. Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  10. Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  11. Iglesias, J.E., Konukoglu, E., Zikic, D., Glocker, B., Van Leemput, K., Fischl, B.: Is synthesizing MRI contrast useful for inter-modality analysis? In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part I. LNCS, vol. 8149, pp. 631–638. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  12. Jog, A., Roy, S., Carass, A., Prince, J.L.: Magnetic resonance image synthesis through patch regression. In: IEEE 10th ISBI, pp. 350–353. IEEE (2013)

    Google Scholar 

  13. Kamyshanska, H., Memisevic, R.: The potential energy of an autoencoder. IEEE Trans. PAMI 37(6), 1261–1273 (2015)

    Article  Google Scholar 

  14. Konukoglu, E., van der Kouwe, A., Sabuncu, M.R., Fischl, B.: Example-based restoration of high-resolution magnetic resonance image acquisitions. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part I. LNCS, vol. 8149, pp. 131–138. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  15. Kroon, D.J., Slump, C.H.: MRI modalitiy transformation in demon registration. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro, ISBI 2009, pp. 963–966. IEEE (2009)

    Google Scholar 

  16. Larochelle, H., Erhan, D., Courville, A., Bergstra, J., Bengio, Y.: An empirical evaluation of deep architectures on problems with many factors of variation. In: Proceedings of the 24th ICML, pp. 473–480 (2007)

    Google Scholar 

  17. Maier, O., Wilms, M., von der Gablentz, J., Krämer, U.M., Münte, T.F., Handels, H.: Extra tree forests for sub-acute ischemic stroke lesion segmentation in MR sequences. J. Neurosci. Methods 240, 89–100 (2015)

    Article  Google Scholar 

  18. Palm, R.B.: Prediction as a candidate for learning deep hierarchical models of data. Master’s thesis (2012)

    Google Scholar 

  19. Rohlfing, T., Russakoff, D.B., Maurer, C.R.: Expectation maximization strategies for multi-atlas multi-label segmentation. In: Taylor, C.J., Noble, J.A. (eds.) IPMI 2003. LNCS, vol. 2732, pp. 210–221. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  20. Rousseau, F.: Brain hallucination. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 497–508. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  21. Roy, S., Carass, A., Prince, J.: A compressed sensing approach for MR tissue contrast synthesis. In: Székely, G., Hahn, H.K. (eds.) IPMI 2011. LNCS, vol. 6801, pp. 371–383. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  22. Tulder, G., Bruijne, M.: Why does synthesized data improve multi-sequence classification? In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 531–538. Springer, Heidelberg (2015). doi:10.1007/978-3-319-24553-9_65

    Chapter  Google Scholar 

  23. Nguyen, H., Zhou, K., Vemulapalli, R.: Cross-domain synthesis of medical images using efficient location-sensitive deep network. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 677–684. Springer, Heidelberg (2015). doi:10.1007/978-3-319-24553-9_83

    Chapter  Google Scholar 

  24. Vemulapalli, R., Van Nguyen, H., Zhou, S.K.: Unsupervised cross-modal synthesis of subject-specific scans. In: Proceedings of the IEEE ICCV, pp. 630–638 (2015)

    Google Scholar 

  25. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)

    MathSciNet  MATH  Google Scholar 

  26. Wein, W., Brunke, S., Khamene, A., Callstrom, M.R., Navab, N.: Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention. Med. Image Anal. 12(5), 577–585 (2008)

    Article  Google Scholar 

  27. Williams, D., Hinton, G.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)

    Article  Google Scholar 

  28. Wolz, R., Chu, C., Misawa, K., Mori, K., Rueckert, D.: Multi-organ abdominal CT segmentation using hierarchically weighted subject-specific atlases. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part I. LNCS, vol. 7510, pp. 10–17. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  29. Ye, D.H., Zikic, D., Glocker, B., Criminisi, A., Konukoglu, E.: Modality propagation: coherent synthesis of subject-specific scans with data-driven regularization. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part I. LNCS, vol. 8149, pp. 606–613. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

Download references

Acknowledgement

We thank NVIDIA Inc. for providing us with a Titan X GPU used for our experiments.

Author information

Authors and Affiliations

  1. PRIAn, IMT School Advanced Studies Lucca, Lucca, Italy

    Vasileios Sevetlidis, Mario Valerio Giuffrida & Sotirios A. Tsaftaris

  2. School of Engineering, University of Edinburgh, Edinburgh, UK

    Vasileios Sevetlidis, Mario Valerio Giuffrida & Sotirios A. Tsaftaris

Authors
  1. Vasileios Sevetlidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mario Valerio Giuffrida
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sotirios A. Tsaftaris
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vasileios Sevetlidis .

Editor information

Editors and Affiliations

  1. University of Edinburgh , Edinburgh, United Kingdom

    Sotirios A. Tsaftaris

  2. University of Sheffield , Sheffield, United Kingdom

    Ali Gooya

  3. University of Sheffield , Sheffield, United Kingdom

    Alejandro F. Frangi

  4. The Johns Hopkins University , Baltimore, Maryland, USA

    Jerry L. Prince

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Sevetlidis, V., Giuffrida, M.V., Tsaftaris, S.A. (2016). Whole Image Synthesis Using a Deep Encoder-Decoder Network. In: Tsaftaris, S., Gooya, A., Frangi, A., Prince, J. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2016. Lecture Notes in Computer Science(), vol 9968. Springer, Cham. https://doi.org/10.1007/978-3-319-46630-9_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46630-9_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46629-3

  • Online ISBN: 978-3-319-46630-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation