Skip to main content
SpringerLink
Log in

Registration of Histopathology Images Using Self Supervised Fine Grained Feature Maps

  • Conference paper
  • First Online:
Book cover Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning (DART 2020, DCL 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12444))

Abstract

Image registration is an important part of many clinical workflows and inclusion of segmentation information of structures of interest improves registration performance. We propose to integrate segmentation information in a registration framework using fine grained feature maps obtained in a self supervised manner. Self supervised feature maps enables use of segmentation information despite the unavailability of manual segmentations. Experimental results show our approach effectively replaces manual segmentation maps and demonstrate the possibility of obtaining state of the art registration performance in real world cases where manual segmentation maps are unavailable.

J. Tong and D. Mahapatra—Equal Contributions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. ANHIR: Automatic non-rigid histological image registration challenge. https://anhir.grand-challenge.org/. Accessed 30 Jan 2020

  2. Bai, W., et al.: Self-supervised learning for cardiac MR image segmentation by anatomical position prediction. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 541–549. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_60

    Chapter  Google Scholar 

  3. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imag. 38(8), 1788–1800 (2019)

    Article  Google Scholar 

  4. Balakrishnan, G., Zhao, A., Sabuncu, M., Guttag, J.: An supervised learning model for deformable medical image registration. In: Proceedings of CVPR, pp. 9252–9260 (2018)

    Google Scholar 

  5. Borovec, J., Munoz-Barrutia, A., Kybic, J.: Benchmarking of image registration methods for differently stained histological slides. In: Proceedings of IEEE ICIP, pp. 3368–3372 (2018)

    Google Scholar 

  6. Fischl, B.: FreeSurfer. NeuroImage 62(2), 774–781 (2015)

    Article  Google Scholar 

  7. Hu, X., Kang, M., Huang, W., Scott, M.R., Wiest, R., Reyes, M.: Dual-stream pyramid registration network. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 382–390. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_43

    Chapter  Google Scholar 

  8. Hu, Y., Gibson, E., Barratt, D.C., Emberton, M., Noble, J.A., Vercauteren, T.: Conditional segmentation in lieu of image registration. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 401–409. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_45

    Chapter  Google Scholar 

  9. Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: NIPS, pp. 2017–2025 (2015)

    Google Scholar 

  10. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014)

    Google Scholar 

  11. Kugler, M., et al.: Robust 3D image reconstruction of pancreatic cancer tumors from histopathological images with different stains and its quantitative performance evaluation. Int. J. Comput. Assist. Radiol. Surg. 14, 2047–2055 (2019). https://doi.org/10.1007/s11548-019-02019-8

    Article  Google Scholar 

  12. Larsson, M., Stenborg, E., Toft, C., Hammarstrand, L., Sattler, T., Kahl, F.: Fine-grained segmentation networks: self-supervised segmentation for improved long-term visual localization. In: Proceedings of ICCV, pp. 31–41 (2019)

    Google Scholar 

  13. Lee, M.C.H., Oktay, O., Schuh, A., Schaap, M., Glocker, B.: Image-and-spatial transformer networks for structure-guided image registration. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 337–345. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_38

    Chapter  Google Scholar 

  14. Liu, L., Hu, X., Zhu, L., Heng, P.-A.: Probabilistic multilayer regularization network for unsupervised 3D brain image registration. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 346–354. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_39

    Chapter  Google Scholar 

  15. Mahapatra, D., Antony, B., Sedai, S., Garnavi, R.: Deformable medical image registration using generative adversarial networks. In: Proceedings of IEEE ISBI, pp. 1449–1453 (2018)

    Google Scholar 

  16. Mahapatra, D., Ge, Z.: Training data independent image registration with gans using transfer learning and segmentation information. In: Proceedings of IEEE ISBI, pp. 709–713 (2019)

    Google Scholar 

  17. Mahapatra, D., Ge, Z.: Training data independent image registration using generative adversarial networks and domain adaptation. Pattern Recogn. 100, 1–14 (2020)

    Article  Google Scholar 

  18. Mahapatra, D., Ge, Z., Sedai, S., Chakravorty, R.: Joint registration and segmentation of xray images using generative adversarial networks. In: Shi, Y., Suk, H.-I., Liu, M. (eds.) MLMI 2018. LNCS, vol. 11046, pp. 73–80. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00919-9_9

    Chapter  Google Scholar 

  19. Mahapatra, D., Sun, Y.: Joint Registration and segmentation of dynamic cardiac perfusion images using MRFs. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6361, pp. 493–501. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15705-9_60

    Chapter  Google Scholar 

  20. Mahapatra, D., Sun, Y.: Integrating segmentation information for improved MRF-based elastic image registration. IEEE Trans. Imag. Proc. 21(1), 170–183 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  21. Maintz, J., Viergever, M.: A survey of medical image registration. Med. Image Anal. 2(1), 1–36 (1998)

    Article  Google Scholar 

  22. Mueller, S.G.: Ways toward an early diagnosis in Alzheimer’s disease: the Alzheimer’s disease neuroimaging initiative (ADNI). Alzheimer’s Dement. 1(1), 55–66 (2005)

    Article  Google Scholar 

  23. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: Proceedings of CVPR, pp. 31–41 (2016)

    Google Scholar 

  24. Pohl, K.M., Fisher, J., Grimson, W.E.L., Kikinis, R., Wells, W.M.: A Bayesian model for joint segmentation and registration. NeuroImage 31(1), 228–239 (2006)

    Article  Google Scholar 

  25. Rohé, M.-M., Datar, M., Heimann, T., Sermesant, M., Pennec, X.: SVF-Net: learning deformable image registration using shape matching. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 266–274. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_31

    Chapter  Google Scholar 

  26. Rueckert, D., Sonoda, L., Hayes, C., Hill, D., Leach, M., Hawkes, D.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 18(8), 712–721 (1999)

    Article  Google Scholar 

  27. Sokooti, H., de Vos, B., Berendsen, F., Lelieveldt, B.P.F., Išgum, I., Staring, M.: Nonrigid image registration using multi-scale 3D convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 232–239. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_27

    Chapter  Google Scholar 

  28. Tibshirani, R., Walther, G., Hastie, T.: Estimating the number of clusters in a data set via the gap statistic. J. Roy. Stat. Soc. B 63(2), 411–423 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  29. de Vos, B.D., Berendsen, F.F., Viergever, M.A., Staring, M., Išgum, I.: End-to-end unsupervised deformable image registration with a convolutional neural network. In: Cardoso, M., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 204–212. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_24

    Chapter  Google Scholar 

  30. Xu, Z., Niethammer, M.: DeepAtlas: joint semi-supervised learning of image registration and segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 420–429. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_47

    Chapter  Google Scholar 

  31. Yezzi, A., Zollei, L., Kapur, T.: A variational framework for joint segmentation and registration. In: Proceedings of MMBIA, pp. 44–51 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Monash University, Melbourne, Australia

    James Tong & Zongyuan Ge

  2. Airdoc Research, Melbourne, Australia

    James Tong, Paul Bonnington, Tom Drummond & Zongyuan Ge

  3. Inception Institute of Artificial Intelligence, Abu Dhabi, UAE

    Dwarikanath Mahapatra

Authors
  1. James Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dwarikanath Mahapatra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul Bonnington
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tom Drummond
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zongyuan Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to James Tong , Dwarikanath Mahapatra or Zongyuan Ge .

Editor information

Editors and Affiliations

  1. Technical University Munich, Munich, Germany

    Shadi Albarqouni

  2. University of Pennsylvania, Philadelphia, PA, USA

    Spyridon Bakas

  3. Imperial College London, London, UK

    Konstantinos Kamnitsas

  4. King's College London, London, UK

    M. Jorge Cardoso

  5. Vanderbilt University, Nashville, TN, USA

    Bennett Landman

  6. NVIDIA Ltd., Cambridge, UK

    Wenqi Li

  7. NVIDIA GmbH and Johnson & Johnson, Munich, Germany

    Fausto Milletari

  8. NVIDIA GmbH, Munich, Germany

    Nicola Rieke

  9. NVIDIA Corporation, Bethesda, MD, USA

    Holger Roth

  10. NVIDIA Corporation, Bethesda, MD, USA

    Daguang Xu

  11. NVIDIA Corporation, Santa Clara, CA, USA

    Ziyue Xu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tong, J., Mahapatra, D., Bonnington, P., Drummond, T., Ge, Z. (2020). Registration of Histopathology Images Using Self Supervised Fine Grained Feature Maps. In: Albarqouni, S., et al. Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning. DART DCL 2020 2020. Lecture Notes in Computer Science(), vol 12444. Springer, Cham. https://doi.org/10.1007/978-3-030-60548-3_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60548-3_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60547-6

  • Online ISBN: 978-3-030-60548-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation