Abstract
We present an end-to-end unsupervised deformable registration approach for high-resolution histopathology images with different stains. Our method comprises two sequential registration networks, where the local affine network can handle small deformations, and the non-rigid network is able to align texture details further. Both networks adopt the multi-magnification structure to improve registration accuracy. We train the proposed networks separately and evaluate them on the dataset provided by the University Hospital Frankfurt, which contains 41 multi-stained histopathology whole-slide images. By comparing with methods using the single-magnification structure, we confirm that the proposed multi-view architecture can significantly improve the performance of the local affine registration algorithm. Moreover, the proposed method achieves high registration accuracy of contents at the cell level and is potentially applicable to other medical image alignment tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Costin, H.N., Rotariu, C.: Registration of multimodal medical images. Comput. Sci. J. Moldova 51(3), 231–254 (2009)
Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning for fast probabilistic diffeomorphic registration. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 729–738. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_82
Feuerstein, M., Heibel, H., Gardiazabal, J., Navab, N., Groher, M.: Reconstruction of 3-D histology images by simultaneous deformable registration. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6892, pp. 582–589. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23629-7_71
Fischer, B., Modersitzki, J.: A unified approach to fast image registration and a new curvature based registration technique. Linear Algebra Appl. 380, 107–124 (2004)
Fu, Y., Lei, Y., Wang, T., Curran, W.J., Liu, T., Yang, X.: Deep learning in medical image registration: a review. Phys. Med. Biol. 65(20), 20TR01 (2020)
Haskins, G., Kruger, U., Yan, P.: Deep learning in medical image registration: a survey. Mach. Vis. Appl., 1–18 (2020). https://doi.org/10.1007/s00138-020-01060-x
Ho, D.J., et al.: Deep multi-magnification networks for multi-class breast cancer image segmentation. Comput. Med. Imaging Graph. 88, 101866 (2021)
Lotz, J., et al.: Patch-based nonlinear image registration for gigapixel whole slide images. IEEE Trans. Biomed. Eng. 63(9), 1812–1819 (2015)
Lv, J., Yang, M., Zhang, J., Wang, X.: Respiratory motion correction for free-breathing 3d abdominal MRI using CNN-based image registration: a feasibility study. Br. J. Radiol. 91(xxxx), 20170788 (2018)
Modersitzki, J.: FAIR: flexible algorithms for image registration. SIAM (2009)
Pishro-Nik, H.: Introduction to Probability, Statistics, and Random Processes. Kappa Research, Athens (2016)
Pitiot, A., Bardinet, E., Thompson, P.M., Malandain, G.: Piecewise affine registration of biological images for volume reconstruction. Med. Image Anal. 10(3), 465–483 (2006)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schütze, H., Manning, C.D., Raghavan, P.: Introduction to Information Retrieval, vol. 39. Cambridge University Press, Cambridge (2008)
Schwier, M., Böhler, T., Hahn, H.K., Dahmen, U., Dirsch, O.: Registration of histological whole slide images guided by vessel structures. J. Pathol. Inform. 4(Suppl) (2013)
Simonovsky, M., Gutiérrez-Becker, B., Mateus, D., Navab, N., Komodakis, N.: A deep metric for multimodal registration. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 10–18. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46726-9_2
So, R.W., Chung, A.C.: A novel learning-based dissimilarity metric for rigid and non-rigid medical image registration by using Bhattacharyya distances. Pattern Recogn. 62, 161–174 (2017)
Uzunova, H., Wilms, M., Handels, H., Ehrhardt, J.: Training CNNs for image registration from few samples with model-based data augmentation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 223–231. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_26
Wodzinski, M., Müller, H.: Unsupervised learning-based nonrigid registration of high resolution histology images. In: Liu, M., Yan, P., Lian, C., Cao, X. (eds.) MLMI 2020. LNCS, vol. 12436, pp. 484–493. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59861-7_49
Wodzinski, M., Müller, H.: DeephistReg: unsupervised deep learning registration framework for differently stained histology samples. Comput. Methods Programs Biomed. 198, 105799 (2021)
Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)
Acknowledgement
H.K. acknowledges support from the European Research Council (ERC) with the consolidator grant CONSYN (nr. 773196). O.C. is supported by the Alexander von Humboldt Foundation Philipp Schwartz Initiative.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cetin, O., Shu, Y., Flinner, N., Ziegler, P., Wild, P., Koeppl, H. (2022). Multi-magnification Networks for Deformable Image Registration on Histopathology Images. In: Hering, A., Schnabel, J., Zhang, M., Ferrante, E., Heinrich, M., Rueckert, D. (eds) Biomedical Image Registration. WBIR 2022. Lecture Notes in Computer Science, vol 13386. Springer, Cham. https://doi.org/10.1007/978-3-031-11203-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-11203-4_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-11202-7
Online ISBN: 978-3-031-11203-4
eBook Packages: Computer ScienceComputer Science (R0)