Skip to main content
Springer Nature Link
Log in

RoTIR: Rotation-Equivariant Network and Transformers for Zebrafish Scale Image Registration

  • Conference paper
  • First Online:
Medical Image Understanding and Analysis (MIUA 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14859))

Included in the following conference series:

  • 562 Accesses

Abstract

Image registration is essential for aligning features of interest from multiple images. With the recent development of deep learning techniques, image registration approaches have advanced to a new level. In this work, we present Rotation-Equivariant network and Transformers for Image Registration (RoTIR), a deep-learning-based method for aligning zebrafish scale images captured by light microscopy. This approach overcomes the challenge of arbitrary rotation, translation detection, and the absence of ground truth data. We employ feature-matching approaches based on Transformers and general E(2)-equivariant steerable CNNs for model creation. Besides, an artificial training dataset is employed for semi-supervised learning. Results show that RoTIR successfully achieves the goal of zebrafish scale image registration.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The code presented in this paper will be made available after final publication at https://github.com/SpikeRXWong/RoTIR-Zebrafish-Scale-Image-Registration.

References

  1. Wang, C.-W., Ka, S.-M., Chen, A.: Robust image registration of biological microscopic images. Sci. Rep. 4, 6050 (2014)

    Article  MATH  Google Scholar 

  2. Kuppala, K., Banda, S., Barige, T.R.: An overview of deep learning methods for image registration with focus on feature-based approaches. Int. J. Image Data Fusion 11, 113–135 (2020)

    Article  MATH  Google Scholar 

  3. Bergen, D.J., Kague, E., Hammond, C.L.: Zebrafish as an emerging model for osteoporosis: a primary testing platform for screening new osteo-active compounds. Front. Endocrinol. 10, 6 (2019)

    Article  Google Scholar 

  4. Dietrich, K., et al.: Skeletal biology and disease modelling in zebrafish. J. Bone Miner. Res. 36, 436–458 (2020)

    Google Scholar 

  5. Fischer, B., Modersitzki, J.: Ill-posed medicine-an introduction to image registration. Inverse Problems 24(3), 0340080 (2008)

    Google Scholar 

  6. Chen, X., Diaz-Pinto, A., Ravikumar, N., Frangi, A.F.: Deep learning in medical image registration. Prog. Biomed. Eng. 3, 012003 (2021)

    Google Scholar 

  7. Weiler, M., Cesa, G.: General e (2)-equivariant steerable CNNs. Adv. Neural Inf. Process. Syst. 32 (2019)

    Google Scholar 

  8. Sun, J., Shen, Z., Wang, Y., Bao, H., Zhou, X.: Loftr: detector-free local feature matching with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8922–8931 (2021)

    Google Scholar 

  9. Liu, Y., Gong, X., Chen, J., Chen, S., Yang, Y.: Rotation-invariant siamese network for low-altitude remote-sensing image registration. IEEE J. Select. Topics Appl. Earth Observ. Remote Sens. 13, 5746–5758 (2020)

    Article  MATH  Google Scholar 

  10. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 91–110 (2004)

    Article  MATH  Google Scholar 

  11. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110, 346–359 (2008)

    Article  Google Scholar 

  12. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: rublee2011orb. In: Proceedings of International Conference on Computer Vision, pp. 2564–2571 (2011)

    Google Scholar 

  13. Bökman, G., Kahl, F.: A case for using rotation invariant features in state of the art feature matchers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5110–5119 (2022)

    Google Scholar 

  14. Cohen, T.S., Welling, M.: Group Equivariant Convolutional Networks. In: International Conference on Machine Learning, PMLR, pp. 2990–2999 (2016)

    Google Scholar 

  15. Lenssen, J.E., Fey, M., Libuschewski, P.: Group equivariant capsule networks. Adv. Neural Inf. Process. Syst. 31 (2018)

    Google Scholar 

  16. Cohen, T.S., Geiger, M., Weiler, M.: A general theory of equivariant CNNs on homogeneous spaces. Adv. Neural Inf. Process. Syst. 32 (2019)

    Google Scholar 

  17. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

    Google Scholar 

  18. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 213–229. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_13

  19. Xia, Y., Li, Y., Xun, L., Yan, Q., Zhang, D.: A convolutional neural network cascade for plantar pressure images registration. Gait Posture 68, 403–408 (2019)

    Article  MATH  Google Scholar 

  20. Cuturi, M.: Sinkhorn distances: Lightspeed computation of optimal transport. Adv. Neural Inf. Process. Syst. 26 (2013)

    Google Scholar 

  21. Sampat, M.P., Wang, Z., Gupta, S., Bovik, A.C., Markey, M.K.: Complex wavelet structural similarity: a new image similarity index. IEEE Trans. Image Process. 18, 2385–2401 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  22. DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: self-supervised interest point detection and description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 224–236 (2018)

    Google Scholar 

  23. Sarlin, P.-E., DeTone, D., Malisiewicz, T., Rabinovich, A.: Superglue: learning feature matching with graph neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4938–4947 (2020)

    Google Scholar 

Download references

Acknowledgments

This work was carried out using the BlueCrystal Phase 4 and BluePebble facility of the Advanced Computing Research Centre, University of Bristol (http://www.bristol.ac.uk/acrc/). We thank the Wolfson Bioimaging Facility (Bristol) for providing access to the Incucyte Zoom live cell imaging system.

Author information

Authors and Affiliations

  1. Visual Information Laboratory, University of Bristol, Bristol, BS1 5DD, UK

    Ruixiong Wang & Alin Achim

  2. School of Biochemistry, University of Bristol, Bristol, BS8 1TD, UK

    Renata Raele-Rolfe

  3. School of Physiology, Pharmacology and Neuroscience, University of Bristol, Bristol, BS8 1TD, UK

    Qiao Tong, Dylan Bergen & Chrissy Hammond

  4. Wolfson Bioimaging Facility, University of Bristol, Bristol, BS8 1TD, UK

    Stephen Cross

Authors
  1. Ruixiong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alin Achim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Renata Raele-Rolfe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiao Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dylan Bergen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chrissy Hammond
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Stephen Cross
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruixiong Wang .

Editor information

Editors and Affiliations

  1. Manchester Metropolitan University, Manchester, Lancashire, UK

    Moi Hoon Yap

  2. Manchester Metropolitan University, Manchester, UK

    Connah Kendrick

  3. Edge Hill University, Ormskirk, UK

    Ardhendu Behera

  4. Aberystwyth University, Aberystwyth, UK

    Timothy Cootes

  5. Aberystwyth University, Aberystwyth, UK

    Reyer Zwiggelaar

Ethics declarations

Compliance with Ethical Standards

All zebrafish experiments were approved by the local Animal Welfare and Ethical Review Board (Bristol AWERB), and were conducted under a Home Office Project Licence.

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, R. et al. (2024). RoTIR: Rotation-Equivariant Network and Transformers for Zebrafish Scale Image Registration. In: Yap, M.H., Kendrick, C., Behera, A., Cootes, T., Zwiggelaar, R. (eds) Medical Image Understanding and Analysis. MIUA 2024. Lecture Notes in Computer Science, vol 14859. Springer, Cham. https://doi.org/10.1007/978-3-031-66955-2_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-66955-2_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-66954-5

  • Online ISBN: 978-3-031-66955-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics