Skip to main content
SpringerLink
Log in

A Hybrid Supervised Fusion Deep Learning Framework for Microscope Multi-Focus Images

  • Conference paper
  • First Online:
Advances in Computer Graphics (CGI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14498))

Included in the following conference series:

  • 226 Accesses

Abstract

The quality of multi-focus microscopic image fusion hinges upon the precision of the image registration technology. However, algorithms for registration tailored specifically for multifocal microscopic images are lacking. Due to the presence of fuzzy regions and weak textures of multi-focus microscope images, the registration of patches is suboptimal. For these problems, this paper formulates a hybrid supervised deep learning model. It can improve the accuracy of registration and fusion. The generalization ability of the model to the actual deformation field enhance by the artificial deformation field. A step of patch movement simulation is employed to blur the multi-focus microscopic images and make synthetic flow, thus emulating distinct fuzzy regions in the two images to be registered, consequently enhancing the model's generalization ability. The experiments demonstrate that our proposed approach is superior to the existing registration algorithms and improves the accuracy of image fusion.

Q. Yang and H. Chen—Co-first author.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)

    Article  Google Scholar 

  2. Li, Z., Mahapatra, D., Tielbeek, J.A., Stoker, J., van Vliet, L.J., Vos, F.M.: Image registration based on autocorrelation of local structure. IEEE Trans. Image Process. 35, 63–75 (2015)

    Article  Google Scholar 

  3. Cao, S.Y., Shen, H.L., Chen, S.J., Li, C.: Boosting structure consistency for multispectral and multimodal image registration. IEEE Trans. Image Process. 29, 5147–5162 (2020)

    Article  Google Scholar 

  4. Dong, Y., Long, T., Jiao, W., He, G., Zhang, Z.: A novel image registration method based on phase correlation using low-rank matrix factorization with mixture of Gaussian. IEEE Trans. Geosci. Remote Sens. 56, 446–460 (2017)

    Article  Google Scholar 

  5. Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey Vision Conference, pp. 147–151 (1988)

    Google Scholar 

  6. Lowe, D.G.: Object recognition from local scale-invariant features. J. Comput. Vis. 60, 91–110 (2004)

    Article  Google Scholar 

  7. Bay, H., Tuytelaars, T., Gool, L.: Surf: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_32

    Chapter  Google Scholar 

  8. Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 430–443. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_34

    Chapter  Google Scholar 

  9. Alcantarilla, P.F., Bartoli, A., Davison, A.J.: KAZE features. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 214–227. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_16

    Chapter  Google Scholar 

  10. Ma, J., Zhou, H., Zhao, J., Gao, Y., Jiang, J., Tian, J.: Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans. Geosci. Remote Sens. 53, 6469–6481 (2015)

    Article  Google Scholar 

  11. Ma, J., Zhao, J., Jiang, J., Zhou, H., Guo, X.: Locality preserving matching. Int. J. Comput. Vision 127, 512–531 (2019)

    Article  MathSciNet  Google Scholar 

  12. Ma, J., Jiang, J., Zhou, H., Zhao, J., Guo, X.: Guided locality preserving feature matching for remote sensing image registration. IEEE Trans. Geosci. Remote Sens. 56, 4435–4447 (2018)

    Article  Google Scholar 

  13. Wu, G., Kim, M., Wang, Q., Munsell, B.C., Shen, D.: Scalable high-performance image registration framework by unsupervised deep feature representations learning. IEEE Trans. Biomed. Eng. 63, 1505–1516 (2015)

    Article  Google Scholar 

  14. Gao, Y., Dai, M., Zhang, Q.: Cross-modal and multi-level feature refinement network for RGB-D salient object detection. Vis. Comput. 39, 3979–3994 (2023). https://doi.org/10.1007/s00371-022-02543-w

    Article  Google Scholar 

  15. Salehi, S.S.M., Khan, S., Erdogmus, D., Gholipour, A.: Real-time deep pose estimation with geodesic loss for image-to-template rigid registration. IEEE Trans. Med. Imaging 38, 470–481 (2018)

    Article  Google Scholar 

  16. Jaderberg, M., Simonyan, K., Zisserman, A.: Spatial transformer networks. Adv. Neural. Inf. Process. Syst. 28, 2017–2025 (2015)

    Google Scholar 

  17. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

    Google Scholar 

  18. Mahapatra, D., Ge, Z., Sedai, S., Chakravorty, R.: Joint registration and segmentation of xray images using generative adversarial networks. In: Shi, Y., Suk, H.-I., Liu, M. (eds.) MLMI 2018. LNCS, vol. 11046, pp. 73–80. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00919-9_9

    Chapter  Google Scholar 

  19. Bai, X., Zhang, Y., Zhou, F., Xue, B.: Quadtree-based multi-focus image fusion using a weighted focus-measure. Inf. Fusion 22, 105–118 (2015)

    Article  Google Scholar 

  20. Panguluri, S.K., Mohan, L.: An effective fuzzy logic and particle swarm optimization based thermal and visible-light image fusion framework using curvelet transform. Optik 243, 167529 (2021)

    Article  Google Scholar 

  21. Roy, M., Mukhopadhyay, S.: A DCT-based multiscale framework for 2D greyscale image fusion using morphological differential features. Vis. Comput. (2023). https://doi.org/10.1007/s00371-023-03052-0

  22. Liu, Y., Chen, X., Peng, H., Wang, Z.: Multi-focus image fusion with a deep convolutional neural network. Inf. Fusion 36, 191–207 (2017)

    Article  Google Scholar 

  23. Guo, X., Nie, R., Cao, J., Zhou, D., Qian, W.: Fully convolutional network-based multifocus image fusion. Neural Comput. 30, 1775–1800 (2018)

    Article  MathSciNet  Google Scholar 

  24. Xie, Z., Zhang, W., Sheng, B., Li, P., Chen, C.P.: BaGFN: broad attentive graph fusion network for high-order feature interactions. IEEE Trans. Neural Netw. Learn. Syst. 34(8), 4499–4513 (2023)

    Article  Google Scholar 

  25. Zhou, Y., Chen, Z., Sheng, B., Li, P., Kim, J., Wu, E.: AFF-Dehazing: attention-based feature fusion network for low-light image Dehazing. Comput. Animat. Virtual Worlds 32(3–4), e2011 (2021)

    Article  Google Scholar 

Download references

Acknowledgment

The project was supported by the Project of National Natural Science Foundation of China (No.62002268, No.61961036), the Guangxi science and technology major special projects innovation driven major projects (No. AA18118036), Natural Science Foundation of Guangxi (No. 2021JJB170060) and Macao Polytechnic University Grant (No. RP/FCA-15/2022).

Author information

Authors and Affiliations

  1. School of Faculty of Applied Sciences, Macao Polytechnic University, Macao, China

    Qiuhui Yang, Yue Sun & Tao Tan

  2. Guangxi Key Laboratory of Machine Vision and Intelligent Control, Wuzhou University, Wuzhou, China

    Qiuhui Yang

  3. Jiangsu JITRI Sioux Technologies Co., Ltd., Suzhou, China

    Hao Chen

  4. School of Computer Sciences and Technology, Zhejiang Sci-Tech University, Hangzhou, Zhejiang, China

    Mingfeng Jiang

  5. Department of Cardiovascular Medicine, Affiliated Hospital of Hangzhou Normal University, Clinical School of Medicine, Hangzhou Normal University, Hangzhou Institute of Cardiovascular Diseases, Hangzhou, China

    Mingwei Wang

  6. Cixi Institute of Biomedical Engineering, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, China

    Jiong Zhang

  7. The Affiliated Ningbo Eye Hospital of Wenzhou Medical University, Ningbo, China

    Jiong Zhang

Authors
  1. Qiuhui Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingfeng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mingwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yue Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tao Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Tan .

Editor information

Editors and Affiliations

  1. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

  2. Shanghai Jiao Tong University, Shanghai, China

    Lei Bi

  3. University of Sydney, Sydney, NSW, Australia

    Jinman Kim

  4. MIRALab-CUI, University of Geneve, Carouge, Geneve, Switzerland

    Nadia Magnenat-Thalmann

  5. Swiss federal Institute of Technology, Lausanne, Switzerland

    Daniel Thalmann

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yang, Q. et al. (2024). A Hybrid Supervised Fusion Deep Learning Framework for Microscope Multi-Focus Images. In: Sheng, B., Bi, L., Kim, J., Magnenat-Thalmann, N., Thalmann, D. (eds) Advances in Computer Graphics. CGI 2023. Lecture Notes in Computer Science, vol 14498. Springer, Cham. https://doi.org/10.1007/978-3-031-50078-7_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-50078-7_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-50077-0

  • Online ISBN: 978-3-031-50078-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation