Abstract
The segmentation of corneal nerves in corneal confocal microscopy (CCM) is of great to the quantification of clinical parameters in the diagnosis of eye-related diseases and systematic diseases. Existing works mainly use convolutional neural networks to improve the segmentation accuracy, while further improvement is needed to mitigate the nerve discontinuity and noise interference. In this paper, we propose a novel corneal nerve segmentation network, named NerveFormer, to resolve the above-mentioned limitations. The proposed NerveFormer includes a Deformable and External Attention Module (DEAM), which exploits the Transformer-based Deformable Attention (TDA) and External Attention (TEA) mechanisms. TDA is introduced to explore the local internal nerve features in a single CCM, while TEA is proposed to model global external nerve features across different CCM images. Specifically, to efficiently fuse the internal and external nerve features, TDA obtains the query set required by TEA, thereby strengthening the characterization ability of TEA. Therefore, the proposed model aggregates the learned features from both single-sample and cross-sample, allowing for better extraction of corneal nerve features across the whole dataset. Experimental results on two public CCM datasets show that our proposed method achieves state-of-the-art performance, especially in terms of segmentation continuity and noise discrimination.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Annunziata, R., Kheirkhah, A., Hamrah, P., Trucco, E.: Scale and curvature invariant ridge detector for tortuous and fragmented structures. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 588–595. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_70
Chen, J., et al.: Transunet: transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)
Colonna, A., Scarpa, F., Ruggeri, A.: Segmentation of corneal nerves using a U-Net-based convolutional neural network. In: Stoyanov, D., et al. (eds.) OMIA/COMPAY -2018. LNCS, vol. 11039, pp. 185–192. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00949-6_22
Dabbah, M.A., Graham, J., Petropoulos, I., Tavakoli, M., Malik, R.A.: Dual-model automatic detection of nerve-fibres in corneal confocal microscopy images. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6361, pp. 300–307. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15705-9_37
Gao, Y., Zhou, M., Metaxas, D.N.: UTNet: a hybrid transformer architecture for medical image segmentation. In: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., Essert, C. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 61–71. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_6
Gu, Z., et al.: Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans. Med. Imaging 38(10), 2281–2292 (2019)
Guimarães, P., Wigdahl, J., Poletti, E., Ruggeri, A.: A fully-automatic fast segmentation of the sub-basal layer nerves in corneal images. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 5422–5425. IEEE (2014)
Guo, M.H., Liu, Z.N., Mu, T.J., Hu, S.M.: Beyond self-attention: External attention using two linear layers for visual tasks. arXiv preprint arXiv:2105.02358 (2021)
Ji, Y., Zhang, R., Wang, H., Li, Z., Wu, L., Zhang, S., Luo, P.: Multi-compound transformer for accurate biomedical image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 326–336. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_31
Mou, L., et al.: CS-net: channel and spatial attention network for curvilinear structure segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 721–730. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_80
Mou, L., et al.: Cs2-net: deep learning segmentation of curvilinear structures in medical imaging. Med.l Image Anal. 67, 101874 (2021)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Su, P., et al.: Corneal nerve tortuosity grading via ordered weighted averaging-based feature extraction. Med. Phys. 47(10), 4983–4996 (2020)
Su, P.Y., Hu, F.R., Chen, Y.M., Han, J.H., Chen, W.L.: Dendritiform cells found in central cornea by in-vivo confocal microscopy in a patient with mixed bacterial keratitis. Ocular Immunology Inflammation 14(4), 241–244 (2006)
Vaswani, A., et al.: Attention is all you need. Advances in Neural Information Processing Systems 30 (2017)
Wei, S., Shi, F., Wang, Y., Chou, Y., Li, X.: A deep learning model for automated sub-basal corneal nerve segmentation and evaluation using in vivo confocal microscopy. Trans. Visi. Sci. Technol. 9(2), 32–32 (2020)
Williams, B.M., et al.: An artificial intelligence-based deep learning algorithm for the diagnosis of diabetic neuropathy using corneal confocal microscopy: a development and validation study. Diabetologia 63(2), 419–430 (2019). https://doi.org/10.1007/s00125-019-05023-4
Yang, C., et al.: Multi-discriminator adversarial convolutional network for nerve fiber segmentation in confocal corneal microscopy images. IEEE J. Biomed. Health Inf. (2021)
Zhang, D., et al.: Automatic corneal nerve fiber segmentation and geometric biomarker quantification. Europ. Phys. J. Plus 135(2), 1–16 (2020). https://doi.org/10.1140/epjp/s13360-020-00127-y
Zhao, Y., et al.: Automated tortuosity analysis of nerve fibers in corneal confocal microscopy. IEEE Trans. Med. Imaging 39(9), 2725–2737 (2020)
Zhao, Y., et al.: Uniqueness-driven saliency analysis for automated lesion detection with applications to retinal diseases. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 109–118. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_13
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable DETR: deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020)
Acknowledgement
This work was supported in part by the National Science Foundation Program of China (62103398 and 61906181), Zhejiang Provincial Natural Science Foundation of China (LR22F020008), in part by the Youth Innovation Promotion Association CAS (2021298), in part by the Ningbo major science and technology task project (2021Z054) and in part by the AME Programmatic Fund (A20H4b0141).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, J. et al. (2022). NerveFormer: A Cross-Sample Aggregation Network for Corneal Nerve Segmentation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13434. Springer, Cham. https://doi.org/10.1007/978-3-031-16440-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-16440-8_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16439-2
Online ISBN: 978-3-031-16440-8
eBook Packages: Computer ScienceComputer Science (R0)
