Abstract
In this paper, we propose a novel multi-view deep learning approach for cervical dysplasia diagnosis (CDD), using multi-views of image data (acetic images and iodine images) from colposcopy. In general, a major challenge to analyzing multi-view medical image data is how to effectively exploit meaningful correlations among such views. We develop a new feature level fusion (FLF) method, which captures comprehensive correlations between the acetic and iodine image views and sufficiently utilizes information from these two views. Our FLF method is based on attention mechanisms and allows one view to assist another view or allows both views to assist mutually to better facilitate feature learning. Specifically, we explore deep networks for two kinds of FLF methods, uni-directional fusion (UFNet) and bi-directional fusion (BFNet). Experimental results show that our methods are effective for characterizing features of cervical lesions and outperform known methods for CDD.
T. Chen, X. Ma and X. Liu—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
SIL is a precancerous lesion and abnormal growth of squamous cells on cervix surface.
- 2.
Also called Cervigrams if the screening method used is digital cervicography. Colposcopy and cervicography both apply 5% acetic acid to the cervix epithelium.
- 3.
HPV tests examine whether the cervix is infected by Human Papilloma Virus; Pap tests check whether cervical cells have abnormal changes related to HPV infection. No matter HPV or Pap test result is abnormal or not, colposcopy will be performed.
References
Guo, Z., Li, X., Huang, H., et al.: Medical image segmentation based on multi-modal convolutional neural network: Study on image fusion schemes. In: ISBI, pp. 903–907 (2018)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2015)
Jusman, Y., Ng, S.C., Abu Osman, N.A.: Intelligent screening systems for cervical cancer. Sci. World J. 2014(2), Article ID 810368 (2014)
Khan, M.J., Werner, C.L., Darragh, T.M., et al.: ASCCP colposcopy standards: role of colposcopy, benefits, potential harms, and terminology for colposcopic practice. J. Lower Genital Tract Dis. 21(4), 223 (2017)
Longatto-Filho, A., Naud, P., Derchain, S.F., et al.: Performance characteristics of Pap test, VIA, VILI, HR-HPV testing, cervicography, and colposcopy in diagnosis of significant cervical pathology. Virchows Arch. 460(6), 577–585 (2012)
Ren, S., He, K., Girshick, R., Jian, S.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NIPS, pp. 91–99 (2015)
Siegel, R.L., Miller, K.D., Jemal, A.: Cancer statistics, 2017. CA Cancer J. Clin. 67(1), 7–30 (2017)
Song, D., Kim, E., Huang, X., et al.: Multimodal entity coreference for cervical dysplasia diagnosis. IEEE TMI 34(1), 229–245 (2015)
Wang, F., Jiang, M., Qian, C., et al.: Residual attention network for image classification. In: CVPR, pp. 6450–6458 (2017)
Xu, T., Huang, X., Kim, E., et al.: Multi-test cervical cancer diagnosis with missing data estimation. In: Medical Imaging: CAD, vol. 9414, p. 94140X (2015)
Xu, T., Kim, E., Huang, X.: Adjustable AdaBoost classifier and pyramid features for image-based cervical cancer diagnosis. In: ISBI, pp. 281–285 (2015)
Xu, T., et al.: A new image data set and benchmark for cervical dysplasia classification evaluation. In: Zhou, L., Wang, L., Wang, Q., Shi, Y. (eds.) MLMI 2015. LNCS, vol. 9352, pp. 26–35. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24888-2_4
Xu, T., Zhang, H., Huang, X., Zhang, S., Metaxas, D.N.: Multimodal deep learning for cervical dysplasia diagnosis. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 115–123. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_14
Yang, X., et al.: Joint detection and diagnosis of prostate cancer in multi-parametric MRI based on multimodal convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 426–434. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_49
Yoo, Y., et al.: Hierarchical multimodal fusion of deep-learned lesion and tissue integrity features in brain MRIs for distinguishing neuromyelitis optica from multiple sclerosis. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 480–488. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_55
Zhou, B., Khosla, A., Lapedriza, A., et al.: Learning deep features for discriminative localization. In: CVPR, pp. 2921–2929 (2016)
Acknowledgment
The research of D.Z. Chen was supported in part by NSF Grant CCF-1617735. The research of the Real Doctor AI Research Centre was partially supported by the Subject of the Major Commissioned Project “Research on China’s Image in the Big Data” of Zhejiang Province’s Social Science Planning Advantage Discipline “Evaluation and Research on the Present Situation of China’s Image” No. 16YSXK01ZD-2YBMinistry of Education of China under grant No. 2017PT18, the Zhejiang University Education Foundation under grants No. K18-511120-004, No. K17-511120-017, and No. K17-518051-021, the Major Scientific Project of Zhejiang Lab under grant No. 2018DG0ZX01, the National Natural Science Foundation of China under grant No. 61672453, and the Key Laboratory of Medical Neurobiology of Zhejiang Province.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, T. et al. (2019). Multi-view Learning with Feature Level Fusion for Cervical Dysplasia Diagnosis. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11764. Springer, Cham. https://doi.org/10.1007/978-3-030-32239-7_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-32239-7_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32238-0
Online ISBN: 978-3-030-32239-7
eBook Packages: Computer ScienceComputer Science (R0)
