Abstract
Automated and accurate classification of histopathological images of pancreatic cancer can lead to higher survival rates for more pancreatic cancer patients in the clinic. However, there are very scarce existing studies for pancreatic cancer, and the diagnosis of pancreatic cancer remains a challenge for pathologists, especially for well-differentiated pancreatic cancer with a clinical histological pattern similar to that of chronic pancreatitis. We propose a hybrid CNN-Transformer model incorporating deformable atrous spatial pyramids (DACTransNet) to perform automated and accurate classification of histopathological images of pancreatic cancer. We elegantly integrate the powerful local feature extraction capability of CNN for spatial features and the global modeling capability of transformer for abstract patterns. Moreover, we imitate pathologists in the clinic by better integrating deformable convolution and multiscale methods to review histopathology slides in pyramidal format. In addition, a migration learning approach was used to improve the classification accuracy of pancreatic cancer histopathology images. The experimental results show that the proposed method not only has a high classification accuracy (up to 96%), but also its good robustness and generalizability as validated by real clinical datasets from multiple centers. Consequently, we provide an effective tool for the clinical diagnosis of pancreatic cancer.
Y. Kou and C. Xia—Contribute equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Siegel, R.L., Miller, K.D., Wagle, N.S., Jemal, A.: Cancer statistics, 2023. CA: Cancer J. Clinicians 73(1), 17–48 (2023)
Pereira, S.P., et al.: Early detection of pancreatic cancer. Lancet. Gastroenterol. Hepatol. (2020)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2015)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269 (2017)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv, abs/2010.11929 (2020)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9992–10002 (2021)
Chen, H., Dou, Q., Wang, X., Qin, J., Heng, P.-A.: Mitosis detection in breast cancer histology images via deep cascaded networks. In: AAAI Conference on Artificial Intelligence (2016)
Tian, Y., et al.: Computer-aided detection of squamous carcinoma of the cervix in whole slide images. arXiv, abs/1905.10959 (2019)
Fu, H., et al.: Automatic pancreatic ductal adenocarcinoma detection in whole slide images using deep convolutional neural networks. Front. Oncol. 11 (2021)
Yang, H., et al.: Deep learning-based six-type classifier for lung cancer and mimics from histopathological whole slide images: a retrospective study. BMC Med. 19 (2021)
Coudray, N., et al.: Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018)
Ianni, J.D., et al.: Tailored for real-world: a whole slide image classification system validated on uncurated multi-site data emulating the prospective pathology workload. Sci. Rep. 10 (2020)
Liu, M., Lanlan, H., Tang, Y., Chu Wang, Yu., He, C.Z., et al.: A deep learning method for breast cancer classification in the pathology images. IEEE J. Biomed. Health Inform. 26, 5025–5032 (2022)
Vuong, T.T.L., Song, B., Kim, K., Cho, Y.M., Kwak, J.T.: Multi-scale binary pattern encoding network for cancer classification in pathology images. IEEE J. Biomed. Health Inform. 26, 1152–1163 (2021)
Zhang, H., et al.: DTFD-mil: double-tier feature distillation multiple instance learning for histopathology whole slide image classification. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18780–18790 (2022)
Hou, W., Huang, H., Peng, Q., Yu, R., Yu, L., Wang, L.: Spatial-hierarchical graph neural network with dynamic structure learning for histological image classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2022)
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., J’egou, H.: Training data-efficient image transformers & distillation through attention. arXiv, abs/2012.12877 (2020)
Chen, H., et al.: Gashis-transformer: a multi-scale visual transformer approach for gastric histopathological image detection. Pattern Recogn. 130, 108827 (2021)
Shao, Z., Bian, H., Chen, Y., Wang, Y., Zhang, J., et al.: TransMil: transformer based correlated multiple instance learning for whole slide image classication. In: Neural Information Processing Systems (2021)
Xiong, Y., et al.: Nyströmformer: a nyström-based algorithm for approximating self-attention. In: AAAI Conference on Artificial Intelligence, vol. 35, pp. 16:14138–16:14148 (2021)
Zhang, T., Yunlu Feng, Yu., Zhao, G.F., Yang, A., Lyu, S., et al.: MSHT: multi-stage hybrid transformer for the rose image analysis of pancreatic cancer. IEEE J. Biomed. Health Inform. 27, 1946–1957 (2021)
Zheng, Y., Gindra, R., Green, E., Burks, E.J., Betke, M., Beane, J.E., et al.: A graph-transformer for whole slide image classification. IEEE Trans. Med. Imaging 41, 3003–3015 (2022)
Wu, H., et al.: CVT: introducing convolutions to vision transformers. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 22–31 (2021)
Dai, J., et al.: Deformable convolutional networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 764–773 (2017)
The cancer genome atlas (TCGA) (2016). http://cancergenome.nih.gov/
Jiao, Y., Li, J., Fei, S.M.: Staining condition visualization in digital histopathological whole-slide images. Multimedia Tools Appl. 81, 17831–17847 (2022)
Acknowledgements
This study was supported by the Natural Science Foundation of Jiangsu Province (No. BK20210291), the National Natural Science Foundation (No. 62101249 and No. 62136004), and the China Postdoctoral Science Foundation (No. 2021TQ0149 and No. 2022M721611).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Kou, Y., Xia, C., Jiao, Y., Zhang, D., Ge, R. (2024). DACTransNet: A Hybrid CNN-Transformer Network for Histopathological Image Classification of Pancreatic Cancer. In: Fang, L., Pei, J., Zhai, G., Wang, R. (eds) Artificial Intelligence. CICAI 2023. Lecture Notes in Computer Science(), vol 14474. Springer, Singapore. https://doi.org/10.1007/978-981-99-9119-8_38
Download citation
DOI: https://doi.org/10.1007/978-981-99-9119-8_38
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9118-1
Online ISBN: 978-981-99-9119-8
eBook Packages: Computer ScienceComputer Science (R0)