skip to main content
10.1145/3633637.3633688acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccprConference Proceedingsconference-collections
research-article

MCARNet: Mixed Convolution-Attention-Residual Network for Hyperspectral Image Classification of Cholangiocarcinoma

Published: 28 February 2024 Publication History

Abstract

Hyperspectral is a well-established form of data in the medical field and plays an irreplaceable role, reflecting not only the size, shape and structure of the target pathological areas, but also the differences in spectral between the chemical composition of the targets. Therefore, hyperspectral image(HSI) can provide more abundant information for medical practitioners when performing pathology analysis. In recent years, while two powerful representational learning methods, which have been fully developed in the long term in deep learning, namely convolution and attention mechanism, have both made significant contributions to the pathological analysis of hyperspectral medical images. However, they have also revealed respective shortcomings in previous works, such as insufficient support of the biological basis of convolution and its weak ability to capture global information, the insufficient ability of attention mechanism to acquire spatial texture features, and both have over-reliance on sample quantity as well as quality. Therefore, this article proposes a Mixed Convolution-Attention-Residual(MCAR) model and a corresponding MCARNet network for the HSI classification task of cholangiocarcinoma by effectively combining multi-branch convolution, attention mechanism and residual structure. We also effectively validate that the proposed model can perform better in the task by parallel extraction of global with local features as well as the construction of residual structure under the condition of few training samples.

References

[1]
National Institutes of Health. (2019). NCI Dictionary of Cancer Terms‐National Cancer Institute. Website: https://www. cancer. gov/publi catio ns/dicti onari es/cancer‐terms. Accessed March, 18.
[2]
Board, P. A. T. E. (2002). Bile Duct Cancer (Cholangiocarcinoma) Treatment (PDQ®): Health Professional Version. PDQ Cancer Information Summaries [Internet].
[3]
World Health Organization. (2019). International agency for research on cancer.
[4]
Bridgewater, J. A., Goodman, K. A., Kalyan, A., & Mulcahy, M. F. (2016). Biliary tract cancer: epidemiology, radiotherapy, and molecular profiling. American Society of Clinical Oncology Educational Book, 36, e194-e203.
[5]
Lu, G., & Fei, B. (2014). Medical hyperspectral imaging: a review. Journal of biomedical optics, 19(1), 010901-010901.
[6]
Kiyotoki, S., Nishikawa, J., Okamoto, T., Hamabe, K., Saito, M., Goto, A., ... & Sakaida, I. (2013). New method for detection of gastric cancer by hyperspectral imaging: a pilot study. Journal of biomedical optics, 18(2), 026010-026010.
[7]
Larsen, E. L., Randeberg, L. L., Olstad, E., Haugen, O. A., Aksnes, A., & Svaasand, L. O. (2011). Hyperspectral imaging of atherosclerotic plaques in vitro. Journal of biomedical optics, 16(2), 026011-026011.
[8]
Li, X., Li, W., Xu, X., & Hu, W. (2017, June). Cell classification using convolutional neural networks in medical hyperspectral imagery. In 2017 2nd international conference on image, vision and computing (ICIVC) (pp. 501-504). IEEE.
[9]
Lu, G., & Fei, B. (2014). Medical hyperspectral imaging: a review. Journal of biomedical optics, 19(1), 010901-010901.
[10]
Jia, S., Jiang, S., Lin, Z., Li, N., Xu, M., & Yu, S. (2021). A survey: Deep learning for hyperspectral image classification with few labeled samples. Neurocomputing, 448, 179-204.
[11]
Huang, Q., Li, W., & Xie, X. (2018, June). Convolutional neural network for medical hyperspectral image classification with kernel fusion. In BIBE 2018; International Conference on Biological Information and Biomedical Engineering (pp. 1-4). VDE.
[12]
Li, X., Li, W., Xu, X., & Hu, W. (2017, June). Cell classification using convolutional neural networks in medical hyperspectral imagery. In 2017 2nd international conference on image, vision and computing (ICIVC) (pp. 501-504). IEEE.
[13]
Huang, Q., Li, W., Zhang, B., Li, Q., Tao, R., & Lovell, N. H. (2019). Blood cell classification based on hyperspectral imaging with modulated Gabor and CNN. IEEE journal of biomedical and health informatics, 24(1), 160-170.
[14]
Wei, X., Li, W., Zhang, M., & Li, Q. (2019). Medical hyperspectral image classification based on end-to-end fusion deep neural network. IEEE Transactions on Instrumentation and Measurement, 68(11), 4481-4492.
[15]
Bengs, M., Gessert, N., Laffers, W., Eggert, D., Westermann, S., Mueller, N. A., ... & Schlaefer, A. (2020). Spectral-spatial recurrent-convolutional networks for in-vivo hyperspectral tumor type classification. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23 (pp. 690-699). Springer International Publishing.
[16]
Manni, F., van der Sommen, F., Fabelo, H., Zinger, S., Shan, C., Edström, E., ... & de With, P. H. (2020). Hyperspectral imaging for glioblastoma surgery: Improving tumor identification using a deep spectral-spatial approach. Sensors, 20(23), 6955.
[17]
Woo, S., Park, J., Lee, J. Y., & Kweon, I. S. (2018). Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV) (pp. 3-19).
[18]
Srinivas, A., Lin, T. Y., Parmar, N., Shlens, J., Abbeel, P., & Vaswani, A. (2021). Bottleneck transformers for visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 16519-16529).
[19]
Bello, I., Zoph, B., Vaswani, A., Shlens, J., & Le, Q. V. (2019). Attention augmented convolutional networks. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 3286-3295).
[20]
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 (pp. 234-241). Springer International Publishing.
[21]
Yang, J., Zhao, Y. Q., & Chan, J. C. W. (2017). Learning and transferring deep joint spectral–spatial features for hyperspectral classification. IEEE Transactions on Geoscience and Remote Sensing, 55(8), 4729-4742.
[22]
Chen, Y., Lin, Z., Zhao, X., Wang, G., & Gu, Y. (2014). Deep learning-based classification of hyperspectral data. IEEE Journal of Selected topics in applied earth observations and remote sensing, 7(6), 2094-2107.
[23]
Liu, Z., Mao, H., Wu, C. Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022). A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11976-11986).
[24]
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
[25]
Zhang, J., Meng, Z., Zhao, F., Liu, H., & Chang, Z. (2022). Convolution transformer mixer for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 19, 1-5.
[26]
Xue, H., Huynh, D. Q., & Reynolds, M. (2018, March). SS-LSTM: A hierarchical LSTM model for pedestrian trajectory prediction. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 1186-1194). IEEE.
[27]
Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., & Sun, J. (2021). Repvgg: Making vgg-style convnets great again. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 13733-13742).

Index Terms

  1. MCARNet: Mixed Convolution-Attention-Residual Network for Hyperspectral Image Classification of Cholangiocarcinoma

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICCPR '23: Proceedings of the 2023 12th International Conference on Computing and Pattern Recognition
    October 2023
    589 pages
    ISBN:9798400707988
    DOI:10.1145/3633637
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 February 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Medical image classification
    2. hyperspectral image
    3. spatial-spectral features

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICCPR 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 21
      Total Downloads
    • Downloads (Last 12 months)21
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media