Taimoor Shakeel Sheikh
ABSTRACT
Automated classification of Whole Slide Images (WSIs) is of great significance for early diagnosis of cancer. Existing approaches are trained on a specific level which affects the analysis performance due to weak supervision of patches and variants. Additionally, it is difficult to distinguish cancer subtype patches accurately from different magnification levels of WSIs. However, this can be improved by employing artificial intelligence models to address these problems, we propose a novel clustering-based cancer diagnosis (CBCD) method for WSI classification. The CBCD constructs three modules: first, we extracted patches from each magnification level of WSIs with respective cancer sub-types. Second, we employed two features (global and local) to learn discriminative and salient information of each patch. Then we find the meaningful cluster regions based on these features to quantify (select) the best patches of salient cancer subtypes by only relying on the collective characteristics of patches from different magnification levels. The clustering techniques used are k-means, gaussian mixture model, and agglomerative clustering. The quality of each clustering technique was determined using adjusted rand, and calinski harabasz scores. Later we used five state-of-the-art (SOTA) deep learning models to learn and classify cancer subtype regions of WSIs based on two types of features of patches. We also showed the results with no clustering techniques in an end-to-end supervised way by directly extracting patches from WSIs. Our method is evaluated on the public WSI dataset (KBSMC) for cancer sub-types classification and achieves better performance and great interpretability compared with the SOTA methods.
- ATTALLAH, O., ANWAR, F., GHANEM, N. M., AND ISMAIL, M. A. Histo-cadx: duo cascaded fusion stages for breast cancer diagnosis from histopathological images. PeerJ Computer Science 7 (2021), e493.Google Scholar
Cross Ref
- BARKER, J., HOOGI, A., DEPEURSINGE, A., AND RUBIN, D. L. Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles. Medical image analysis 30 (2016), 60–71.Google Scholar
- BENTAIEB, A., AND HAMARNEH, G. Predicting cancer with a recurrent visual attention model for histopathology images. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part II 11 (2018), Springer, pp. 129–137.Google Scholar
- CAMPANELLA, G., HANNA, M. G., GENESLAW, L., MIRAFLOR, A., WERNECK KRAUSS SILVA, V., BUSAM, K. J., BROGI, E., REUTER, V. E., KLIMSTRA, D. S., AND FUCHS, T. J. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature medicine 25, 8 (2019), 1301–1309.Google Scholar
- CHIKONTWE, P., KIM, M., NAM, S. J., GO, H., AND PARK, S. H. Multiple instance learning with center embeddings for histopathology classification. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part V 23 (2020), Springer, pp. 519–528.Google ScholarDigital Library
- GERTYCH, A., ING, N., MA, Z., FUCHS, T. J., SALMAN, S., MOHANTY, S., BHELE, S., VELASQUEZ-VACCA, A., AMIN, M. B., AND KNUDSEN, B. S. Machine learning approaches to analyze histological images of tissues from radical prostatectomies. Computerized Medical Imaging and Graphics 46 (2015), 197–208.Google ScholarCross Ref
- HOU, L., SAMARAS, D., KURC, T. M., GAO, Y., DAVIS, J. E., AND SALTZ, J. H. Patch-based convolutional neural network for whole slide tissue image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (2016), pp. 2424–2433.Google ScholarCross Ref
- ILSE, M., TOMCZAK, J., AND WELLING, M. Attention-based deep multiple instance learning. In International conference on machine learning (2018), PMLR, pp. 2127–2136.Google Scholar
- LE VUONG, T. T., KIM, K., SONG, B., AND KWAK, J. T. Joint categorical and ordinal learning for cancer grading in pathology images. Medical image analysis 73 (2021), 102206.Google Scholar
- LI, B., LI, Y., AND ELICEIRI, K. W. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2021), pp. 14318–14328.Google ScholarCross Ref
- LI, Y., WU, J., AND WU, Q. Classification of breast cancer histology images using multi-size and discriminative patches based on deep learning. Ieee Access 7 (2019), 21400–21408.Google ScholarCross Ref
- LIU, L., FENG, W., CHEN, C., LIU, M., QU, Y., AND YANG, J. Classification of breast cancer histology images using msmv-pfenet. Scientific Reports 12, 1 (2022), 17447.Google Scholar
- LU, M. Y., CHEN, R. J., WANG, J., DILLON, D., AND MAHMOOD, F. Semi-supervised histology classification using deep multiple instance learning and contrastive predictive coding. arXiv preprint arXiv:1910.10825 (2019).Google Scholar
- LU, M. Y., WILLIAMSON, D. F., CHEN, T. Y., CHEN, R. J., BARBIERI, M., AND MAHMOOD, F. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature biomedical engineering 5, 6 (2021), 555–570.Google Scholar
- MERCAN, C., AKSOY, S., MERCAN, E., SHAPIRO, L. G., WEAVER, D. L., AND ELMORE, J. G. Multi-instance multi-label learning for multi-class classification of whole slide breast histopathology images. IEEE transactions on medical imaging 37, 1 (2017), 316–325.Google Scholar
- MERCAN, C., AKSOY, S., MERCAN, E., SHAPIRO, L. G., WEAVER, D. L., AND ELMORE, J. G. Multi-instance multi-label learning for multi-class classification of whole slide breast histopathology images. IEEE transactions on medical imaging 37, 1 (2017), 316–325.Google Scholar
- RELAN, D., MACGILLIVRAY, T., BALLERINI, L., AND TRUCCO, E. Retinal vessel classification: sorting arteries and veins. In 2013 35th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2013), IEEE, pp. 7396–7399.Google Scholar
- REN, J., HACIHALILOGLU, I., SINGER, E. A., FORAN, D. J., AND QI, X. Adversarial domain adaptation for classification of prostate histopathology whole-slide images. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part II 11 (2018), Springer, pp. 201–209.Google ScholarDigital Library
- SHEIKH, T. S., LEE, Y., AND CHO, M. Histopathological classification of breast cancer images using a multi-scale input and multi-feature network. Cancers 12, 8 (2020), 2031.Google ScholarCross Ref
- SHRIVASTAVA, A., KANT, K., SENGUPTA, S., KANG, S.-J., KHAN, M., ALI, S. A., MOORE, S. R., AMADI, B. C., KELLY, P., BROWN, D. E., ET AL. Deep learning for visual recognition of environmental enteropathy and celiac disease. In 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI) (2019), IEEE, pp. 1–4.Google ScholarCross Ref
- TELLEZ, D., LITJENS, G., VAN DER LAAK, J., AND CIOMPI, F. Neural image compression for gigapixel histopathology image analysis. IEEE transactions on pattern analysis and machine intelligence 43, 2 (2019), 567–578.Google Scholar
- THAWANI, R., MCLANE, M., BEIG, N., GHOSE, S., PRASANNA, P., VELCHETI, V., AND MADABHUSHI, A. Radiomics and radiogenomics in lung cancer: a review for the clinician. Lung cancer 115 (2018), 34–41.Google Scholar
- TONG, L., HOFFMAN, R., DESHPANDE, S. R., AND WANG, M. D. Predicting heart rejection using histopathological whole-slide imaging and deep neural network with dropout. In 2017 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI) (2017), IEEE, pp. 1–4.Google ScholarCross Ref
- WANG, D., KHOSLA, A., GARGEYA, R., IRSHAD, H., AND BECK, A. H. Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016).Google Scholar
- WANG, X., CHEN, H., GAN, C., LIN, H., DOU, Q., TSOUGENIS, E., HUANG, Q., CAI, M., AND HENG, P.-A. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE transactions on cybernetics 50, 9 (2019), 3950–3962.Google Scholar
- WU, H., LYU, X., AND WEN, Z. Automatic texture exemplar extraction based on global and local textureness measures. Computational Visual Media 4 (2018), 173–184.Google ScholarCross Ref
- XIA, Y., NIE, L., ZHANG, L., YANG, Y., HONG, R., AND LI, X. Weakly supervised multilabel clustering and its applications in computer vision.IEEE transactions on cybernetics 46, 12 (2016), 3220–3232.Google ScholarCross Ref
- XIAO, Y., LIU, B., HAO, Z., AND CAO, L. A similarity-based classification framework for multiple-instance learning. IEEE transactions on cybernetics 44, 4 (2013), 500–515.Google Scholar
- XIE, C., MUHAMMAD, H., VANDERBILT, C. M., CASO, R., YARLAGADDA, D. V. K., CAMPANELLA, G., AND FUCHS, T. J. Beyond classification: Whole slide tissue histopathology analysis by end-to-end part learning. In Medical Imaging with Deep Learning (2020), PMLR, pp. 843–856.Google Scholar
- YAO, J., ZHU, X., JONNAGADDALA, J., HAWKINS, N., AND HUANG, J. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Medical Image Analysis 65 (2020), 101789.Google ScholarCross Ref
- YOSHIDA, H., SHIMAZU, T., KIYUNA, T., MARUGAME, A., YAMASHITA, Y., COSATTO, E., TANIGUCHI, H., SEKINE, S., AND OCHIAI, A. Automated histological classification of whole-slide images of gastric biopsy specimens. Gastric Cancer 21, 2 (2018), 249–257.Google ScholarCross Ref
- ZHU, W., LOU, Q., VANG, Y. S., AND XIE, X. Deep multi-instance networks with sparse label assignment for whole mammogram classification. In International Conference on Medical Image Computing and Computer-Assisted Intervention (2017), Springer, pp. 603–611.Google ScholarDigital Library
- ZHU, X., YAO, J., ZHU, F., AND HUANG, J. Wsisa: Making survival prediction from whole slide histopathological images. In Proceedings of the IEEE conference on computer vision and pattern recognition (2017), pp. 7234–7242.Google ScholarCross Ref
Index Terms
- Clustering-Based Cancer Diagnosis Model for Whole Slide Image
Recommendations
Histological Detection of High-Risk Benign Breast Lesions from Whole Slide Images
Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017AbstractAccurate diagnosis of high-risk benign breast lesions is crucial in patient management since they are associated with an increased risk of invasive breast cancer development. Since it is not yet possible to identify the occult cancer patients ...
Deep learning with whole slide images can improve the prognostic risk stratification with stage III colorectal cancer
Highlights- An innovative whole slide image signature is developed to facilitate the prognostic risk stratification for stage III colorectal cancer patients.
Abstract Background and ObjectiveAdjuvant chemotherapy is recommended as standard treatment for colorectal cancer (CRC) with stage III according to TNM stage. However, outcomes are varied even among patients receiving similar ...
Multi-modality Fusion Based Lung Cancer Survival Analysis with Self-supervised Whole Slide Image Representation Learning
Pattern Recognition and Computer VisionAbstractWhole slide pathological images (WSIs) are the gold standard for lung cancer prognosis. However, due to their high resolution and limited annotations, lung cancer survival analysis based on WSIs becomes a challenging task. Some recent methods that ...
Comments