Abstract
Whole Slide Image (WSI) classification is an important part of pathological diagnosis. Although previous approaches (such as DSMIL and CLAM) have achieved good results, the classification performance is still unsatisfactory because the learned features of WSI lack discrimination and the correlation among sub-characteristics of tumor images are ignored. In this paper, we proposed a Metric Learning Constraint Network (referred to as MLCN). Particularly, MLCN benefits from two aspects: 1) It enhances the discriminative power of features by enlarging inter-class distance and narrowing intra-class distance in both slide-level and patch-level. 2) It learns a more powerful feature aggregator by proposing the bilinear gated attention mechanism to capture relations among sub-characteristics of tumor issues. Experiments on CAMELYON16 and TCGA Kidney datasets validate the effectiveness of our approach, and we achieved state-of-the-art performance compared to other popular methods. The codes will be available soon.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning, pp. 2127–2136 (2018)
Lu, M.Y., Williamson, D.F., Chen, T.Y., et al.: Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5(6), 555–570 (2021)
Bejnordi, B.E., Veta, M., Van Diest, P.J., Van Ginneken, B., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017)
Campanella, G., Hanna, M.G., Geneslaw, L., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25(8), 1301–1309 (2019)
Li, B., Li, Y., Eliceiri, K.W.: Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318–14328 (2021)
Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1449–1457 (2015)
Carreira, J., Caseiro, R., Batista, J., Sminchisescu, C.: Semantic segmentation with second-order pooling. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7578, pp. 430–443. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33786-4_32
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)
Wen, Y., Zhang, K., Li, Z., Qiao, Yu.: A discriminative feature learning approach for deep face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 499–515. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_31
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y
Cornish, T.C., Swapp, R.E., Kaplan, K.J.: Whole-slide imaging: routine pathologic diagnosis. Adv. Anat. Pathol. 19(3), 152–159 (2012)
Pantanowitz, L., Valenstein, P.N., Evans, A.J., et al.: Review of the current state of whole slide imaging in pathology. J. Pathol. Inform. 2(1), 36 (2011)
Feng, J., Zhou, Z.H.: Deep MIML network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, No. 1 (2017)
Pinheiro, P.O., Collobert, R.: From image-level to pixel-level labeling with convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1713–1721 (2015)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Wang, D., Khosla, A., Gargeya, R., et al.: Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016)
Hashimoto, N., Fukushima, D., Koga, R., et al.: Multi-scale domain-adversarial multiple-instance CNN for cancer subtype classification with unannotated histopathological images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3852–3861 (2020)
Komura, D., Ishikawa, S.: Machine learning methods for histopathological image analysis. Comput. Struct. Biotechnol. J. 16, 34–42 (2018)
Kaya, M., Bilge, H.Ş: Deep metric learning: a survey. Symmetry 11(9), 1066 (2019)
Cheplygina, V., Tax, D.M., Loog, M.: Multiple instance learning with bag dissimilarities. Pattern Recogn. 48(1), 264–275 (2015)
Dauphin, Y.N., Fan, A., Auli, M., et al.: Language modeling with gated convolutional networks. In: International Conference on Machine Learning, pp. 933–941 (2017)
Wang, X., Yan, Y., Tang, P., et al.: Revisiting multiple instance neural networks. Pattern Recogn. 74, 15–24 (2018)
Kraus, O.Z., Ba, J.L., Frey, B.J.: Classifying and segmenting microscopy images with deep multiple instance learning. Bioinformatics 32(12), i52–i59 (2016)
Atlas, T.C.G. Website (2006). https://portal.gdc.cancer.gov/
Jun, W., Jean-Daniel, Z.: Solving the multiple-instance problem: a lazy learning approach. In: Proceedings of the 17th International Conference on Machine Learning, pp. 1119–1125 (2000)
Zhang, Q., Goldman, S.: EM-DD: an improved multiple-instance learning technique. Adv. Neural Inf. Process. Syst. 14 (2001)
Gärtner, T., Flach, P.A., Kowalczyk, A., et al.: Multi-instance kernels. In: ICML, vol. 2, p. 7 (2002)
Ilse, M., Tomczak, J.M., Welling, M.: Deep multiple instance learning for digital histopathology. In: Handbook of Medical Image Computing and Computer Assisted Intervention, pp. 521–546 (2020)
Han, Z., Wei, B., Hong, Y., et al.: Accurate screening of COVID-19 using attention-based deep 3D multiple instance learning. IEEE Trans. Med. Imaging 39(8), 2584–2594 (2020)
Yan, R., Ren, F., Wang, Z., et al.: Breast cancer histopathological image classification using a hybrid deep neural network. Methods 173(1), 52–60 (2020)
Acknowledgements
The research is supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (No. XDA16021400), and the NSFC projects grants (61932018, 62072441 and 62072280).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shi, B., Liu, X., Zhang, F. (2022). MLCN: Metric Learning Constrained Network for Whole Slide Image Classification with Bilinear Gated Attention Mechanism. In: Qin, W., Zaki, N., Zhang, F., Wu, J., Yang, F. (eds) Computational Mathematics Modeling in Cancer Analysis. CMMCA 2022. Lecture Notes in Computer Science, vol 13574. Springer, Cham. https://doi.org/10.1007/978-3-031-17266-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-17266-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17265-6
Online ISBN: 978-3-031-17266-3
eBook Packages: Computer ScienceComputer Science (R0)
