Abstract
The paper presents a novel deep learning approach, which extracts latent information from trained Deep Neural Networks (DNNs) and derives concise representations that are analyzed in an effective, transparent way for prediction in medical imaging. A novel methodology is presented, in which deep neural architectures that have been trained to provide highly accurate predictions over existing datasets are adapted, in a consistent way, to make predictions over different contexts and datasets. Unified prediction is then achieved over the original and the new datasets. Successful application is illustrated through a large experimental study for prediction of Parkinson’s disease from MRI and DaTScans, as well as for prediction of COVID-19 from CT scans and X-rays.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aljalbout, E., Golkov, V., Siddiqui, Y., Strobel, M., Cremers, D.: Clustering with deep learning: taxonomy and new methods. arXiv preprint arXiv:1801.07648 (2018)
Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. Technical report, Stanford (2006)
Avrithis, Y., Tsapatsoulis, N., Kollias, S.: Broadcast news parsing using visual cues: a robust face detection approach. In: 2000 IEEE International Conference on Multimedia and Expo, ICME 2000, Proceedings of the Latest Advances in the Fast Changing World of Multimedia (Cat. No. 00TH8532), vol. 3, pp. 1469–1472. IEEE (2000)
Azizi, S., et al.: Detection and grading of prostate cancer using temporal enhanced ultrasound: combining deep neural networks and tissue mimicking simulations. Int. J. Comput. Assist. Radiol. Surg. 12, 1293–1305 (2017). https://doi.org/10.1007/s11548-017-1627-0
Cohen, J.P., Morrison, P., Dao, L., Roth, K., Duong, T.Q., Ghassemi, M.: COVID-19 image data collection: prospective predictions are the future. arXiv preprint arXiv:2006.11988 (2020)
De Sousa Ribeiro, F., Leontidis, G., Kollias, S.: Introducing routing uncertainty in capsule networks. In: Advances in Neural Information Processing Systems, vol. 33 (2020)
Esteva, A., Robicquet, A., et al.: A guide to deep learning in healthcare. Nat. Med. 25, 24–29 (2019)
Glimm, B., Kazakov, Y., Kollia, I., Stamou, G.: Lower and upper bounds for SPARQL queries over OWL ontologies. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
He, X., et al.: Sample-efficient deep learning for COVID-19 diagnosis based on CT scans. medRxiv (2020)
Jiang, S., Kaiser, M., Yang, S., Kollias, S., Krasnogor, N.: A scalable test suite for continuous dynamic multiobjective optimization. IEEE Trans. Cybern. 50(6), 2814–2826 (2019)
Kappeler, A., et al.: Combining deep learning and unsupervised clustering to improve scene recognition performance. In: 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–6. IEEE (2015)
Kollia, I., Simou, N., Stafylopatis, A., Kollias, S.: Semantic image analysis using a symbolic neural architecture. Image Anal. Stereol. 29(3), 159–172 (2010)
Kollia, I., Simou, N., Stamou, G., Stafylopatis, A.: Interweaving knowledge representation and adaptive neural networks (2009)
Kollia, I., Stafylopatis, A.G., Kollias, S.: Predicting Parkinson’s disease using latent information extracted from deep neural networks. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2019)
Kollias, D., et al.: Deep transparent prediction through latent representation analysis. arXiv preprint arXiv:2009.07044 (2020)
Kollias, D., Cheng, S., Pantic, M., Zafeiriou, S.: Photorealistic facial synthesis in the dimensional affect space. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11130, pp. 475–491. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11012-3_36
Kollias, D., Sharmanska, V., Zafeiriou, S.: Face behavior\(\backslash \)a la carte: expressions, affect and action units in a single network. arXiv preprint arXiv:1910.11111 (2019)
Kollias, D., Tagaris, A., Stafylopatis, A., Kollias, S., Tagaris, G.: Deep neural architectures for prediction in healthcare. Complex Intell. Syst. 4(2), 119–131 (2017). https://doi.org/10.1007/s40747-017-0064-6
Kollias, D., Yu, M., Tagaris, A., Leontidis, G., Stafylopatis, A., Kollias, S.: Adaptation and contextualization of deep neural network models. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8. IEEE (2017)
Kollias, D., Zafeiriou, S.P.: Exploiting multi-CNN features in CNN-RNN based dimensional emotion recognition on the OMG in-the-wild dataset. IEEE Trans. Affect. Comput., 1 (2020). https://doi.org/10.1109/TAFFC.2020.3014171
Kollias, D., Marandianos, G., Raouzaiou, A., Stafylopatis, A.G.: Interweaving deep learning and semantic techniques for emotion analysis in human-machine interaction. In: 2015 10th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP), pp. 1–6. IEEE (2015)
Marek, K., et al.: The Parkinson progression marker initiative (PPMI). Prog. Neurobiol. 95, 629–635 (2011)
Mei, W., Deng, W.: Deep visual domain adaptation: a survey. Neurocomputing 312, 135–153 (2018)
Min, E., Guo, X., Liu, Q., Zhang, G., Cui, J., Long, J.: A survey of clustering with deep learning: from the perspective of network architecture. IEEE Access 6, 39501–39514 (2018)
Parisi, G.I., Kemker, R., Part, J.L., Kanan, C., Wermter, S.: Continual lifelong learning with neural networks: a review. Neural Netw. 113, 54–71 (2019)
Rapantzikos, K., Tsapatsoulis, N., Avrithis, Y., Kollias, S.: Bottom-up spatiotemporal visual attention model for video analysis. IET Image Process. 1(2), 237–248 (2007)
De Sousa Ribeiro, F., Calivá, F., Swainson, M., Gudmundsson, K., Leontidis, G., Kollias, S.: Deep Bayesian self-training. Neural Comput. Appl. 32(9), 4275–4291 (2019). https://doi.org/10.1007/s00521-019-04332-4
Ribeiro, F.D.S., Leontidis, G., Kollias, S.D.: Capsule routing via variational bayes. In: AAAI, pp. 3749–3756 (2020)
Tagaris, A., Kollias, D., Stafylopatis, A.: Assessment of Parkinson’s disease based on deep neural networks. In: Boracchi, G., Iliadis, L., Jayne, C., Likas, A. (eds.) EANN 2017. CCIS, vol. 744, pp. 391–403. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65172-9_33
Tagaris, A., Kollias, D., Stafylopatis, A., Tagaris, G., Kollias, S.: Machine learning for neurodegenerative disorder diagnosis-survey of practices and launch of benchmark dataset. Int. J. Artif. Intell. Tools 27(03), 1850011 (2018)
Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., Liu, C.: A survey on deep transfer learning. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11141, pp. 270–279. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01424-7_27
Wallace, M., Maglogiannis, I., Karpouzis, K., Kormentzas, G., Kollias, S.: Intelligent one-stop-shop travel recommendations using an adaptive neural network and clustering of history. Inf. Technol. Tour. 6(3), 181–193 (2003)
Wingate, J., Kollia, I., Bidaut, L., Kollias, S.: A unified deep learning approach for prediction of Parkinson’s disease. arXiv preprint arXiv:1911.10653 (2019)
Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, pp. 478–487 (2016)
Yang, B., Fu, X., Sidiropoulos, N.D., Hong, M.: Towards k-means-friendly spaces: simultaneous deep learning and clustering. In: International Conference on Machine Learning, pp. 3861–3870 (2017)
Yu, M., Kollias, D., Wingate, J., Siriwardena, N., Kollias, S.: Machine learning for predictive modelling of ambulance calls (2021)
Acknowledgment
We thank the Department of Neurology of the Georgios Gennimatas General Hospital, Athens, Greece, for providing the dataset with Parkinson’s data. The PPMI data used in the preparation of this article were obtained from the Parkinson’s Progression Markers Initiative (PPMI) database (www.ppmi-info.org/data) and we thank them for this.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kollias, D. et al. (2021). Transparent Adaptation in Deep Medical Image Diagnosis. In: Heintz, F., Milano, M., O'Sullivan, B. (eds) Trustworthy AI - Integrating Learning, Optimization and Reasoning. TAILOR 2020. Lecture Notes in Computer Science(), vol 12641. Springer, Cham. https://doi.org/10.1007/978-3-030-73959-1_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-73959-1_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73958-4
Online ISBN: 978-3-030-73959-1
eBook Packages: Computer ScienceComputer Science (R0)