Skip to main content
SpringerLink
Log in

Representation Learning with Information Theory to Detect COVID-19 and Its Severity

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13807))

Included in the following conference series:

Abstract

Successful data representation is a fundamental factor in machine learning based medical imaging analysis. Deep Learning (DL) has taken an essential role in robust representation learning. However, the inability of deep models to generalize to unseen data can quickly overfit intricate patterns. Thereby, the importance of implementing strategies to aid deep models in discovering useful priors from data to learn their intrinsic properties. Our model, which we call a dual role network (DRN), uses a dependency maximization approach based on Least Squared Mutual Information (LSMI). LSMI leverages dependency measures to ensure representation invariance and local smoothness. While prior works have used information theory dependency measures like mutual information, these are known to be computationally expensive due to the density estimation step. In contrast, our proposed DRN with LSMI formulation does not require the density estimation step and can be used as an alternative to approximate mutual information. Experiments on the CT based COVID-19 Detection and COVID-19 Severity Detection Challenges of the 2nd COV19D competition [24] demonstrate the effectiveness of our method compared to the baseline method of such competition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://mlearn.lincoln.ac.uk/eccv-2022-ai-mia/.

  2. 2.

    For the complete derivation of the LSMI estimator, readers are referred to [60].

  3. 3.

    https://vcmi.inesctec.pt/aimia_eccv/.

  4. 4.

    https://stoic2021.grand-challenge.org/.

  5. 5.

    https://pytorch.org/.

  6. 6.

    https://optuna.readthedocs.io/en/stable/index.html.

  7. 7.

    https://monai.io/.

References

  1. Abrahamyan, L., Ziatchin, V., Chen, Y., Deligiannis, N.: Bias loss for mobile neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6556–6566 (2021)

    Google Scholar 

  2. Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2623–2631 (2019)

    Google Scholar 

  3. Ali, S.M., Silvey, S.D.: A general class of coefficients of divergence of one distribution from another. J. Roy. Stat. Soc. Ser. B (Methodol.) 28(1), 131–142 (1966)

    Google Scholar 

  4. Anwar, T.: Covid19 diagnosis using autoML from 3D CT scans. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp. 503–507, October 2021

    Google Scholar 

  5. Arsenos, A., Kollias, D., Kollias, S.: A large imaging database and novel deep neural architecture for Covid-19 diagnosis. In: 2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP), pp. 1–5. IEEE (2022)

    Google Scholar 

  6. Belghazi, M.I., et al.: Mutual information neural estimation. In: International Conference on Machine Learning, pp. 531–540. PMLR (2018)

    Google Scholar 

  7. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)

    Article  Google Scholar 

  8. Bergstra, J., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: Advances in Neural Information Processing Systems, vol. 24 (2011)

    Google Scholar 

  9. Bergstra, J., Yamins, D., Cox, D.: Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In: International Conference on Machine Learning, pp. 115–123. PMLR (2013)

    Google Scholar 

  10. Bernheim, A., et al.: Chest CT findings in coronavirus disease-19 (Covid-19): relationship to duration of infection. Radiology 295(3), 200463 (2020)

    Google Scholar 

  11. Bortsova, G., Dubost, F., Hogeweg, L., Katramados, I., de Bruijne, M.: Semi-supervised medical image segmentation via learning consistency under transformations. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 810–818. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_90

    Chapter  Google Scholar 

  12. Chen, X., et al.: Anatomy-regularized representation learning for cross-modality medical image segmentation. IEEE Trans. Med. Imaging 40(1), 274–285 (2020)

    Article  MathSciNet  Google Scholar 

  13. Cui, W., et al.: Semi-supervised brain lesion segmentation with an adapted mean teacher model. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 554–565. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_43

    Chapter  Google Scholar 

  14. Cutillo, C.M., Sharma, K.R., Foschini, L., Kundu, S., Mackintosh, M., Mandl, K.D.: Machine intelligence in healthcare-perspectives on trustworthiness, explainability, usability, and transparency. NPJ Digit. Med. 3(1), 1–5 (2020)

    Article  Google Scholar 

  15. Dong, D., et al.: The role of imaging in the detection and management of Covid-19: a review. IEEE Rev. Biomed. Eng. 14, 16–29 (2021)

    Article  Google Scholar 

  16. Goncharov, M., et al.: CT-based Covid-19 triage: deep multitask learning improves joint identification and severity quantification. Med. Image Anal. 71, 102054 (2021)

    Article  Google Scholar 

  17. Greenspan, H., San José Estépar, R., Niessen, W.J., Siegel, E., Nielsen, M.: Position paper on COVID-19 imaging and AI: from the clinical needs and technological challenges to initial AI solutions at the lab and national level towards a new era for AI in healthcare. Med. Image Anal. 66, 101800 (2020)

    Google Scholar 

  18. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  19. Hjelm, D., et al.: Learning deep representations by mutual information estimation and maximization. In: International Conference on Learning Representations (ICLR), April 2019

    Google Scholar 

  20. Hofmanninger, J., Prayer, F., Pan, J., Röhrich, S., Prosch, H., Langs, G.: Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem. Eur. Radiol. Exp. 4(1), 1–13 (2020). https://doi.org/10.1186/s41747-020-00173-2

    Article  Google Scholar 

  21. Hou, J., Xu, J., Feng, R., Zhang, Y., Shan, F., Shi, W.: CMC-Cov19D: contrastive mixup classification for Covid-19 diagnosis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 454–461 (2021)

    Google Scholar 

  22. Hsu, C.C., Chen, G.L., Wu, M.H.: Visual transformer with statistical test for Covid-19 classification. arXiv preprint arXiv:2107.05334 (2021)

  23. Hu, W., Miyato, T., Tokui, S., Matsumoto, E., Sugiyama, M.: Learning discrete representations via information maximizing self-augmented training. In: International Conference on Machine Learning, pp. 1558–1567. PMLR (2017)

    Google Scholar 

  24. Kollias, D., Arsenos, A., Kollias, S.: AI-MIA: Covid-19 detection & severity analysis through medical imaging. arXiv preprint arXiv:2206.04732 (2022)

  25. Kollias, D., Arsenos, A., Soukissian, L., Kollias, S.: MIA-Cov19d: Covid-19 detection through 3-D chest CT image analysis. arXiv preprint arXiv:2106.07524 (2021)

  26. Kollias, D., Arsenos, A., Soukissian, L., Kollias, S.: MIA-Cov19d: Covid-19 detection through 3-D chest CT image analysis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 537–544 (2021)

    Google Scholar 

  27. Kollias, D., et al.: Deep transparent prediction through latent representation analysis. arXiv preprint arXiv:2009.07044 (2020)

  28. Kollias, D., Tagaris, A., Stafylopatis, A., Kollias, S., Tagaris, G.: Deep neural architectures for prediction in healthcare. Complex Intell. Syst. 4(2), 119–131 (2018)

    Article  Google Scholar 

  29. Kollias, D., et al.: Transparent adaptation in deep medical image diagnosis. In: TAILOR, p. 251–267 (2020)

    Google Scholar 

  30. Li, X., Yu, L., Chen, H., Fu, C.W., Xing, L., Heng, P.A.: Transformation-consistent self-ensembling model for semisupervised medical image segmentation. IEEE Trans. Neural Netw. Learn. Syst. 32(2), 523–534 (2021)

    Article  Google Scholar 

  31. Lin, T., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell. 42(2), 318–327 (2020)

    Article  Google Scholar 

  32. Liu, Q., Yu, L., Luo, L., Dou, Q., Heng, P.A.: Semi-supervised medical image classification with relation-driven self-ensembling model. IEEE Trans. Med. Imaging 39(11), 3429–3440 (2020)

    Article  Google Scholar 

  33. Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)

    Google Scholar 

  34. Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976–11986 (2022)

    Google Scholar 

  35. Miron, R., Moisii, C., Dinu, S., Breaban, M.E.: Evaluating volumetric and slice-based approaches for Covid-19 detection in chest CTs. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 529–536 (2021)

    Google Scholar 

  36. Mukhoti, J., Kulharia, V., Sanyal, A., Golodetz, S., Torr, P., Dokania, P.: Calibrating deep neural networks using focal loss. Adv. Neural. Inf. Process. Syst. 33, 15288–15299 (2020)

    Google Scholar 

  37. Ning, Z., Tu, C., Di, X., Feng, Q., Zhang, Y.: Deep cross-view co-regularized representation learning for glioma subtype identification. Med. Image Anal. 73, 102160 (2021)

    Article  Google Scholar 

  38. Peng, J., Pedersoli, M., Desrosiers, C.: Boosting semi-supervised image segmentation with global and local mutual information regularization. CoRR abs/2103.04813 (2021)

    Google Scholar 

  39. Polyak, B.T., Juditsky, A.B.: Acceleration of stochastic approximation by averaging. SIAM J. Control. Optim. 30(4), 838–855 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  40. Prokop, M., Van Everdingen, W., van Rees, V., et al.: CoRads: a categorical CT assessment scheme for patients suspected of having Covid-19-definition and evaluation. Radiology 296, E97–E104 (2020)

    Article  Google Scholar 

  41. Ranschaert, E.R., Morozov, S., Algra, P.R.: Artificial Intelligence in Medical Imaging: Opportunities. Applications and Risks. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-94878-2

    Book  Google Scholar 

  42. Rubenstein, P., Bousquet, O., Djolonga, J., Riquelme, C., Tolstikhin, I.O.: Practical and consistent estimation of f-divergences. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc. (2019)

    Google Scholar 

  43. Sangalli, S., Erdil, E., Hötker, A., Donati, O., Konukoglu, E.: Constrained optimization to train neural networks on critical and under-represented classes. Adv. Neural. Inf. Process. Syst. 34, 25400–25411 (2021)

    Google Scholar 

  44. Seo, H., Yu, L., Ren, H., Li, X., Shen, L., Xing, L.: Deep neural network with consistency regularization of multi-output channels for improved tumor detection and delineation. IEEE Trans. Med. Imaging 40(12), 3369–3378 (2021)

    Article  Google Scholar 

  45. Shi, F., et al.: Large-scale screening to distinguish between Covid-19 and community-acquired pneumonia using infection size-aware classification. Phys. Med. Biol. 66(6), 065031 (2021)

    Article  Google Scholar 

  46. Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Routledge, Abingdon (2018)

    Google Scholar 

  47. Sinha, S., Dieng, A.B.: Consistency regularization for variational auto-encoders. Adv. Neural. Inf. Process. Syst. 34, 12943–12954 (2021)

    Google Scholar 

  48. Su, H., Shi, X., Cai, J., Yang, L.: Local and global consistency regularized mean teacher for semi-supervised nuclei classification. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 559–567. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_62

    Chapter  Google Scholar 

  49. Sugiyama, M.: Machine learning with squared-loss mutual information. Entropy 15(1), 80–112 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  50. Sugiyama, M., Suzuki, T., Kanamori, T.: Density Ratio Estimation in Machine Learning, 1st edn. Cambridge University Press, New York (2012)

    Book  MATH  Google Scholar 

  51. Suzuki, T., Sugiyama, M.: Sufficient dimension reduction via squared-loss mutual information estimation. In: AISTATS (2010)

    Google Scholar 

  52. Tan, W., Liu, J.: A 3D CNN network with BERT for automatic Covid-19 diagnosis from CT-scan images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 439–445 (2021)

    Google Scholar 

  53. Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 1195–1204 (2017)

    Google Scholar 

  54. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1999). https://doi.org/10.1007/978-1-4757-3264-1

    Book  MATH  Google Scholar 

  55. Wang, B., et al.: AI-assisted CT imaging analysis for Covid-19 screening: building and deploying a medical AI system. Appl. Soft Comput. 98, 106897 (2021)

    Article  Google Scholar 

  56. Wang, X., et al.: A weakly-supervised framework for Covid-19 classification and lesion localization from chest CT. IEEE Trans. Med. Imaging 39(8), 2615–2625 (2020)

    Article  Google Scholar 

  57. Wightman, R.: Pytorch image models (2019). https://github.com/rwightman/pytorch-image-models, https://doi.org/10.5281/zenodo.4414861

  58. Wu, Y.H., et al.: JCS: an explainable Covid-19 diagnosis system by joint classification and segmentation. IEEE Trans. Image Process. 30, 3113–3126 (2021)

    Article  Google Scholar 

  59. Xie, X., Zhong, Z., Zhao, W., Zheng, C., Wang, F., Liu, J.: Chest CT for typical coronavirus disease 2019 (Covid-19) pneumonia: relationship to negative RT-PCR testing. Radiology 296(2), E41–E45 (2020)

    Google Scholar 

  60. Yamada, M., Sigal, L., Raptis, M., Toyoda, M., Chang, Y., Sugiyama, M.: Cross-domain matching with squared-loss mutual information. IEEE TPAMI 37(9), 1764–1776 (2015)

    Article  Google Scholar 

  61. Yamada, M., Sugiyama, M.: Cross-domain object matching with model selection. In: AISTATS (2011)

    Google Scholar 

  62. Zhang, L., Wen, Y.: A transformer-based framework for automatic Covid19 diagnosis in chest CTs. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 513–518 (2021)

    Google Scholar 

Download references

Acknowledgement

We want to thank the organizers of the 2nd COV19D Competition occurring in the ECCV 2022 Workshop: AI-enabled Medical Image Analysis - Digital Pathology & Radiology/COVID19 for providing access to extensive and high-quality data to benchmark our model. This research has been partially financed by the European Union under the Horizon 2020 Research and Innovation programme under grant agreement 101016131 (ICOVID).

Author information

Authors and Affiliations

  1. Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel, Pleinlaan 2, 1050, Brussels, Belgium

    Abel Díaz Berenguer, Tanmoy Mukherjee, Yifei Da, Matías Nicolás Bossa, Maryna Kvasnytsia, Jef Vandemeulebroucke, Nikos Deligiannis & Hichem Sahli

  2. Interuniversity Microelectronics Centre (IMEC), Kapeldreef 75, 3001, Heverlee, Belgium

    Tanmoy Mukherjee, Jef Vandemeulebroucke, Nikos Deligiannis & Hichem Sahli

Authors
  1. Abel Díaz Berenguer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tanmoy Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yifei Da
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matías Nicolás Bossa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Maryna Kvasnytsia
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jef Vandemeulebroucke
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nikos Deligiannis
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Hichem Sahli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abel Díaz Berenguer .

Editor information

Editors and Affiliations

  1. IBM Research - MIT-IBM Watson AI Lab, Massachusetts, USA

    Leonid Karlinsky

  2. Technion – Israel Institute of Technology, Haifa, Israel

    Tomer Michaeli

  3. Kyoto University, Kyoto, Japan

    Ko Nishino

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Berenguer, A.D. et al. (2023). Representation Learning with Information Theory to Detect COVID-19 and Its Severity. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13807. Springer, Cham. https://doi.org/10.1007/978-3-031-25082-8_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25082-8_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25081-1

  • Online ISBN: 978-3-031-25082-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation