Skip to main content
SpringerLink
Log in

Exploring Optimal Configurations in Active Learning for Medical Imaging

  • Conference paper
  • First Online:
Artificial Intelligence XL (SGAI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14381))

  • 478 Accesses

Abstract

Medical imaging is a critical component of clinical decision-making, patient diagnosis, treatment planning, intervention, and therapy. However, due to the shortage of qualified radiologists, there is an increasing burden on healthcare practitioners, which underscores the need to develop reliable automated methods. Despite the development of novel computational techniques, interpreting medical images remains challenging due to noise and varying acquisition conditions. One promising solution to improve the reliability and accuracy of automated medical image analysis is Interactive Machine Learning (IML), which integrates human expertise into the model training process. Active learning (AL) is an important IML technique that can iteratively query for informative samples to be labeled by humans, leading to more data-efficient learning. To fully leverage the potential of active learning, however, it is crucial to understand the optimal setup for different components of an AL system. This paper presents an evaluation of the effectiveness of different combinations of data representation, model capacity, and query strategy for active learning systems designed for medical image classification tasks. The results of this evaluation show that employing raw image representations as input, in conjunction with a ResNet50 model and margin-based queries, yields more reliable and accurate automated methods for medical image analysis.

This publication has emanated from research conducted with the financial support of Science Foundation Ireland under Grant number 18/CRT/6183. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    ResNet50 model pre-trained on the Imagenet dataset. Available at: https://pytorch.org/vision/main/models/generated/torchvision.models.resnet50.html.

References

  1. Aghdam, H.H., et al.: Active learning for deep detection neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3672–3680 (2019)

    Google Scholar 

  2. Amershi, S., et al.: Power to the people: the role of humans in interactive machine learning. AI Mag. 350(4), 105–120 (2014)

    Google Scholar 

  3. Angluin, D.: Queries and concept learning. Mach. Learn. 2, 319–342 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  4. Arrieta, A.B., et al.: Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58, 82–115 (2020)

    Article  Google Scholar 

  5. Atlas, L., Cohn, D., Ladner, R.: Training connectionist networks with queries and selective sampling. In: Advances in Neural Information Processing Systems, vol. 2 (1989)

    Google Scholar 

  6. Baum, E.B.: Neural net algorithms that learn in polynomial time from examples and queries. IEEE Trans. Neural Netw. 20(1), 5–19 (1991)

    Article  MathSciNet  Google Scholar 

  7. Belkin, M., et al.: Reconciling modern machine-learning practice and the classical bias-variance trade-off. In: Proceedings of the National Academy of Sciences, vol. 1160. no. (32), pp. 15849–15854 (2019)

    Google Scholar 

  8. Berg, S., et al.: Ilastik: interactive machine learning for (bio) image analysis. Nat. Methods 160(12), 1226–1232 (2019)

    Article  Google Scholar 

  9. Branson, S., Perona, P., Belongie, S.: Strong supervision from weak annotation: Interactive training of deformable part models. In: 2011 International Conference on Computer Vision, pp. 1832–1839. IEEE (2011)

    Google Scholar 

  10. Budd, S., Robinson, E.C., Kainz, B.: A survey on active learning and human-in-the-loop deep learning for medical image analysis. Med. Image Anal. 71, 102062 (2021)

    Article  Google Scholar 

  11. Cho, J.W., et al.: MCDAL: maximum classifier discrepancy for active learning. IEEE Trans. Neural Netw. Learn. Syst. (2022)

    Google Scholar 

  12. Chung, M.-H., et al.: Interactive machine learning for data exfiltration detection: Active learning with human expertise. In: 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 280–287. IEEE (2020)

    Google Scholar 

  13. Cohn, D., Atlas, L., Ladner, R.: Improving generalization with active learning. Mach. Learn. 15, 201–221 (1994)

    Article  Google Scholar 

  14. Dagan, I., Engelson, S.P.: Committee-based sampling for training probabilistic classifiers. In: Machine Learning Proceedings 1995, pp. 150–157. Elsevier (1995)

    Google Scholar 

  15. Dasgupta, S., Kalai, A.T., Tauman, A.: Analysis of perceptron-based active learning. J. Mach. Learn. Res. 100(2) 2009

    Google Scholar 

  16. Du, X., Zhong, D., Shao, H.: Building an active palmprint recognition system. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1685–1689. IEEE (2019)

    Google Scholar 

  17. Dudley, J.J., Kristensson, P.O.: A review of user interface design for interactive machine learning. ACM Trans. Interact. Intell. Syst. (TiiS) 80(2), 1–37 (2018)

    Google Scholar 

  18. Fails, J.A., Olsen Jr, D.R.: Interactive machine learning. In: Proceedings of the 8th International Conference on Intelligent User Interfaces, pp. 39–45 (2003)

    Google Scholar 

  19. Goyal, P., et al.: Accurate, large minibatch SGD: training imageNet in 1 hour. arXiv preprint arXiv:1706.02677 (2017)

  20. He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  21. Ho, Y., Wookey, S.: The real-world-weight cross-entropy loss function: modeling the costs of mislabeling. IEEE Access 8, 4806–4813 (2019)

    Article  Google Scholar 

  22. Holzinger, A.: Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Inform. 30(2), 119–131 (2016)

    Article  Google Scholar 

  23. Holzinger, A., Plass, M., Holzinger, K., Crişan, G.C., Pintea, C.-M., Palade, V.: Towards interactive machine learning (iML): applying ant colony algorithms to solve the traveling salesman problem with the human-in-the-loop approach. In: Buccafurri, F., Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-ARES 2016. LNCS, vol. 9817, pp. 81–95. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-45507-5_6

    Chapter  MATH  Google Scholar 

  24. Hu, R., Namee, B.M., Delany, S.J.: Sweetening the dataset: using active learning to label unlabelled datasets. In: Proceedings of AICS, vol. 8, pp. 53–62 (2008)

    Google Scholar 

  25. Jiang, L., Liu, S., Chen, C., Recent research advances on interactive machine learning: Recent research advances on interactive machine learning. J. Vis. 22, 401–417 (2019)

    Article  Google Scholar 

  26. Kelly, D., et al.: Methods for evaluating interactive information retrieval systems with users. Found. Trends® Inf. Retrieval, 30(1–2), 1–224 (2009)

    Google Scholar 

  27. Kim, M., et al.: Topiclens: efficient multi-level visual topic exploration of large-scale document collections. IEEE Trans. Vis. Comput. Graph. 230(1), 151–160 (2016)

    Article  Google Scholar 

  28. King, R.D., et al.: The automation of science. Science 3240(5923), 85–89 (2009)

    Article  Google Scholar 

  29. Kose, I., Gokturk, M., Kilic, K.: An interactive machine-learning-based electronic fraud and abuse detection system in healthcare insurance. Appl. Soft Comput. 36, 283–299 (2015)

    Article  Google Scholar 

  30. Lewis, D.D., Catlett, J.: Heterogeneous uncertainty sampling for supervised learning. In: Machine Learning Proceedings 1994, pp. 148–156. Elsevier (1994)

    Google Scholar 

  31. Lin, X., Parikh, D.: Active learning for visual question answering: an empirical study. arXiv preprint arXiv:1711.01732 (2017)

  32. Liu, Z., et al.: Patterns and sequences: Interactive exploration of clickstreams to understand common visitor paths. IEEE Trans. Vis. Comput. Graph. 230(1), 321–330 (2016)

    Article  Google Scholar 

  33. McCallum, A., Nigam, K., et al.: Employing EM and pool-based active learning for text classification. In: ICML, vol. 98, pp. 350–358. Citeseer (1998)

    Google Scholar 

  34. Rawat, S., et al.: How useful is image-based active learning for plant organ segmentation? Plant Phenomics, 2022 (2022)

    Google Scholar 

  35. Sabato, S., Hess, T.: Interactive algorithms: pool, stream and precognitive stream. J. Mach. Learn. Res. 18, 1–39 (2017)

    MathSciNet  MATH  Google Scholar 

  36. Scheffer, T., Decomain, C., Wrobel, S.: Active hidden Markov models for information extraction. In: Hoffmann, F., Hand, D.J., Adams, N., Fisher, D., Guimaraes, G. (eds.) IDA 2001. LNCS, vol. 2189, pp. 309–318. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44816-0_31

    Chapter  Google Scholar 

  37. Schumann, R., Rehbein, I.: Active learning via membership query synthesis for semi-supervised sentence classification. In: Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pp. 472–481 (2019)

    Google Scholar 

  38. Settles, B.: Active learning literature survey (2009)

    Google Scholar 

  39. Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 270(3), 379–423 (1948)

    Article  MathSciNet  MATH  Google Scholar 

  40. Smith, S.L., et al.: Don’t decay the learning rate, increase the batch size. arXiv preprint arXiv:1711.00489 (2017)

  41. Teso, S., Kersting, K.: Explanatory interactive machine learning. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 239–245 (2019)

    Google Scholar 

  42. The Royal College of Radiologists. Rcr clinical radiology census report 2021 (2021). https://www.rcr.ac.uk/clinical-radiology/rcr-clinical-radiology-census-report-2021 . Accessed 27 Feb 2023

  43. Wang, Y., et al.: Efficient DNN training with knowledge-guided layer freezing. arXiv preprint arXiv:2201.06227 (2022)

  44. Yang, J., et al.: MedMNIST v2-a large-scale lightweight benchmark for 2D and 3D biomedical image classification. Sci. Data 100(1), 41 (2023)

    Article  Google Scholar 

  45. Betül Yüce, A., Yaslan, Y.: A disagreement based co-active learning method for sleep stage classification. In: 2016 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 1–4. IEEE (2016)

    Google Scholar 

  46. Zhan, X., et al.: A comparative survey of deep active learning. arXiv preprint arXiv:2203.13450 (2022)

  47. Zhang, Y., Lease, M., Wallace, B.: Active discriminative text representation learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, University College Dublin, Dublin, Ireland

    Alec Parise & Brian Mac Namee

Authors
  1. Alec Parise
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brian Mac Namee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alec Parise .

Editor information

Editors and Affiliations

  1. University of Portsmouth, Portsmouth, UK

    Max Bramer

  2. DFKI: German Research Center for Artificial Intelligence, Oldenburg, Germany

    Frederic Stahl

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Parise, A., Mac Namee, B. (2023). Exploring Optimal Configurations in Active Learning for Medical Imaging. In: Bramer, M., Stahl, F. (eds) Artificial Intelligence XL. SGAI 2023. Lecture Notes in Computer Science(), vol 14381. Springer, Cham. https://doi.org/10.1007/978-3-031-47994-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47994-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47993-9

  • Online ISBN: 978-3-031-47994-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation