Abstract
Shortage of annotated data is one of the greatest bottlenecks in biomedical image analysis. Meta learning studies how learning systems can increase in efficiency through experience and could thus evolve as an important concept to overcome data sparsity. However, the core capability of meta learning-based approaches is the identification of similar previous tasks given a new task - a challenge largely unexplored in the biomedical imaging domain. In this paper, we address the problem of quantifying task similarity with a concept that we refer to as task fingerprinting. The concept involves converting a given task, represented by imaging data and corresponding labels, to a fixed-length vector representation. In fingerprint space, different tasks can be directly compared irrespective of their data set sizes, types of labels or specific resolutions. An initial feasibility study in the field of surgical data science (SDS) with 26 classification tasks from various medical and non-medical domains suggests that task fingerprinting could be leveraged for both (1) selecting appropriate data sets for pretraining and (2) selecting appropriate architectures for a new task. Task fingerprinting could thus become an important tool for meta learning in SDS and other fields of biomedical image analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We use the SciPy [42] implementation of the Wasserstein distance.
- 2.
We used the implementations from [43] and the models CSPNet (cspdarknet53, cspresnext50), ECA-Net (ecaresnet50d), EfficientNet (tf_efficientnet_b2_ns), DPN (dpn68b), MixNet (mixnet_xl), RegNetY (regnety_032), ResNeXt (swsl_resnext50_32x4d), ReXNet (rexnet_200), VovNet2 (ese_vovnet39b) and Xception (xception). Please refer to the documentation of [43] for full references of these.
References
Achille, A., et al.: Task2Vec: Task Embedding for Meta-Learning. ArXiv (2019)
Achille, A., et al.: The Information Complexity of Learning Tasks, their Structure and their Distance. ArXiv (2019)
Ali, S., et al.: Endoscopy artifact detection (EAD 2019) challenge dataset. ArXiv (2019)
Allan, M., et al.: 2017 robotic instrument segmentation challenge. ArXiv (2019)
Bhattacharjee, B., et al.: P2L: predicting transfer learning for images and semantic relations. In: CVPR Workshops (2020)
Borgli, H., et al.: Hyperkvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci. Data 7(1), 1–14 (2020)
Cheplygina, V.: Cats or cat scans: transfer learning from natural or medical image source datasets? ArXiv (2018)
Cheplygina, V., et al.: Exploring the similarity of medical imaging classification problems. In: CVII-STENT/LABELS@MICCAI (2017)
Cheplygina, V., et al.: Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med. Image Anal. 54, 280–296 (2019)
Cohen, G., et al.: EMNIST: an extension of MNIST to handwritten letters. ArXiv (2017)
Combalia, M., et al.: BCN20000: dermoscopic lesions in the wild. ArXiv (2019)
Dabak, A., et al.: Relations between kullback-leibler distance and fisher information (2002)
Deng, J., et al.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition (2009)
Faria, S.M., et al.: Light field image dataset of skin lesions. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2019)
Fei-Fei, L., et al.: Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In: CVPR Workshops (2004)
Gretton, A., et al.: A kernel two-sample test. J. Mach. Learn. Res. 13(1), 723–773 (2012)
Griffin, G., et al.: Caltech-256 object category dataset (2007)
Gutman, D., et al.: Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) (2018)
He, K., et al.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Hospedales, T.M., et al.: Meta-learning in neural networks: a survey. ArXiv (2020)
Kamada, T., et al.: An algorithm for drawing general undirected graphs. Inf. Process. Lett. 31(1), 7–15 (1989)
Kawahara, J., et al.: Seven-point checklist and skin lesion classification using multitask multimodal neural nets. IEEE J. Biomed. Health Inform. 23(2), 538–546 (2019)
Khosla, A., et al.: Novel dataset for fine-grained image categorization. In: First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, CO (2011)
Kingma, D.P., et al.: Adam: a method for stochastic optimization. CoRR (2015)
Krizhevsky, A.: Learning multiple layers of features from tiny images (2009)
LeCun, Y., et al.: Gradient-based learning applied to document recognition (1998)
Leibetseder, A., et al.: LAPGYN4: a dataset for 4 automatic content analysis problems in the domain of laparoscopic gynecology. In: Proceedings of the 9th ACM Multimedia Systems Conference (2018)
Leibetseder, A., et al.: Glenda: gynecologic laparoscopy endometriosis dataset. In: MMM (2020)
Maier-Hein, L., et al.: Can masses of non-experts train highly accurate image classifiers? - a crowdsourcing approach to instrument segmentation in laparoscopic images. In: MICCAI International Conference on Medical Image Computing and Computer-Assisted Intervention (2014)
Maier-Hein, L., et al.: Surgical Data Science – from Concepts to Clinical Translation. ArXiv, October 2020
Moccia, S., et al.: Confident texture-based laryngeal tissue classification for early stage diagnosis support. J. Med. Imaging 4(3), 034502 (2017)
Moccia, S., et al.: Learning-based classification of informative laryngoscopic frames. Comput. Methods Programs Biomed. 158, 21–30 (2018)
Netzer, Y., et al.: Reading digits in natural images with unsupervised feature learning (2011)
Pogorelov, K., et al.: Nerthus: a bowel preparation quality video dataset. In: Proceedings of the 8th ACM on Multimedia Systems Conference, MMSys 2017. ACM, New York (2017)
Raghu, M., et al.: Transfusion: understanding transfer learning for medical imaging. In: NeurIPS (2019)
Ross, T., et al.: Exploiting the potential of unlabeled endoscopic video data with self-supervised learning. Int. J. Comput. Assist. Radiol. Surg. 13(6), 925–933 (2018). https://doi.org/10.1007/s11548-018-1772-0
Ross, T., et al.: Robust medical instrument segmentation challenge 2019 (2020)
Rubner, Y., et al.: A metric for distributions with applications to image databases. In: Sixth International Conference on Computer Vision (1998)
Tschandl, P., et al.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5(1), 1–9 (2018)
Twinanda, A.P., et al.: Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imaging 36(1), 86–97 (2017)
Vilalta, R., et al.: A perspective view and survey of meta-learning. Artif. Intell. Rev. 18(2), 77–95 (2005)
Virtanen, P., et al.: SciPy 1.0: fundamental algorithms for scientific computing in python. Nat. Methods 17(3), 261–272 (2020)
Wightman, R.: Pytorch image models (2019). https://doi.org/10.5281/zenodo.4414861
Yuan, P., et al.: Few is enough: task-augmented active meta-learning for brain cell classification. In: Marte, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 367–377. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_36
Zhang, L., et al.: Generalizing deep learning for medical image segmentation to unseen domains via deep stacked transformation. IEEE Trans. Med. Imaging 39(7), 2531–2540 (2020)
Acknowledgements
We would like to thank our colleagues Minu Dietlinde Tizabi, Tim Adler, Thuy Nuong Tran, Tobias Ross and Lucas-Raphael Müller for their valuable feedback on the drafts of this work. This project has been funded by the Surgical Oncology Program of the National Center for Tumor Diseases (NCT) Heidelberg. The present contribution is also supported by the Helmholtz Imaging Platform (HIP), a platform of the Helmholtz Incubator on Information and Data Science.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Godau, P., Maier-Hein, L. (2021). Task Fingerprinting for Meta Learning inBiomedical Image Analysis. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12904. Springer, Cham. https://doi.org/10.1007/978-3-030-87202-1_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-87202-1_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87201-4
Online ISBN: 978-3-030-87202-1
eBook Packages: Computer ScienceComputer Science (R0)
