MetaMed: Few-shot medical image classification using gradient-based meta-learning
Introduction
Humans have the inborn ability to learn quickly by discerning objects from a few samples, acquire new skills in a short time, and make decisions from prior experience and knowledge. They are capable of learning new things and adapting their skills to newer tasks. Recently, the concept of meta-learning (‘learning to learn’) inspired by human intelligence has become quite popular among research communities in machine learning and deep learning. Researchers are motivated to design artificial intelligence (AI) based systems with faster learning capability and ease of adaptation to newer tasks with few samples. However, designing this type of fast adaptable learning system is quite challenging. Machine learning and deep learning have given a new edge to computer vision, and the healthcare industry [1]. Machine learning models are usually trained from scratch for any new tasks, and besides that, they also depend on handcrafted features. Recent developments in deep learning have shown great success in solving challenging tasks and have been adopted across various domains [2], [3]. In a recent work, the authors proposed VGG16 as a feature extractor by modifying the framework for complex breast tumor prediction and applied the concept of transfer learning, which resulted in significant improvement in the performance of complex histopathology data. They used the strength of the deep learning architecture for automatic feature extraction [4] and validated the designed model for both human breast cancer and canine mammary cancer histopathological image datasets. Although deep learning models have proven their adaptability for learning in different domains, these models require high computational resources with huge data samples.
High computing resources and an enormous data requirement are key challenges for deep learning-based architectures in critical applications, such as in the medical field, while reliance on effective feature extraction technique is a major challenge for machine learning-based applications. Therefore, researchers are interested in developing some efficient feature extractors. Recently, researchers have proposed an efficient feature extractor for histopathology images for classification and addressed the challenges of uncertainty, and high computational resource requirement that is quite useful for imagery data-based applications [5]. Also, there are abundant opportunities and challenges in deep learning that attract the attention of researchers [6] while offering potential directions in the design of deep learning architectures for critical applications.
A prerequisite for deep learning is enormous amounts of labeled data, collection of which is very lengthy, expensive, time-consuming, and very difficult for applications of critical nature like the medical field. To address this problem, researches are being conducted to overcome the issue of huge data set requirement by developing new methods and techniques. Additionally, collection of annotated datasets is a difficult task for rare and emerging diseases where limited samples are available. The data collected from different sites in the healthcare sector may have significantly different distributions due to variations in patient cohorts or image acquisition devices. Therefore, learning a model that generalizes to new clinical sites would be of paramount significance in healthcare applications. Unlike common diseases with the availability of large amounts of labeled data, automated diagnosis of rare and complex diseases such as cancer with extremely small data regimes is a difficult task. Under these circumstances understanding the nature of diseases such as cancer, which is estimated to spread as 27 million cases by 2030 [7], becomes really challenging.
Thus, fast and accurate diagnosis of cancer using computational tools is a priority research problem for better management of cancer patients. But due to the limitation of data size, it is difficult to apply deep learning based approaches for automated cancer diagnosis. However, to address these challenges approaches like image generation using Generative Adversarial Network (GAN) [8] have been considered to handle the scarcity of data. In this work, authors used the artificial images generated by the GAN to train a recognition model for thyroid disease. Several studies [2], [4] adopted transfer learning based approach in which pre-trained convolutional networks that are trained on non-medical data like ImageNet are utilized. But to get significant performance for diagnosis, substantial fine-tuning is required, resulting in a longer training period. Thus, in order to overcome the challenges of scarcity of labeled data and limited resources, the research community needs to focus on developing efficient computational models with faster learning capabilities even with small datasets, and adaptability to new tasks.
Recently, meta-learning based models for few-shot learning are gaining popularity to address the above-stated challenges. Meta-learning [9], [10] is a long-standing topic and fosters the learning strategies to solve the problem of Artificial General Intelligence (AGI). Though we are far away from AGI, meta-learning has made great strides in image recognition, reinforcement learning, and regression. Human intelligence and their quick adaptation for new skills from prior experiences are the key factors for developing this new paradigm, known as meta-learning. In the case of healthcare applications, learning a model that generalizes to new clinical sites would be of high significance.
Meta-learning based models usually consist of two constituents which are meta learner and learner. Here, a meta learner is one who is trained on the distribution of tasks to teach by acting as a teacher. The other component a learner (who is a student) and learns how to update its parameters. Recently it has received wide attention from researchers to solve the problem of limited resources, scarcity of labeled samples, and domain generalization. Zero-Shot Learning [11], [12] is another promising way to recognize an unseen rare class object by transferring the knowledge gained from model training on seen classes. Different works have been reported in this direction. In a study, the authors proposed StyleGuide [13], a sketch-based image retrieval method. The retrieval of images in this method is done by utilizing the idea of ‘content style decomposition’ while initiating the process of fake image generation guided in a certain pattern (style). This process is also used to address the issue of image retrieval through zero-shot style. It was further extended for generalized zero-shot learning for unseen class image retrieval. There has been efforts from Meta-DermDiagnosis [14] to apply meta-learning techniques to skin lesion classification on ISIC 2018 dataset. Meta-DermDiagnosis used gradient-based meta-learning algorithm ‘Reptile’ [15] along with Group Equivariant Convolution for disease identification. In another recent work, a novel Difficulty Aware Meta-Learning (DAML) [16] based on meta learning method is proposed that optimizes the meta optimization part by monitoring dynamically the importance of learning task while giving more emphasis to a hard task. Recently, MetaCOVID [17] has been proposed where deep meta-learning with contrastive loss and ‘n’-shot learning is used for fast diagnosis of COVID-19. In this work, a fine-tuned VGG16-based encoder is applied for feature extraction, and then the Siamese network is used for COVID-19 classification.
In view of the advantages of meta-learning approach highlighted above, we are inspired to use the concept of meta-learning for medical image classification. We have made efforts to use gradient-based meta-learning algorithm and integrated augmentations like CutOut, MixUp, and CutMix on medical imaging datasets to enhance the overall performance. To the best of our knowledge, this type of integrated approach has not been used in prior works. We have validated the effectiveness of the proposed model on three publicly available medical image datasets collected from the real-world clinical settings namely: i) BreakHis dataset [18] comprising histopathological images of breast tumor tissues, ii) ISIC 2018 [19] skin lesion dataset consisting of dermoscopic images of skin lesions and, iii) Pap smear dataset [20] comprising microscopic images of cervical smears stained using Papanicolaou method for detecting precancerous and cancerous cells in cervical cancer.
Motivation and contribution We are motivated by the skills of human intelligence and its capability to quickly learn from fewer samples, as well as, the ability to utilize prior knowledge and experience for solving new problems. The non-availability of huge amount of labeled medical data, especially in the case of rare diseases is very challenging. Additionally, data collection in cases such as a cancer diagnosis is costly and tedious. Thus, an automatic diagnostic model is needed as a support system for medical experts so as to enable a faster and more accurate diagnosis of critical diseases. Though deep learning and machine learning based models are developed to tackle these tasks, they have their own limitations. Deep learning algorithms require a humongous amount of data in order to precisely classify given set of test images. This is mostly not available in the case of rare diseases. Training on limited examples can have an adverse impact on generalization capability and may also severely bias the model. Researchers have adopted techniques like transfer learning [21] and explored different fine-tuning techniques for medical datasets. Fine-tuning requires that the dataset on which the pre-trained model is trained should have similar data distribution to the dataset on which the model is to be fine-tuned. In the case of medical image classification, we do not have a large database like ImageNet on which we can do pre-training. To palliate the scarcity of data, efforts have been made to generate synthetic data using data augmentation [22] and GAN [23]. Training GAN requires tuning of multiple parameters to generate good virtual images. Thus, there is a need for new strategies to deal with the problem of data scarcity. Meta-learning is one such paradigm that has opened abundant possibilities to tackle various challenges, especially in the medical domain. So, to overcome all these challenges and to enhance the generalization capability of the classification model with better confidence for rare diseases we have proposed an approach based on meta-learning which works with few data samples. The major contributions to this work are as follows:
- •
Our work proposes the use of gradient-based meta-learning approach for medical image classification problems where there is a scarcity of high quality annotated data, and we simulate this problem by posing it as a few-shot learning problem.
- •
Our work proposes to regularize the model using advanced augmentation techniques like MixUp, CutOut and CutMix during the meta-training stage.
- •
Our work can be generalized to any modality of medical data, and we have proved this by conducting experiments on three different real-world datasets - BreakHis, ISIC 2018 and Pap Smear.
- •
Thorough analysis of confidence scores for predictions made by the model trained using transfer learning and meta-learning reveals that, meta-learning technique significantly improves the model confidence compared to transfer learning.
- •
Our work, significantly brings down the need for collection and annotation of huge quantity of data for deep learning based solutions in medical domain.
The paper is organized as follows: Section 2 outlines the basic definitions and concepts related to the proposed MetaMed. Section 3 deals with detailing intricacy of MetaMed. Section 4 presents the experimental validation and discussion of MetaMed. Finally, Section 5 concludes by accentuating the findings of this study and outlines the possible future directions.
Section snippets
Preliminaries
This part presents basic definitions and conceptual detail of the technique from prior literature which is quite relevant to the proposed work.
MetaMed
We describe a few-shot image classification problem in this section and propose a meta-learning based solution for datasets with long-tailed distribution.
Experimental results
We evaluated our approach on three medical image datasets. All evaluations are done for 2-way and 3-way classification tasks and for 3-shot, 5-shot, and 10-shot learning with traditional and advanced augmentation techniques. We also evaluated transfer-learning approach, where the model is trained on all meta-train classes and then fine-tuned on few-shots from the meta-test set. This work is the first to explore the use of gradient based meta-learning for multiple modalities of medical images
Conclusion
In this study, we proposed a MetaMed approach that relies on meta-learning by formulating the medical image classification for low data regime as a few-shot learning problem. We have also integrated advanced augmentation techniques like CutOut, MixUp, and CutMix to regularize the model, and its efficacy is validated on three complex medical image datasets. By performing rigorous analysis, we conclude that meta-learning performs better than transfer-learning, as we average across all dataset
Declaration of Competing Interest
Authors declare that they have no conflict of interest.
Rishav Singh has around 8 years of experience including 6 years of extensive IT experience with proven expertise in full R, Cassandra, Machine Learning, Cloudera Hadoop (Map Reduce), Hbase, Hive. His current research area includes Machine Learning, Deep Learning, Biometrics, Medical Image analysis and pattern recognition. Currently he is serving as an Assistant Professor at NIT Delhi, India. He has authored and co-authored several peer-reviewed articles and has filled two patents.
References (40)
- et al.
A survey on deep learning in medical image analysis
Med. Image Anal.
(2017) - et al.
Deep feature learning for histopathological image classification of canine mammary tumors and human breast cancer
Inf. Sci.
(2020) - et al.
Zero-shot event detection via event-adaptive concept relevance mining
Pattern Recognit.
(2019) - et al.
Guided CNN for generalized zero-shot and open-set recognition using visual and semantic prototypes
Pattern Recognit.
(2020) - et al.
Metacovid: A siamese neural network framework with contrastive loss for n-shot diagnosis of COVID-19 patients
Pattern Recognit.
(2021) - et al.
Imbalanced breast cancer classification using transfer learning
IEEE/ACM Trans. Comput. Biol. Bioinf.
(2021) - et al.
Deep learning in robotics: survey on model structures and training strategies
IEEE Trans. Syst. Man. Cybern.
(2021) - et al.
Comhisp: a novel feature extractor for histopathological image classification based on fuzzy SVM with within-class relative density
IEEE Trans. Fuzzy Syst.
(2021) - et al.
Recent trends in nature inspired computation with applications to deep learning
2020 10th International Conference on Cloud Computing, Data Science & Engineering (Confluence)
(2020) - F. Bray, I. Soerjomataram, The Changing Global Burden of Cancer: Transitions in Human Development and Implications for...
Medical image synthesis with generative adversarial networks for tissue recognition
2018 IEEE International Conference on Healthcare Informatics (ICHI)
Evolutionary Principles in Self-Referential Learning, or on Learning how to Learn: The Meta-meta-... Hook
Learning to learn: introduction and overview
Learning to Learn
Styleguide: zero-shot sketch-based image retrieval using style-guided image generation
IEEE Trans. Multimed.
Meta-dermdiagnosis: few-shot skin disease identification using meta-learning
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
Difficulty-aware meta-learning for rare disease diagnosis
International Conference on Medical Image Computing and Computer-Assisted Intervention
A dataset for breast cancer histopathological image classification
IEEE Trans. Biomed. Eng.
Pap-smear benchmark data for pattern classification
Nature Inspired Smart Information Systems (NiSIS)
Cited by (99)
Transformers, convolutional neural networks, and few-shot learning for classification of histopathological images of oral cancer
2024, Expert Systems with ApplicationsMedical image identification methods: A review
2024, Computers in Biology and MedicineDataMap: Dataset transferability map for medical image classification
2024, Pattern RecognitionFew-shot learning-based human behavior recognition model
2024, Computers in Human BehaviorA few-shot disease diagnosis decision making model based on meta-learning for general practice
2024, Artificial Intelligence in Medicine
Rishav Singh has around 8 years of experience including 6 years of extensive IT experience with proven expertise in full R, Cassandra, Machine Learning, Cloudera Hadoop (Map Reduce), Hbase, Hive. His current research area includes Machine Learning, Deep Learning, Biometrics, Medical Image analysis and pattern recognition. Currently he is serving as an Assistant Professor at NIT Delhi, India. He has authored and co-authored several peer-reviewed articles and has filled two patents.
Vandana Bharti received her M.Tech in Computer Science from the Birla Institute of Technology (BIT), Mesra, India, in 2016. Currently, she is a Ph.D. scholar and teaching assistant in the Department of Computer Science and Engineering at IIT-BHU Varanasi, India. She has authored and co-authored several peer-reviewed articles. Her research interests include nature-inspired optimization, multiobjective optimization, hyperspectral imaging, pattern classification, machine learning, and deep learning. She is also a student member of the ACM and IEEE.
Vishal Purohit is currently working as research intern at IIT Varanasi (BHU). He has B.E in Electronics and Communication engineering from KLS Gogte Institute of Technology (VTU). He has worked in the industry for almost 2 years on application of Deep Learning algorithms in medical domain. He is also incoming MS ECE student at Purdue University.
Abhinav Kumar received his M.Tech in Computer Science from the Birla Institute of Technology (BIT), Mesra, India, in 2016. Currently, he is a Ph.D. scholar and teaching assistant in the Department of Computer Science and Engineering at IIT-BHU Varanasi, India. He has authored and co-authored several peer-reviewed articles, and has filed one patent. His research interests include medical imaging, pattern classification, machine learning, and deep learning. He is also a student member of the ACM and IEEE.
Amit Kumar Singh is currently an Assistant Professor with the Computer Science and Engineering Department, National Institute of Technology Patna, Bihar, India. He has authored over 100 peer-reviewed journals, conference publications, and book chapters. He has authored three books and edited four books with internationally recognized publishers such as Springer and Elsevier. He is the associate editor of IEEE Access (Since 2016), IET Image Processing (Since 2020), and former member of the editorial board of Multimedia Tools and Applications, Springer (2015–2019). He has edited various international journal special issues as a lead guest editor. He has obtained the memberships from several international academic organizations such as ACM and IEEE. His research interests include multimedia data hiding, image processing, biometrics, & Cryptography.
Sanjay Kumar Singh is currently a Professor in the Department of Computer Science and Engineering, IIT (BHU), Varanasi, India. He has authored or co-authored more than 150 national & international journal publications, book chapters, and conference papers. He has 4 patents filed to his credit. His current research interests include machine learning, deep learning, computer vision, medical image analysis, pattern recognition, and biometrics. He is a senior member of the IEEE, ACM, and computer society of India. He is also a guest editorial board member and reviewer for many international journals of repute.
- 1
Co-first author (First and Co-first authors have equal contribution).