Elsevier

Pattern Recognition

Volume 120, December 2021, 108111
Pattern Recognition

MetaMed: Few-shot medical image classification using gradient-based meta-learning

https://doi.org/10.1016/j.patcog.2021.108111Get rights and content

Highlights

  • Efficacy of a gradient-based meta-learning algorithm for few-shot learning problem on real-world non-uniformly distributed medical image datasets is analyzed.

  • Our experiments empirically validate that the use of meta-learning increases the confidence of predictions and robustness.

  • Our work proves that normal augmentation strategies fail to regularize the network in gradient-based meta-learning problems.

  • Hence, we integrated advanced augmentation strategies that can generate virtual samples as well as labels.

  • Showcasing the advantages of advanced augmentation techniques on three complex medical image datasets.

  • Our work significantly reduces the need to collect and annotate large data for deep learning applications in the medical domain.

Abstract

The occurrence of long-tailed distributions and unavailability of high-quality annotated images is a common phenomenon in medical datasets. The use of conventional Deep Learning techniques to obtain an unbiased model with high generalization accuracy for such datasets is a challenging task. Thus, we formulated a few-shot learning problem and presented a meta-learning-based “MetaMed” approach. The model presented here can adapt to rare disease classes with the availability of few images, and less compute. MetaMed is validated on three publicly accessible medical datasets – Pap smear, BreakHis, and ISIC 2018. We used advanced image augmentation techniques like CutOut, MixUp, and CutMix to overcome the problem of over-fitting. Our approach has shown promising results on all the three datasets with an accuracy of more than 70%. Inclusion of advanced augmentation techniques regularizes the model and increases the generalization capability by  2–5%. Comparative analysis of MetaMed against transfer learning demonstrated that MetaMed classifies images with a higher confidence score and on average outperforms transfer learning for 3, 5, and 10-shot tasks for both 2-way and 3-way classification.

Introduction

Humans have the inborn ability to learn quickly by discerning objects from a few samples, acquire new skills in a short time, and make decisions from prior experience and knowledge. They are capable of learning new things and adapting their skills to newer tasks. Recently, the concept of meta-learning (‘learning to learn’) inspired by human intelligence has become quite popular among research communities in machine learning and deep learning. Researchers are motivated to design artificial intelligence (AI) based systems with faster learning capability and ease of adaptation to newer tasks with few samples. However, designing this type of fast adaptable learning system is quite challenging. Machine learning and deep learning have given a new edge to computer vision, and the healthcare industry [1]. Machine learning models are usually trained from scratch for any new tasks, and besides that, they also depend on handcrafted features. Recent developments in deep learning have shown great success in solving challenging tasks and have been adopted across various domains [2], [3]. In a recent work, the authors proposed VGG16 as a feature extractor by modifying the framework for complex breast tumor prediction and applied the concept of transfer learning, which resulted in significant improvement in the performance of complex histopathology data. They used the strength of the deep learning architecture for automatic feature extraction [4] and validated the designed model for both human breast cancer and canine mammary cancer histopathological image datasets. Although deep learning models have proven their adaptability for learning in different domains, these models require high computational resources with huge data samples.

High computing resources and an enormous data requirement are key challenges for deep learning-based architectures in critical applications, such as in the medical field, while reliance on effective feature extraction technique is a major challenge for machine learning-based applications. Therefore, researchers are interested in developing some efficient feature extractors. Recently, researchers have proposed an efficient feature extractor for histopathology images for classification and addressed the challenges of uncertainty, and high computational resource requirement that is quite useful for imagery data-based applications [5]. Also, there are abundant opportunities and challenges in deep learning that attract the attention of researchers [6] while offering potential directions in the design of deep learning architectures for critical applications.

A prerequisite for deep learning is enormous amounts of labeled data, collection of which is very lengthy, expensive, time-consuming, and very difficult for applications of critical nature like the medical field. To address this problem, researches are being conducted to overcome the issue of huge data set requirement by developing new methods and techniques. Additionally, collection of annotated datasets is a difficult task for rare and emerging diseases where limited samples are available. The data collected from different sites in the healthcare sector may have significantly different distributions due to variations in patient cohorts or image acquisition devices. Therefore, learning a model that generalizes to new clinical sites would be of paramount significance in healthcare applications. Unlike common diseases with the availability of large amounts of labeled data, automated diagnosis of rare and complex diseases such as cancer with extremely small data regimes is a difficult task. Under these circumstances understanding the nature of diseases such as cancer, which is estimated to spread as 27 million cases by 2030 [7], becomes really challenging.

Thus, fast and accurate diagnosis of cancer using computational tools is a priority research problem for better management of cancer patients. But due to the limitation of data size, it is difficult to apply deep learning based approaches for automated cancer diagnosis. However, to address these challenges approaches like image generation using Generative Adversarial Network (GAN) [8] have been considered to handle the scarcity of data. In this work, authors used the artificial images generated by the GAN to train a recognition model for thyroid disease. Several studies [2], [4] adopted transfer learning based approach in which pre-trained convolutional networks that are trained on non-medical data like ImageNet are utilized. But to get significant performance for diagnosis, substantial fine-tuning is required, resulting in a longer training period. Thus, in order to overcome the challenges of scarcity of labeled data and limited resources, the research community needs to focus on developing efficient computational models with faster learning capabilities even with small datasets, and adaptability to new tasks.

Recently, meta-learning based models for few-shot learning are gaining popularity to address the above-stated challenges. Meta-learning [9], [10] is a long-standing topic and fosters the learning strategies to solve the problem of Artificial General Intelligence (AGI). Though we are far away from AGI, meta-learning has made great strides in image recognition, reinforcement learning, and regression. Human intelligence and their quick adaptation for new skills from prior experiences are the key factors for developing this new paradigm, known as meta-learning. In the case of healthcare applications, learning a model that generalizes to new clinical sites would be of high significance.

Meta-learning based models usually consist of two constituents which are meta learner and learner. Here, a meta learner is one who is trained on the distribution of tasks to teach by acting as a teacher. The other component a learner (who is a student) and learns how to update its parameters. Recently it has received wide attention from researchers to solve the problem of limited resources, scarcity of labeled samples, and domain generalization. Zero-Shot Learning [11], [12] is another promising way to recognize an unseen rare class object by transferring the knowledge gained from model training on seen classes. Different works have been reported in this direction. In a study, the authors proposed StyleGuide [13], a sketch-based image retrieval method. The retrieval of images in this method is done by utilizing the idea of ‘content style decomposition’ while initiating the process of fake image generation guided in a certain pattern (style). This process is also used to address the issue of image retrieval through zero-shot style. It was further extended for generalized zero-shot learning for unseen class image retrieval. There has been efforts from Meta-DermDiagnosis [14] to apply meta-learning techniques to skin lesion classification on ISIC 2018 dataset. Meta-DermDiagnosis used gradient-based meta-learning algorithm ‘Reptile’ [15] along with Group Equivariant Convolution for disease identification. In another recent work, a novel Difficulty Aware Meta-Learning (DAML) [16] based on meta learning method is proposed that optimizes the meta optimization part by monitoring dynamically the importance of learning task while giving more emphasis to a hard task. Recently, MetaCOVID [17] has been proposed where deep meta-learning with contrastive loss and ‘n’-shot learning is used for fast diagnosis of COVID-19. In this work, a fine-tuned VGG16-based encoder is applied for feature extraction, and then the Siamese network is used for COVID-19 classification.

In view of the advantages of meta-learning approach highlighted above, we are inspired to use the concept of meta-learning for medical image classification. We have made efforts to use gradient-based meta-learning algorithm and integrated augmentations like CutOut, MixUp, and CutMix on medical imaging datasets to enhance the overall performance. To the best of our knowledge, this type of integrated approach has not been used in prior works. We have validated the effectiveness of the proposed model on three publicly available medical image datasets collected from the real-world clinical settings namely: i) BreakHis dataset [18] comprising histopathological images of breast tumor tissues, ii) ISIC 2018 [19] skin lesion dataset consisting of dermoscopic images of skin lesions and, iii) Pap smear dataset [20] comprising microscopic images of cervical smears stained using Papanicolaou method for detecting precancerous and cancerous cells in cervical cancer.

Motivation and contribution We are motivated by the skills of human intelligence and its capability to quickly learn from fewer samples, as well as, the ability to utilize prior knowledge and experience for solving new problems. The non-availability of huge amount of labeled medical data, especially in the case of rare diseases is very challenging. Additionally, data collection in cases such as a cancer diagnosis is costly and tedious. Thus, an automatic diagnostic model is needed as a support system for medical experts so as to enable a faster and more accurate diagnosis of critical diseases. Though deep learning and machine learning based models are developed to tackle these tasks, they have their own limitations. Deep learning algorithms require a humongous amount of data in order to precisely classify given set of test images. This is mostly not available in the case of rare diseases. Training on limited examples can have an adverse impact on generalization capability and may also severely bias the model. Researchers have adopted techniques like transfer learning [21] and explored different fine-tuning techniques for medical datasets. Fine-tuning requires that the dataset on which the pre-trained model is trained should have similar data distribution to the dataset on which the model is to be fine-tuned. In the case of medical image classification, we do not have a large database like ImageNet on which we can do pre-training. To palliate the scarcity of data, efforts have been made to generate synthetic data using data augmentation [22] and GAN [23]. Training GAN requires tuning of multiple parameters to generate good virtual images. Thus, there is a need for new strategies to deal with the problem of data scarcity. Meta-learning is one such paradigm that has opened abundant possibilities to tackle various challenges, especially in the medical domain. So, to overcome all these challenges and to enhance the generalization capability of the classification model with better confidence for rare diseases we have proposed an approach based on meta-learning which works with few data samples. The major contributions to this work are as follows:

  • Our work proposes the use of gradient-based meta-learning approach for medical image classification problems where there is a scarcity of high quality annotated data, and we simulate this problem by posing it as a few-shot learning problem.

  • Our work proposes to regularize the model using advanced augmentation techniques like MixUp, CutOut and CutMix during the meta-training stage.

  • Our work can be generalized to any modality of medical data, and we have proved this by conducting experiments on three different real-world datasets - BreakHis, ISIC 2018 and Pap Smear.

  • Thorough analysis of confidence scores for predictions made by the model trained using transfer learning and meta-learning reveals that, meta-learning technique significantly improves the model confidence compared to transfer learning.

  • Our work, significantly brings down the need for collection and annotation of huge quantity of data for deep learning based solutions in medical domain.

The paper is organized as follows: Section 2 outlines the basic definitions and concepts related to the proposed MetaMed. Section 3 deals with detailing intricacy of MetaMed. Section 4 presents the experimental validation and discussion of MetaMed. Finally, Section 5 concludes by accentuating the findings of this study and outlines the possible future directions.

Section snippets

Preliminaries

This part presents basic definitions and conceptual detail of the technique from prior literature which is quite relevant to the proposed work.

MetaMed

We describe a few-shot image classification problem in this section and propose a meta-learning based solution for datasets with long-tailed distribution.

Experimental results

We evaluated our approach on three medical image datasets. All evaluations are done for 2-way and 3-way classification tasks and for 3-shot, 5-shot, and 10-shot learning with traditional and advanced augmentation techniques. We also evaluated transfer-learning approach, where the model is trained on all meta-train classes and then fine-tuned on few-shots from the meta-test set. This work is the first to explore the use of gradient based meta-learning for multiple modalities of medical images

Conclusion

In this study, we proposed a MetaMed approach that relies on meta-learning by formulating the medical image classification for low data regime as a few-shot learning problem. We have also integrated advanced augmentation techniques like CutOut, MixUp, and CutMix to regularize the model, and its efficacy is validated on three complex medical image datasets. By performing rigorous analysis, we conclude that meta-learning performs better than transfer-learning, as we average across all dataset

Declaration of Competing Interest

Authors declare that they have no conflict of interest.

Rishav Singh has around 8 years of experience including 6 years of extensive IT experience with proven expertise in full R, Cassandra, Machine Learning, Cloudera Hadoop (Map Reduce), Hbase, Hive. His current research area includes Machine Learning, Deep Learning, Biometrics, Medical Image analysis and pattern recognition. Currently he is serving as an Assistant Professor at NIT Delhi, India. He has authored and co-authored several peer-reviewed articles and has filled two patents.

References (40)

  • Q. Zhang et al.

    Medical image synthesis with generative adversarial networks for tissue recognition

    2018 IEEE International Conference on Healthcare Informatics (ICHI)

    (2018)
  • J. Schmidhuber

    Evolutionary Principles in Self-Referential Learning, or on Learning how to Learn: The Meta-meta-... Hook

    (1987)
  • S. Thrun et al.

    Learning to learn: introduction and overview

    Learning to Learn

    (1998)
  • T. Dutta et al.

    Styleguide: zero-shot sketch-based image retrieval using style-guided image generation

    IEEE Trans. Multimed.

    (2020)
  • K. Mahajan et al.

    Meta-dermdiagnosis: few-shot skin disease identification using meta-learning

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops

    (2020)
  • A. Nichol, J. Schulman, Reptile: a scalable metalearning algorithm, 2(3) (2018) 4. arXiv preprint...
  • X. Li et al.

    Difficulty-aware meta-learning for rare disease diagnosis

    International Conference on Medical Image Computing and Computer-Assisted Intervention

    (2020)
  • F.A. Spanhol et al.

    A dataset for breast cancer histopathological image classification

    IEEE Trans. Biomed. Eng.

    (2015)
  • J. Zou, X. Ma, C. Zhong, Y. Zhang, Dermoscopic image analysis for ISIC challenge 2018, (2018). arXiv preprint...
  • J. Jantzen et al.

    Pap-smear benchmark data for pattern classification

    Nature Inspired Smart Information Systems (NiSIS)

    (2005)
  • Cited by (99)

    • Medical image identification methods: A review

      2024, Computers in Biology and Medicine
    View all citing articles on Scopus

    Rishav Singh has around 8 years of experience including 6 years of extensive IT experience with proven expertise in full R, Cassandra, Machine Learning, Cloudera Hadoop (Map Reduce), Hbase, Hive. His current research area includes Machine Learning, Deep Learning, Biometrics, Medical Image analysis and pattern recognition. Currently he is serving as an Assistant Professor at NIT Delhi, India. He has authored and co-authored several peer-reviewed articles and has filled two patents.

    Vandana Bharti received her M.Tech in Computer Science from the Birla Institute of Technology (BIT), Mesra, India, in 2016. Currently, she is a Ph.D. scholar and teaching assistant in the Department of Computer Science and Engineering at IIT-BHU Varanasi, India. She has authored and co-authored several peer-reviewed articles. Her research interests include nature-inspired optimization, multiobjective optimization, hyperspectral imaging, pattern classification, machine learning, and deep learning. She is also a student member of the ACM and IEEE.

    Vishal Purohit is currently working as research intern at IIT Varanasi (BHU). He has B.E in Electronics and Communication engineering from KLS Gogte Institute of Technology (VTU). He has worked in the industry for almost 2 years on application of Deep Learning algorithms in medical domain. He is also incoming MS ECE student at Purdue University.

    Abhinav Kumar received his M.Tech in Computer Science from the Birla Institute of Technology (BIT), Mesra, India, in 2016. Currently, he is a Ph.D. scholar and teaching assistant in the Department of Computer Science and Engineering at IIT-BHU Varanasi, India. He has authored and co-authored several peer-reviewed articles, and has filed one patent. His research interests include medical imaging, pattern classification, machine learning, and deep learning. He is also a student member of the ACM and IEEE.

    Amit Kumar Singh is currently an Assistant Professor with the Computer Science and Engineering Department, National Institute of Technology Patna, Bihar, India. He has authored over 100 peer-reviewed journals, conference publications, and book chapters. He has authored three books and edited four books with internationally recognized publishers such as Springer and Elsevier. He is the associate editor of IEEE Access (Since 2016), IET Image Processing (Since 2020), and former member of the editorial board of Multimedia Tools and Applications, Springer (2015–2019). He has edited various international journal special issues as a lead guest editor. He has obtained the memberships from several international academic organizations such as ACM and IEEE. His research interests include multimedia data hiding, image processing, biometrics, & Cryptography.

    Sanjay Kumar Singh is currently a Professor in the Department of Computer Science and Engineering, IIT (BHU), Varanasi, India. He has authored or co-authored more than 150 national & international journal publications, book chapters, and conference papers. He has 4 patents filed to his credit. His current research interests include machine learning, deep learning, computer vision, medical image analysis, pattern recognition, and biometrics. He is a senior member of the IEEE, ACM, and computer society of India. He is also a guest editorial board member and reviewer for many international journals of repute.

    1

    Co-first author (First and Co-first authors have equal contribution).

    View full text