Two-phase non-invasive multi-disease detection via sublingual region

https://doi.org/10.1016/j.compbiomed.2021.104782Get rights and content

Highlights

  • To the best of our knowledge, this is the first quantitative study on the sublingual vein for non-invasive multi-disease detection.

  • In this paper, we propose a two-phase sublingual-based disease detection framework for effective non-invasive multi-disease detection.

  • We generate multiple feature representations based on the sublingual vein region and obtain the optimal multi-feature representation.

  • We performed extensive experiments to prove that the proposed method is highly effective in disease detection.

Abstract

Non-invasive multi-disease detection is an active technology that detects human diseases automatically. By observing images of the human body, computers can make inferences on disease detection based on artificial intelligence and computer vision techniques. The sublingual vein, lying on the lower part of the human tongue, is a critical identifier in non-invasive multi-disease detection, reflecting health status. However, few studies have fully investigated non-invasive multi-disease detection via the sublingual vein using a quantitative method. In this paper, a two-phase sublingual-based disease detection framework for non-invasive multi-disease detection was proposed. In this framework, sublingual vein region segmentation was performed on each image in the first phase to achieve the region with the highest probability of covering the sublingual vein. In the second phase, features in this region were extracted, and multi-class classification was applied to these features to output a detection result. To better represent the characterisation of the obtained sublingual vein region, multi-feature representations were generated of the sublingual vein region (based on color, texture, shape, and latent representation). The effectiveness of sublingual-based multi-disease detection was quantitatively evaluated, and the proposed framework was based on 1103 sublingual vein images from patients in different health status categories. The best multi-feature representation was generated based on color, texture, and latent representation features with the highest accuracy of 98.05%.

Introduction

The sublingual vein is a pair of veins located on the lower surface of the tongue lying on either side of the lingual frenulum. This is critical evidence for disease detection in traditional Chinese medicine (TCM) [36]. Studies show that blood stasis on the sublingual vein has a close relationship with health status in humans [3,5,35,42,43,49]. The sublingual vein of a normal person usually appears to be dark red with no vasodilation. If the vascular structure is not clear or is present in a faint color, there is a higher probability that some abnormal health status exists. In addition, since the sublingual vein is connected to the viscera, it can reflect the status of viscera-related diseases. By applying computerised disease diagnosis methods [8,13,37], disease detection can be performed according to information from multiple modalities, such as image, sound, and odour [6,17,18,44,47].

Recently, researchers have frequently focused on non-invasive computerized disease detection methods [3,8,17,30,44,45,47]. Non-invasive disease detection methods do not require the extraction of bodily fluids from patients and are a pain-free diagnosis method. In contrast, conventional diagnosis methods, such as diabetes mellitus diagnosis, require a blood test, causing patients pain and discomfort. One popular and commonly used non-invasive detection method is computerised tongue diagnosis, which originated from TCM [8,36,37] and believes that the appearance of the human tongue reflects the health status of the body. Extensive studies have focused on the anterior part of the tongue [24,37,38,46,50], while there are few studies on the sublingual vein (underside of the tongue). As mentioned in Ref. [5], the sublingual vessels connect directly with the internal organs, including the heart, liver, and kidneys. In addition, the sublingual vein is sensitive to blood stasis and can reflect the activity of blood circulation. Therefore, the sublingual vein can be a disease indicator for internal organs (e.g., kidney disease, gastritis, high blood pressure). Since the sublingual collaterals stem from the tongue and can be effectively used in computerised methods for detecting diabetes mellitus [17,44], it is possible to detect them from the sublingual vein [35]. found that the color of the sublingual vein has a positive relationship with blood stasis, which means computerised automatic disease detection is feasible via sublingual vein images [3,5,35]. performed segmentation based on blood stasis for disease detection. The aforementioned methods have four drawbacks and limitations: 1) pixel-wise segmentation on the sublingual vein is ineffective in some circumstances, since the sublingual vein can be faint or a pale color (at times), which is challenging to extract as an image; 2) these methods are designed for binary disease detection, rather than multi-disease detection; 3) there is no unified framework for multi-disease detection, including sublingual vein extraction, feature representation, and classification; and 4) most of the methods did not focus on investigating the effectiveness of the sublingual vein in disease detection.

In this paper, a two-phase sublingual-based disease detection framework was proposed for non-invasive multi-disease detection. In the first phase, the sublingual vein region was segmented on the raw image, which was captured using a non-invasive imaging device. In the second phase, features were extracted from the sublingual vein region, such as color, texture, geometry, and latent representation. Subsequently, a multi-class classification was performed on the features extracted from the sublingual vein region. To segment the sublingual vein region, Yolov3 architecture [29] was applied to first detect the bounding box, which is the area covering the sublingual vein with the highest probability. To better represent the characterisation of the obtained sublingual vein region, multiple feature representations were generated: color, texture, shape, and latent representation. For the color feature, a color histogram [26] and color moments [34]were calculated. The texture feature was extracted using LBP [22] and Gabor filters [9]. Regarding the geometry feature, nine descriptors were defined: length, width, length-width ratio, area, smaller half-distance, circle area, circle area ratio, square area, and square area ratio. For the latent representation, the stacked sparse autoencoder (SSAE) [14], which is widely used to extract latent representations from an image [41] was applied. Each group of features was first tested individually in the experiments by applying a quantitative evaluation. Finally, the optimal multi-feature representation was determined for multi-disease detection. Overall, there were four main contributions of this paper:

  • 1)

    To the best of our knowledge, this is the first time such a study on the sublingual vein for non-invasive multi-disease detection has been established using a quantitative method.

  • 2)

    A two-phase sublingual-based disease detection framework was proposed for effective non-invasive multi-disease detection.

  • 3)

    Multiple feature representations based on the sublingual vein region were generated, and the optimal multi-feature representation was obtained empirically.

  • 4)

    The experiments demonstrated that the proposed method was highly effective for disease detection.

The remaining parts of the paper are organised as follows: In section 2 related works and details of the sublingual vein image are first introduced. This is followed by a presentation of the methodology of the two-phase non-invasive multi-disease detection method using the sublingual vein region in section 3. In section 4, a quality evaluation, quantitative evaluation, as well as discussion of the proposed method, were performed before discussing the experiments in detail. Finally, a conclusion is reached in section 5.

Section snippets

Non-invasive disease detection

Non-invasive disease detection methods perform diagnosis based on the appearance of the human body without extracting bodily fluids [44]. These methods are based on the presumption that changes in the health status of internal organs will cause different reflections on the appearance of the human body [47]. Zhang et al. [44] analysed the tongue images of patients to detect diabetes mellitus and showed the feasibility of disease detection using the human tongue [47]. applied the image blocks

Two-phase non-invasive multi-disease detection via the sublingual vein region: overview

A two-phase disease detection framework was proposed based on the sublingual vein images, as shown in section 2. The overall workflow of the framework is depicted in Fig. 3. The framework performs disease detection using several steps. First, the sublingual vein region from any given image is segmented with a trained sublingual vein region detector (section 3.2). The trained sublingual vein region detector segments the region with the highest probability covering the most pixels in the

Experimental settings

Sublingual vein images were collected from patients in different health status categories, namely chronic kidney disease, diabetes mellitus, high blood pressure, and chronic gastritis. In addition, a healthy control group was added. The distribution of samples in these categories is presented in Table 1 (the 200 images for fine-tuning the segmenter are not included). All images were captured at the Guangdong Provincial Hospital of Traditional Chinese Medicine. The label for each image was

Conclusion

In this paper, a two-phase non-invasive multi-disease detection method was proposed using the sublingual vein region. In the first phase, the sublingual vein region was segmented from its raw image. In the second phase, a multi-feature representation was generated based on the sublingual vein region. To perform multi-disease detection, KNN, SVM, ProCRC, LDA, SRC, CRC, Random Forest, and RSLDA were applied. Our proposed method achieved 98.05% accuracy (precision: 97.76%, recall: 97.27%, and

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (50)

  • X. Wang et al.

    A high quality color imaging system for computerized tongue image analysis

    Expert Syst. Appl.

    (2013)
  • Z. Yan et al.

    Computerized feature quantification of sublingual veins from color sublingual images

    Comput. Methods Progr. Biomed.

    (2009)
  • Q. Zhang et al.

    Computational traditional Chinese medicine diagnosis: a literature survey

    Comput. Biol. Med.

    (2021)
  • L. Zhi et al.

    Classification of hyperspectral medical tongue images for tongue diagnosis

    Comput. Med. Imag. Graph.

    (2007)
  • N.S. Altman

    An introduction to kernel and nearest-neighbor nonparametric regression

    Am. Statistician

    (1992)
  • S. Cai et al.

    A probabilistic collaborative representation based approach for pattern classification

  • F. Chen et al.

    Computerized analysis of tongue sub-lingual veins to detect lung and breast cancers

  • C. Cortes et al.

    Support-vector networks

    Mach. Learn.

    (1995)
  • Z. David et al.

    Medical Biometrics: Computerized TCM Data Analysis

    (2016)
  • H.G. Feichtinger et al.

    Gabor Analysis and Algorithms: Theory and Applications

    (2012)
  • D. Gregorysmith

    Theoretical Foundations of Chinese-medicine-systems of Correspondence-Porkert

    (1979)
  • T.K. Ho

    Random decision forests

  • B. Kirschbaum

    Atlas of Chinese Tongue Dignosis

    (2000)
  • M.A. Kramer

    Nonlinear principal component analysis using autoassociative neural networks

    AIChE J.

    (1991)
  • A. Krizhevsky et al.

    Imagenet classification with deep convolutional neural networks

    Adv. Neural Inf. Process. Syst.

    (2012)
  • Cited by (5)

    • Missing-view completion for fatty liver disease detection

      2022, Computers in Biology and Medicine
      Citation Excerpt :

      To segment the sublingual vein region, we implemented a deep learning model, i.e., Yolov3. This model can use global information for performing detection inference [33,34]. Specifically, it was provided with a more precise detection box prediction for the sublingual vein region, while overlooking the useless parts of the input image.

    • A multi-step approach for tongue image classification in patients with diabetes

      2022, Computers in Biology and Medicine
      Citation Excerpt :

      We compare the clustering results of various features, including TDAS features, VQ-VAE features, LBP features, and HOG features. The LBP and HOG features are calculated by the Local Binary Pattern algorithm [43,55] and the Histogram of Oriented Gradient algorithm [56,57], commonly used in tongue diagnosis analysis. K-means performs clustering analysis based on all data without being split into train and test.

    • CADxReport: Chest x-ray report generation using co-attention mechanism and reinforcement learning

      2022, Computers in Biology and Medicine
      Citation Excerpt :

      Earlier ARRG systems relied on manual feature extraction and sentence retrieval [10–12]. However, recent advancements in deep learning for abnormality detection [13–19], disease classification [20–22] and paragraph generation [23,24] have eliminated the need for human intervention. In the recent studies, various convolutional neural networks (CNNs) are used for feature extraction and various recurrent neural networks (RNNs), specifically long short term memory (LSTM) networks [25] and gated recurrent units (GRUs) [26], are used for word and paragraph generation.

    This work was support in part by the University of Macau (File no. MYRG2018-00053-FST), in part by the Open Research Fund of the Beijing Key Laboratory of Big Data Technology for Food Safety under Project BTBD-2021KF05, and in part by the Shenzhen Research Institute of Big Data.

    View full text