Two-phase non-invasive multi-disease detection via sublingual region☆
Introduction
The sublingual vein is a pair of veins located on the lower surface of the tongue lying on either side of the lingual frenulum. This is critical evidence for disease detection in traditional Chinese medicine (TCM) [36]. Studies show that blood stasis on the sublingual vein has a close relationship with health status in humans [3,5,35,42,43,49]. The sublingual vein of a normal person usually appears to be dark red with no vasodilation. If the vascular structure is not clear or is present in a faint color, there is a higher probability that some abnormal health status exists. In addition, since the sublingual vein is connected to the viscera, it can reflect the status of viscera-related diseases. By applying computerised disease diagnosis methods [8,13,37], disease detection can be performed according to information from multiple modalities, such as image, sound, and odour [6,17,18,44,47].
Recently, researchers have frequently focused on non-invasive computerized disease detection methods [3,8,17,30,44,45,47]. Non-invasive disease detection methods do not require the extraction of bodily fluids from patients and are a pain-free diagnosis method. In contrast, conventional diagnosis methods, such as diabetes mellitus diagnosis, require a blood test, causing patients pain and discomfort. One popular and commonly used non-invasive detection method is computerised tongue diagnosis, which originated from TCM [8,36,37] and believes that the appearance of the human tongue reflects the health status of the body. Extensive studies have focused on the anterior part of the tongue [24,37,38,46,50], while there are few studies on the sublingual vein (underside of the tongue). As mentioned in Ref. [5], the sublingual vessels connect directly with the internal organs, including the heart, liver, and kidneys. In addition, the sublingual vein is sensitive to blood stasis and can reflect the activity of blood circulation. Therefore, the sublingual vein can be a disease indicator for internal organs (e.g., kidney disease, gastritis, high blood pressure). Since the sublingual collaterals stem from the tongue and can be effectively used in computerised methods for detecting diabetes mellitus [17,44], it is possible to detect them from the sublingual vein [35]. found that the color of the sublingual vein has a positive relationship with blood stasis, which means computerised automatic disease detection is feasible via sublingual vein images [3,5,35]. performed segmentation based on blood stasis for disease detection. The aforementioned methods have four drawbacks and limitations: 1) pixel-wise segmentation on the sublingual vein is ineffective in some circumstances, since the sublingual vein can be faint or a pale color (at times), which is challenging to extract as an image; 2) these methods are designed for binary disease detection, rather than multi-disease detection; 3) there is no unified framework for multi-disease detection, including sublingual vein extraction, feature representation, and classification; and 4) most of the methods did not focus on investigating the effectiveness of the sublingual vein in disease detection.
In this paper, a two-phase sublingual-based disease detection framework was proposed for non-invasive multi-disease detection. In the first phase, the sublingual vein region was segmented on the raw image, which was captured using a non-invasive imaging device. In the second phase, features were extracted from the sublingual vein region, such as color, texture, geometry, and latent representation. Subsequently, a multi-class classification was performed on the features extracted from the sublingual vein region. To segment the sublingual vein region, Yolov3 architecture [29] was applied to first detect the bounding box, which is the area covering the sublingual vein with the highest probability. To better represent the characterisation of the obtained sublingual vein region, multiple feature representations were generated: color, texture, shape, and latent representation. For the color feature, a color histogram [26] and color moments [34]were calculated. The texture feature was extracted using LBP [22] and Gabor filters [9]. Regarding the geometry feature, nine descriptors were defined: length, width, length-width ratio, area, smaller half-distance, circle area, circle area ratio, square area, and square area ratio. For the latent representation, the stacked sparse autoencoder (SSAE) [14], which is widely used to extract latent representations from an image [41] was applied. Each group of features was first tested individually in the experiments by applying a quantitative evaluation. Finally, the optimal multi-feature representation was determined for multi-disease detection. Overall, there were four main contributions of this paper:
- 1)
To the best of our knowledge, this is the first time such a study on the sublingual vein for non-invasive multi-disease detection has been established using a quantitative method.
- 2)
A two-phase sublingual-based disease detection framework was proposed for effective non-invasive multi-disease detection.
- 3)
Multiple feature representations based on the sublingual vein region were generated, and the optimal multi-feature representation was obtained empirically.
- 4)
The experiments demonstrated that the proposed method was highly effective for disease detection.
The remaining parts of the paper are organised as follows: In section 2 related works and details of the sublingual vein image are first introduced. This is followed by a presentation of the methodology of the two-phase non-invasive multi-disease detection method using the sublingual vein region in section 3. In section 4, a quality evaluation, quantitative evaluation, as well as discussion of the proposed method, were performed before discussing the experiments in detail. Finally, a conclusion is reached in section 5.
Section snippets
Non-invasive disease detection
Non-invasive disease detection methods perform diagnosis based on the appearance of the human body without extracting bodily fluids [44]. These methods are based on the presumption that changes in the health status of internal organs will cause different reflections on the appearance of the human body [47]. Zhang et al. [44] analysed the tongue images of patients to detect diabetes mellitus and showed the feasibility of disease detection using the human tongue [47]. applied the image blocks
Two-phase non-invasive multi-disease detection via the sublingual vein region: overview
A two-phase disease detection framework was proposed based on the sublingual vein images, as shown in section 2. The overall workflow of the framework is depicted in Fig. 3. The framework performs disease detection using several steps. First, the sublingual vein region from any given image is segmented with a trained sublingual vein region detector (section 3.2). The trained sublingual vein region detector segments the region with the highest probability covering the most pixels in the
Experimental settings
Sublingual vein images were collected from patients in different health status categories, namely chronic kidney disease, diabetes mellitus, high blood pressure, and chronic gastritis. In addition, a healthy control group was added. The distribution of samples in these categories is presented in Table 1 (the 200 images for fine-tuning the segmenter are not included). All images were captured at the Guangdong Provincial Hospital of Traditional Chinese Medicine. The label for each image was
Conclusion
In this paper, a two-phase non-invasive multi-disease detection method was proposed using the sublingual vein region. In the first phase, the sublingual vein region was segmented from its raw image. In the second phase, a multi-feature representation was generated based on the sublingual vein region. To perform multi-disease detection, KNN, SVM, ProCRC, LDA, SRC, CRC, Random Forest, and RSLDA were applied. Our proposed method achieved 98.05% accuracy (precision: 97.76%, recall: 97.27%, and
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (50)
- et al.
Objective auscultation for traditional Chinese medical diagnosis using novel acoustic parameters
Comput. Methods Progr. Biomed.
(2000) - et al.
Objective assessment of blood stasis using computerized inspection of sublingual veins
Comput. Methods Progr. Biomed.
(2002) - et al.
How to standardize the pulse-taking method of traditional Chinese medicine pulse diagnosis
Comput. Biol. Med.
(2013) - et al.
Automated tongue diagnosis on the smartphone and its applications
Comput. Methods Progr. Biomed.
(2019) - et al.
Body surface feature-based multi-modal learning for diabetes mellitus detection
Inf. Sci.
(2019) - et al.
Joint discriminative and collaborative representation for fatty liver disease diagnosis
Expert Syst. Appl.
(2017) - et al.
Sublingual vein extraction algorithm based on hyperspectral tongue imaging technology
Comput. Med. Imag. Graph.
(2011) - et al.
An extensive analysis of various texture feature extractors to detect diabetes mellitus using facial specific regions
Comput. Biol. Med.
(2017) - et al.
An improved noninvasive method to detect diabetes mellitus using the probabilistic collaborative representation based classifier
Inf. Sci.
(2018) - et al.
Traditional Chinese medicine
Lancet
(2008)
A high quality color imaging system for computerized tongue image analysis
Expert Syst. Appl.
Computerized feature quantification of sublingual veins from color sublingual images
Comput. Methods Progr. Biomed.
Computational traditional Chinese medicine diagnosis: a literature survey
Comput. Biol. Med.
Classification of hyperspectral medical tongue images for tongue diagnosis
Comput. Med. Imag. Graph.
An introduction to kernel and nearest-neighbor nonparametric regression
Am. Statistician
A probabilistic collaborative representation based approach for pattern classification
Computerized analysis of tongue sub-lingual veins to detect lung and breast cancers
Support-vector networks
Mach. Learn.
Medical Biometrics: Computerized TCM Data Analysis
Gabor Analysis and Algorithms: Theory and Applications
Theoretical Foundations of Chinese-medicine-systems of Correspondence-Porkert
Random decision forests
Atlas of Chinese Tongue Dignosis
Nonlinear principal component analysis using autoassociative neural networks
AIChE J.
Imagenet classification with deep convolutional neural networks
Adv. Neural Inf. Process. Syst.
Cited by (5)
A review of traditional Chinese medicine diagnosis using machine learning: Inspection, auscultation-olfaction, inquiry, and palpation
2024, Computers in Biology and MedicineMissing-view completion for fatty liver disease detection
2022, Computers in Biology and MedicineCitation Excerpt :To segment the sublingual vein region, we implemented a deep learning model, i.e., Yolov3. This model can use global information for performing detection inference [33,34]. Specifically, it was provided with a more precise detection box prediction for the sublingual vein region, while overlooking the useless parts of the input image.
A multi-step approach for tongue image classification in patients with diabetes
2022, Computers in Biology and MedicineCitation Excerpt :We compare the clustering results of various features, including TDAS features, VQ-VAE features, LBP features, and HOG features. The LBP and HOG features are calculated by the Local Binary Pattern algorithm [43,55] and the Histogram of Oriented Gradient algorithm [56,57], commonly used in tongue diagnosis analysis. K-means performs clustering analysis based on all data without being split into train and test.
CADxReport: Chest x-ray report generation using co-attention mechanism and reinforcement learning
2022, Computers in Biology and MedicineCitation Excerpt :Earlier ARRG systems relied on manual feature extraction and sentence retrieval [10–12]. However, recent advancements in deep learning for abnormality detection [13–19], disease classification [20–22] and paragraph generation [23,24] have eliminated the need for human intervention. In the recent studies, various convolutional neural networks (CNNs) are used for feature extraction and various recurrent neural networks (RNNs), specifically long short term memory (LSTM) networks [25] and gated recurrent units (GRUs) [26], are used for word and paragraph generation.
- ☆
This work was support in part by the University of Macau (File no. MYRG2018-00053-FST), in part by the Open Research Fund of the Beijing Key Laboratory of Big Data Technology for Food Safety under Project BTBD-2021KF05, and in part by the Shenzhen Research Institute of Big Data.