Skip to main content
SpringerLink
Log in

Three feature streams based on a convolutional neural network for early esophageal cancer identification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Esophageal cancer is the eighth most common cancer in the world. Currently, the incidence of esophageal cancer is increasing annually. Early detection of esophageal cancer can significantly improve the prognosis and quality of life of patients. However, detection of esophageal tumors remains a challenge because it depends on the experience, expertise and skills of the endoscopy physician. Early esophageal cancer can be easily misdiagnosed because the lesions are not obvious. Therefore, we propose a deep learning method based on a convolutional neural network (CNN) model for automatically classifying early esophageal cancer. We first process the color esophageal images by using grayscale processing, Meijering filtering and hybrid Hessian filtering before entering the network. Then, we input the processed images into the corresponding streams for feature extraction and conduct feature fusion through the add layer. Finally, the fusion features are classified and detected. We train and test the three-stream–esophageal cancer classification network (TS-ECCN) using 635 images. Macroaveraging and microaveraging, which are more suitable for measuring multiple classification problems, are used as statistical indicators to evaluate the accuracy of the model diagnosis. The areas under the receiver operating characteristic (ROC) curves (AUCs) were 0.97, 0.99 and 0.98 for classifying early cancer, cancer and normal tissue, respectively. The macroprecision, macrorecall and macro-F score of the TS-ECCN were 88.7%, 86% and 87.1%, respectively. The microprecision, microrecall and micro-F score were 89.4%. We validated the ability of a CNN-based algorithm to automatically classify early esophageal cancer from endoscopic images. This algorithm will greatly reduce the workload of endoscopists, improve the efficiency and accuracy of examinations, and become an effective means of clinical auxiliary diagnosis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Alagappan M, Brown JRG, Mori Y, Berzin TM (2018) Artificial intelligence in gastrointestinal endoscopy : the future is almost here. World J Gastrointestinal Endoscopy 10(10):239–249

    Article  Google Scholar 

  2. Balasubramanian M (2002) The isomap algorithm and topological stability. Science 295(5552):7a–77a

  3. Bang C, Cho B, Baik G (2020) Automated classification of gastric neoplasms in endoscopic images using a convolutional neural network. c Georg Thieme Verlag KG 52:eP295 1

  4. Bo L, Ren X, Fox D (2010) Kernel descriptors for visual recognition[C]//Adv Neural Inf Process Systems. 244–252

  5. Bochkovskiy A, Wang CY, Liao HYM (2020) Yolov4: Optimal speed and accuracy of object detection. [J]. arXiv preprint arXiv:2004.10934

  6. Boeriu A (2015) Narrow-band imaging with magnifying endoscopy for the evaluation of gastrointestinal lesions. World J Gastrointestinal Endoscopy 7(2):110

    Article  Google Scholar 

  7. Borgli H, Thambawita V, Smedsrud PH, Hicks SA, Lange TD Hyperkvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci Data 7(1):1–14

  8. Chen Y, Xie X, Zhang T, Bai J, Hou M (2020) A deep residual compensation extreme learning machine and applications. J Forecast 39(6):986–999

    Article  Google Scholar 

  9. CMPS P, DTHD M, RBP A, Guedes HG, Kumbhari V, EGHd M (2017) Lugol’s iodine chromoendoscopy versus narrow band image enhanced endoscopy for the detection of esophageal cancer in patients with stenosis secondary to caustic/corrosive agent ingestion. Arquivos de gastroenterologia 54(3):250–254

    Article  Google Scholar 

  10. Dayan P, Abbott L (2001) Theoretical neuroscience: computational and mathematical modeling of neural systems. Philos Psychol 15(1):154–155

    MATH  Google Scholar 

  11. Ding Z, Shi H, Zhang H, Meng L, Hou X (2019) Gastroenterologist-level identifcation of small-bowel diseases and normal variants by capsule endoscopy using a deep-learning model. Gastroenterology 157(4)

  12. Du X, Li Y, Yao J, Chen B, Yang X (2018) Loid-eec: Localizing and identifying early esophageal cancer based on deep learning in screening chromoendoscopy. in: the 2018 the 2nd International Conference, pp.17–22

  13. Duchi J, Hazan E, Singer Y (n.d.) Adaptive subgradient methods for online learning and stochastic optimization. Vol. 12

  14. Frangi RF, Niessen WJ, Vincken KL, Viergever MA (1998) Multiscale vessel enhancement filtering. Lect Notes Comput Sci 1496:130–137

    Article  Google Scholar 

  15. Godbole S, Sarawagi S (2004) Discriminative methods for multi-labeled classification :22–30

  16. Guo LJ, Xiao X, Wu CC, Zeng X, Zhang Y, Du NPJ, Shuai Bai NP, Jia Xie NP, Zhiwei Zhang MS, Li BS a Y (2020) Real- time automated diagnosis of precancerous lesions and early esophageal squamous cell carcinoma using a deep learning model (with videos). Gastrointest Endosc 91(1):41–51

    Article  Google Scholar 

  17. Hai X, Haijun L, Xianlu Z, Yejun H, Guozhen C, Ahmed E, Guanghui Y, Jiantao W, Guoming Z, Baiying L, Amd-gan (2020) Attention encoder and multi-branch structure based generative adversarial networks for fundus disease detection from scanning laser ophthalmoscopy images. Neural Netw 132:477–490

    Article  Google Scholar 

  18. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. in: IEEE Conference on Computer Vision & Pattern Recognition, pp.770–778

  19. Hinton G, Roweis S (2003) Stochastic neighbor embedding. Adv Neural Inf Proces Syst 15(4):833–840

    Google Scholar 

  20. Hirasawa T, Aoyama K, Tanimoto T, Ishihara S, Tada T (2018) Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer Official J Int Gastric Cancer Assoc Japanese Gastric Cancer Assoc 87(Suppl 1):1–8

    Google Scholar 

  21. Huang G, Liu Z, Maaten LVD, Weinberger KQ (2017) Densely connected convolutional networks. in: CVPR, pp. 4700–4708

  22. Kandiah K, Chedgy FJQ, Subramaniam S, Thayalasekaran S, Handari PB (2017) Early squamous neoplasia of the esophagus: The endoscopic approach to diagnosis and management. Saudi J Gastroenterology Official J Saudi Gastroenterology Assoc 23(2):75–81

    Google Scholar 

  23. Kazuhiro G, Takashi O, Masahiro Y, Nagaaki O, Hirohisa M (2004) Appearance of enhanced tissue features in narrow- band endoscopic imaging. J Biomed Opt 9(3):568–578

    Article  Google Scholar 

  24. Kingma DP, Ba J (2022) Adam: A method for stochastic optimization. arXiv

  25. Kruskal JB, Wish M, Uslaner EM (1978) Multidimensional scaling. Book on demand pod

  26. Laurens VDM, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9(2605):2579–2605

    MATH  Google Scholar 

  27. Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp.2117–2125

  28. Liu DY, Gan T, Rao NN, Xing YW, Zheng J, Li S, Luo CS, Zhou ZJ, Wan YL (2016) Identification of lesion images from gastrointestinal endoscope based on feature extraction of combinational methods with and without learning process. Med Image Anal 32:281–294

    Article  Google Scholar 

  29. Meijering E, Jacob M, Sarria JXCF, Steiner P, Hirling H, Unser M (2002) Design and validation of a tool for neurite tracing and analysis in fluorescence microscopy images. Cytometry Part A 58(2):167–176

    Google Scholar 

  30. Moenig S, Chevallay M, Niclauss N, Zilli T, Fang W, Bansal A, Hoeppner J (2018) Early esophageal cancer: the significance of surgery, endoscopy, and chemoradiation. Ann N Y Acad Sci 1434:115–123

    Article  Google Scholar 

  31. Nelder JA, Mead R (1965) A simplex method for function minimization comput. Comput J, (4):4

  32. Ng CC, Yap MH, Costen N, Li B (2014) Automatic wrinkle detection using hybrid hessian filter. in: Asian Conference on Computer Vision, pp.609–622

  33. Njei B, Mccarty TR, Birk JW (2016) Trends in esophageal cancer survival in United States adults from 1973 to 2009: a seer database analysis. J Gastroenterol Hepatol 31(6):1141–1146

    Article  Google Scholar 

  34. Redmon J, Farhadi A (n.d.) Yolov3: An incremental improvement,.arXiv e-prints

  35. Sabri B, Fethi J, Mohammed EA, Quan Z (2017) Optimal classifier for imbalanced data using matthews correlation coefficient metric. PLoS One 12(6):e0177678

    Article  Google Scholar 

  36. Saito H, Aoki T, Mmath KA, Bsc YK, Tsuboi A, Yamada A, Fujishiro M, Oka S, Ishihara S, Matsuda T a (2020) Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 92(1):144–151

    Article  Google Scholar 

  37. Sebastiani F (2001) Machine learning in automated text categorization. ACM Comput Surv 34:1–47

    Article  Google Scholar 

  38. Shanno DF (1970) Conditioning of quasi-newton methods for function minimization. Math Comput 24(111):647–656

    Article  MathSciNet  Google Scholar 

  39. Simonyan K, Zisserman A. (n.d.) Very deep convolutional networks for large-scale image recognition. Computer Sci

  40. Sofka M, Stewart, Charles V (2007) Erratum to “retinal vessel centerline extraction using multiscale matched filters, confidence and edge measures”. IEEE Trans Med Imaging 26(1):133–133

    Article  Google Scholar 

  41. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp.2818–2826

  42. Thambawita V, Jha D, Riegler M, Halvorsen P, Hammer HL, Johansen HD, Johansen D (2018) The medico-task 2018: Disease detection in the gastrointestinal tract using global features and deep learning in: MediaEval 2018, 2018

  43. Van Der Sommen F, Zinger S, Schoon EJ, De With P (2014) Supportive automatic annotation of early esophageal cancer using local gabor and color features. Neurocomputing 144:92–106

    Article  Google Scholar 

  44. Van Riel S, Van Der Sommen F, Zinger S, Schoon EJ, de With PH (2018) Automatic detection of early esophageal cancer with cnns using transfer learning. in: 2018 25th IEEE International Conference on Image Processing (ICIP), IEEE, pp.1383–1387

  45. Wang Z, Xiao Y, Li Y, Zhang J, Lu F, Hou M, Liu X (2020) Automatically discriminating and localizing covid-19 from community-acquired pneumonia on chest x-rays. Pattern Recogn 110:107613

    Article  Google Scholar 

  46. Wang Z, Xiao Y, Weng F, Li X, Meng Y (n.d.) Rjaunlab: Automatic multi-class recognition of jaundice on photos of subjects with region annotation networks. J Digit Imaging (9)

  47. Wang Z, Meng Y, Weng F, Chen Y, Lu F, Liu X, Hou M, Zhang J (n.d.) An effective cnn method for fully automated segmenting subcutaneous and visceral adipose tissue on ct scans. Annals BiomedEng, 48(5)

  48. Wani S, Drahos J, Cook MB, Rastogi A, Bansal A, Yen R, Sharma P, Das A (2014) Comparison of endoscopic therapies and surgical resection in patients with early esophageal cancer: a population-based study. Gastrointest Endosc 79(2):224–232

    Article  Google Scholar 

  49. Whiteman DC (2014) Esophageal cancer: priorities for prevention. Current Epidemiology Reports 1(3):138–148

    Article  MathSciNet  Google Scholar 

  50. Wu X, Chen H, Gan T, Chen J, Ngo CW, Peng Q (2016) Automatic hookworm detection in wireless capsule endoscopy images. IEEE Trans Med Imaging 35(7):1741–1752

  51. Yamaguchi J, Yoneyama A, Minamoto T (2015) Automatic detection of early esophageal cancer from endoscope image using fractal dimension and discrete wavelet transform. in: 2015 12th International Conference on Information Technology-New Generations, IEEE, pp. 317–322

  52. Yuan Y, Meng MQ (2017) Deep learning for polyp recognitionin wireless capsule endoscopy images. Med Phys 44(4):1379–1389

  53. Zhang SW, Zheng RS, Zuo TT, Zeng HM, He J (2016) [mortality and survival analysis of esophageal cancer in china]., Zhonghua Zhong Liu Za Zhi, 38(9)709–715

Download references

Acknowledgments

This work was supported by Scientific Research Fund of Hunan Provincial Education Department(grant number 20C0402), Hunan First Normal University(grant number XYS16N03) and the Projects of the National Social Science Foundation of China (grant number 82073019 and 82073018).

Author information

Author notes

  1. Zheng Wang and Zhaoying Li contributed equally to this work.

Authors and Affiliations

  1. School of Mathematics and Statistics, Central South University, Changsha, 410083, China

    Zheng Wang, Zhaoying Li & Muzhou Hou

  2. School of Computer Science, Hunan First Normal University, Changsha, 410205, China

    Zheng Wang

  3. Department of Gastroenterology of Xiangya Hospital, Central South University, Changsha, 410008, China

    Ying Xiao, Xiaowei Liu & Shuijiao Chen

  4. Hunan International Scientific and Technological Cooperation Base of Artificial Intelligence Computer-Aided Diagnosis and Treatment for Digestive Disease, Changsha, 410008, China

    Ying Xiao, Xiaowei Liu & Shuijiao Chen

Authors
  1. Zheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhaoying Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaowei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Muzhou Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shuijiao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Muzhou Hou or Shuijiao Chen.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Li, Z., Xiao, Y. et al. Three feature streams based on a convolutional neural network for early esophageal cancer identification. Multimed Tools Appl 81, 38001–38018 (2022). https://doi.org/10.1007/s11042-022-13135-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13135-0

Keywords

Search

Navigation