Skip to main content
SpringerLink
Log in

Multi-energy CT material decomposition using graph model improved CNN

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

In spectral CT imaging, the coefficient image of the basis material obtained by the material decomposition technique can estimate the tissue composition, and its accuracy directly affects the disease diagnosis. Although the precision of material decomposition is increased by employing convolutional neural networks (CNN), extracting the non-local features from the CT image is restricted using the traditional CNN convolution operator. A graph model built by multi-scale non-local self-similar patterns is introduced into multi-material decomposition (MMD). We proposed a novel MMD method based on graph edge–conditioned convolution U-net (GECCU-net) to enhance material image quality. The GECCU-net focuses on developing a multi-scale encoder. At the network coding stage, three paths are applied to capture comprehensive image features. The local and non-local feature aggregation (LNFA) blocks are designed to integrate the local and non-local features from different paths. The graph edge–conditioned convolution based on non-Euclidean space excavates the non-local features. A hybrid loss function is defined to accommodate multi-scale input images and avoid over-smoothing of results. The proposed network is compared quantitatively with base CNN models on the simulated and real datasets. The material images generated by GECCU-net have less noise and artifacts while retaining more information on tissue. The Structural SIMilarity (SSIM) of the obtained abdomen and chest water maps reaches 0.9976 and 0.9990, respectively, and the RMSE reduces to 0.1218 and 0.4903 g/cm3. The proposed method can improve MMD performance and has potential applications.

Graphical Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bhayana R, Parakh A, Kambadakone A (2020) Material decomposition with dual-and multi-energy computed tomography. MRS Commun 10:558–565. https://doi.org/10.1557/mrc.2020.86

    Article  CAS  Google Scholar 

  2. Franco PN, Spasiano CM, Maino C, De Ponti E, Ragusi M, Giandola T, Terrani S, Peroni M, Corso R, Ippolito D (2023) Principles and applications of dual-layer spectral CT in gastrointestinal imaging. Diagnostics 13:1740. https://doi.org/10.3390/diagnostics13101740

    Article  PubMed  PubMed Central  Google Scholar 

  3. Wang S, Cai A, Wu W, Zhang T, Liu F, Yu H (2023) IMD-MTFC: image-domain material decomposition via material-image tensor factorization and clustering for spectral CT. IEEE Trans Radiation Plasma Med Sci 7:382–393. https://doi.org/10.1109/TRPMS.2023.3234613

    Article  Google Scholar 

  4. Fang W, Wu DF, Kim K, Kalra MK, Singh R, Li L, Li QZ (2021) Iterative material decomposition for spectral CT using self-supervised Noise2Noise prior. Phys Med Biol 66. https://doi.org/10.1088/1361-6560/ac0afd

  5. Geng MF, Tian ZF, Jiang Z, You YF, Feng XM, Xia Y, Yang K, Ren QS, Meng XX, Maier A et al (2021) PMS-GAN: parallel multi-stream generative adversarial network for multi-material decomposition in spectral computed tomography. IEEE Trans Med Imaging 40:571–584. https://doi.org/10.1109/TMI.2020.3031617

    Article  PubMed  Google Scholar 

  6. Xue Y, Jiang YK, Yang CL, Lyu QH, Wang J, Luo C, Zhang LH, Desrosiers C, Feng K, Sun XN et al (2019) Accurate multi-material decomposition in dual-energy CT: a phantom study. IEEE Transactions Comput Imaging 5:515–529. https://doi.org/10.1109/TCI.2019.2909192

    Article  Google Scholar 

  7. Barber RF, Sidky EY, Schmidt TG, Pan XC (2016) An algorithm for constrained one-step inversion of spectral CT data. Phys Med Biol 61:3784–3818. https://doi.org/10.1088/0031-9155/61/10/3784

    Article  CAS  PubMed Central  Google Scholar 

  8. Feng M, Ji X, Zhang R, Treb K, Dingle AM, Li K (2021) An experimental method to correct low-frequency concentric artifacts in photon counting CT. Phys Med Biol 66:175011. https://doi.org/10.1088/1361-6560/ac1833

    Article  CAS  Google Scholar 

  9. Kim B, Shim H, Baek J (2022) A streak artifact reduction algorithm in sparse-view CT using a self-supervised neural representation. Med Phys 49:7497–7515. https://doi.org/10.1002/mp.15885

    Article  PubMed  Google Scholar 

  10. He Y, Zeng L, Xu Q, Wang Z, Yu H, Shen Z, Yang Z, Zhou R (2023) Spectral CT reconstruction via low-rank representation and structure preserving regularization. Phys Med Biol 68:025011. https://doi.org/10.1109/TMI.2020.2983414

  11. Ren L, Mccollough CH, Yu L (2018) Three-material decomposition in multi-energy CT: impact of prior information on noise and bias. In:SPIE, p 363–368. https://doi.org/10.1117/12.2294953

  12. Tao SZ, Rajendran K, Mccollough CH, Leng S (2018) Material decomposition with prior knowledge aware iterative denoising (MD-PKAID). Phys Med Biol 63. https://doi.org/10.1088/1361-6560/aadc90

  13. Zavala-Mondragon LA, Engel KJ, Menser B, Ruijters D, Van Der Sommen F (2021) Iterative reconstruction anti-correlated ROF model for noise reduction in dual-energy CBCT imaging. In: SPIE, p 661–670. https://doi.org/10.1117/12.2579500

  14. Wu WW, Yu HJ, Chen PJ, Luo FL, Liu FL, Wang Q, Zhu YN, Zhang YB, Feng J, Yu HY (2020) Dictionary learning based image-domain material decomposition for spectral CT. Phys Med Biol 65. https://doi.org/10.1088/1361-6560/aba7ce

  15. Niu SZ, Zhang Y, Zhong YC, Liu GL, Lu SH, Zhang XL, Hu SZ, Wang TH, Yu GH, Wang J (2018) Iterative reconstruction for photon-counting CT using prior image constrained total generalized variation. Comput Biol Med 103:167–182. https://doi.org/10.1016/j.compbiomed.2018.10.022

    Article  PubMed  PubMed Central  Google Scholar 

  16. Yang QS, Yan PK, Zhang YB, Yu HY, Shi YY, Mou XQ, Kalra MK, Zhang Y, Sun L, Wang G (2018) Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE Trans Med Imaging 37:1348–1357. https://doi.org/10.1109/TMI.2018.2827462

    Article  PubMed  PubMed Central  Google Scholar 

  17. Shi ZF, Wang N, Kong FN, Cao HS, Cao QJ (2022) A semi-supervised learning method of latent features based on convolutional neural networks for CT metal artifact reduction. Med Phys 49:3845–3859. https://doi.org/10.1002/mp.15633

    Article  PubMed  Google Scholar 

  18. Yu LQ, Zhang ZC, Li XM, Xing L (2021) Deep sinogram completion with image prior for metal artifact reduction in CT images. IEEE Trans Med Imaging 40:228–238. https://doi.org/10.1109/TMI.2020.3025064

    Article  PubMed  Google Scholar 

  19. You CY, Li G, Zhang Y, Zhang XL, Shan HM, Li MZ, Ju SH, Zhao Z, Zhang ZY, Cong WX et al (2020) CT super-resolution GAN constrained by the identical, residual, and cycle learning ensemble (GAN-CIRCLE). IEEE Trans Med Imaging 39:188–203. https://doi.org/10.1109/TMI.2019.2922960

    Article  PubMed  Google Scholar 

  20. Zhang SN, Zhao ZQ, Qiu L, Liang D, Wang K, Xu J, Zhao J, Sun JQ (2023) Automatic vertebral fracture and three-column injury diagnosis with fracture visualization by a multi-scale attention-guided network. Med Biol Eng Compu 61:1661–1674. https://doi.org/10.1007/s11517-023-02805-2

    Article  Google Scholar 

  21. Apostolopoulos ID, Pintelas EG, Livieris IE, Apostolopoulos DJ, Papathanasiou ND, Pintelas PE, Panayiotakis GS (2021) Automatic classification of solitary pulmonary nodules in PET/CT imaging employing transfer learning techniques. Med Biol Eng Compu 59:1299–1310. https://doi.org/10.1007/s11517-021-02378-y

    Article  Google Scholar 

  22. An MJ, Li JH, Xu XY, Schoepf UJ, Savage RH, Cao KL, Song Q, Wang ZY, Liu Z, Li YW et al (2023) A deep learning-based fully automatic and clinical-ready framework for regional myocardial segmentation and myocardial ischemia evaluation. Med Biol Eng Compu. https://doi.org/10.1007/s11517-023-02798-y

    Article  Google Scholar 

  23. Hong ZF, Chen MZ, Hu WJ, Yan SY, Qu AP, Chen LN, Chen JX (2023) Dual encoder network with transformer-CNN for multi-organ segmentation. Med Biol Eng Compu 61:661–671. https://doi.org/10.1007/s11517-022-02723-9

    Article  Google Scholar 

  24. Kawahara D, Saito A, Ozawa S, Nagata Y (2021) Image synthesis with deep convolutional generative adversarial networks for material decomposition in dual-energy CT from a kilovoltage CT. Comput Biol Med 128. https://doi.org/10.1016/j.compbiomed.2020.104111

  25. Wang GS, Liu Z, Huang ZY, Zhang N, Luo HH, Liu LJ, Shen H, Che CW, Niu TY, Liang D et al. (2022) Improved GAN: using a transformer module generator approach for material decomposition. Comput Biol Med 149. https://doi.org/10.1016/j.compbiomed.2022.105952

  26. Clark DP, Holbrook M, Badea CT (2018) Multi-energy CT decomposition using convolutional neural networks. In: Medical Imaging 2018: Phys Med Imaging https://doi.org/10.1117/12.2293728

  27. Wu XC, He P, Long ZR, Guo XD, Chen MY, Ren XZ, Chen PJ, Deng LZ, An K, Li PC et al (2019) Multi-material decomposition of spectral CT images via Fully Convolutional DenseNets. J X-Ray Sci Technol 27:461–471. https://doi.org/10.3233/XST-190500

    Article  Google Scholar 

  28. Gong H, Tao SZ, Rajendran K, Zhou W, Mccollough CH, Leng S (2020) Deep-learning-based direct inversion for material decomposition. Med Phys 47:6294–6309. https://doi.org/10.1002/mp.14523

    Article  CAS  PubMed  Google Scholar 

  29. Salehjahromi M, Zhang Y, Yu H (2017) A spectral CT denoising algorithm based on weighted block matching 3D filtering. In:SPIE, p 65–76. https://doi.org/10.1117/12.2273213

  30. Zhang Y, Salehjahromi M, Yu H (2019) Tensor decomposition and non-local means based spectral CT image denoising. J Xray Sci Technol 27:397–416. https://doi.org/10.3233/XST-180413

    Article  PubMed  PubMed Central  Google Scholar 

  31. Bruna J, Zaremba W, Szlam A, Lecun Y (2013) Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203. https://doi.org/10.48550/arXiv.1312.6203

  32. Ding Y, Zhang Z, Zhao X, Hong D, Cai W, Yu C, Yang N, Cai W (2022) Multi-feature fusion: graph neural network and CNN combining for hyperspectral image classification. Neurocomputing 501:246–257. https://doi.org/10.1016/j.neucom.2022.06.031

    Article  Google Scholar 

  33. Xuan P, Wu X, Cui H, Jin Q, Wang L, Zhang T, Nakaguchi T, Duh HBL (2023) Multi-scale random walk driven adaptive graph neural network with dual-head neighboring node attention for CT segmentation. Appl Soft Comput 133:109905. https://doi.org/10.1016/j.asoc.2022.109905

  34. Gürler Z, Gharsallaoui MA, Rekik I, Alzheimer’s Dis Neuroimaging I (2023) Template-based graph registration network for boosting the diagnosis of brain connectivity disorders. Computer Med Imaging Graph 103. https://doi.org/10.1016/j.compmedimag.2022.102140

  35. Zhou Y, Zheng HX, Huang X, Hao SF, Li DA, Zhao JM (2022) Graph neural networks: taxonomy, advances, and trends. ACM Trans Intell Syst Technol 13. https://doi.org/10.1145/3495161

  36. He YJ, Zhao H, Wong STC (2021) Deep learning powers cancer diagnosis in digital pathology. Computer Med Imaging Graph 88. https://doi.org/10.1016/j.compmedimag.2020.101820

  37. Valsesia D, Fracastoro G, Magli E (2020) Deep graph-convolutional image denoising. IEEE Trans Image Process 29:8226–8237. https://doi.org/10.1109/TIP.2020.3013166

    Article  ADS  MathSciNet  Google Scholar 

  38. Chen K, Pu X, Ren Y, Qiu H, Li H, Sun J (2020) Low-dose CT image blind denoising with graph convolutional networks. In: Kwok JT, Chan JH, King I (eds) Yang H, Pasupa K, Leung AC-S. Neural information processing. Springer International Publishing, Cham, pp 423–435

    Google Scholar 

  39. Shi ZF, Li JZ, Li HL, Hu QX, Cao QJ (2019) A virtual monochromatic imaging method for spectral CT based on Wasserstein generative adversarial network with a hybrid loss. IEEE Access 7:110992–111011. https://doi.org/10.1109/ACCESS.2019.2934508

    Article  Google Scholar 

  40. Shi ZF, Li HL, Cao QJ, Wang ZQ, Cheng M (2021) A material decomposition method for dual-energy CT via dual interactive Wasserstein generative adversarial networks. Med Phys 48:2891–2905. https://doi.org/10.1002/mp.14828

    Article  PubMed  Google Scholar 

  41. Simonovsky M, Komodakis N, Ieee (2017) Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: 30TH IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017). p 29–38

  42. Chen GH, Tang J, Leng SH (2008) Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets. Med Phys 35:660–663. https://doi.org/10.1118/1.2836423

    Article  PubMed  Google Scholar 

  43. Segars WP, Sturgeon G, Mendonca S, Grimes J, Tsui BMW (2010) 4D XCAT phantom for multimodality imaging research. Med Phys 37:4902–4915. https://doi.org/10.1118/1.3480985

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Shi ZF, Yang HY, Cong WX, Wang G (2016) An edge-on charge-transfer design for energy-resolved x-ray detection. PHYSICS IN MEDICINE AND BIOLOGY 61:4183–4200. https://doi.org/10.1088/0031-9155/61/11/4183

    Article  ADS  Google Scholar 

  45. Walsh MF, Nik SJ, Procz S, Pichotka M, Bell ST, Bateman CJ, Doesburg RMN, De Ruiter N, Chernoglazov AI, Panta RK et al. (2013) Spectral CT data acquisition with Medipix3.1. J Instrument 8. https://doi.org/10.1088/1748-0221/8/10/P10012

  46. Chandra TB, Verma K (2020) Analysis of quantum noise-reducing filters on chest X-ray images: a review. Measurement 153:107426. https://doi.org/10.1016/j.measurement.2019.107426

  47. Zhang L, Zhang L, Mou XQ, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20:2378–2386. https://doi.org/10.1109/TIP.2011.2109730

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  48. Kingma D P, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. https://doi.org/10.48550/arXiv.1412.6980

Download references

Acknowledgements

The authors would like to thank Prof. Segars from Duke University for providing the phantom datasets, and we are also grateful to Prof. Anthony Butler and Hannah Prebble from MARS Bioimaging Ltd. for sharing the mouse datasets.

Funding

This work was funded by the National Natural Science Foundation of China through grant 62071326.

Author information

Authors and Affiliations

  1. School of Microelectronics, Tianjin University, Tianjin, 300072, China

    Zaifeng Shi, Fanning Kong, Ming Cheng, Huaisheng Cao & Shunxin Ouyang

  2. Tianjin Key Laboratory of Imaging and Sensing Microelectronic Technology, Tianjin, China

    Zaifeng Shi

  3. School of Mathematical Sciences, Tianjin Normal University, Tianjin, 300387, China

    Qingjie Cao

Authors
  1. Zaifeng Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fanning Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huaisheng Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shunxin Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qingjie Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zaifeng Shi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Z., Kong, F., Cheng, M. et al. Multi-energy CT material decomposition using graph model improved CNN. Med Biol Eng Comput 62, 1213–1228 (2024). https://doi.org/10.1007/s11517-023-02986-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02986-w

Keywords

Search

Navigation