Skip to main content

Deep Learning Methods for Image Guidance in Radiation Therapy

  • Conference paper
  • First Online:
Artificial Neural Networks in Pattern Recognition (ANNPR 2020)

Abstract

Image guidance became one of the most important key technologies in radiation therapy in the last two decades. Nowadays medical images play a key role in virtually every aspect of the common treatment workflows. Advances in imaging hardware and algorithmic processing are enablers for substantial treatment quality improvements like online adaptation of the treatment, accounting for anatomical changes of the day, or intra-fraction motion monitoring and organ position verification during treatment. Going through this rapid development, an important observation is that further improvements of various methods heavily rely on model knowledge. In a classical sense such model knowledge is for example provided by mathematically formulated physical assumptions to ill-posed problems or by expert systems and heuristics. Recently, it became evident that in various applications such classical approaches get outperformed by data driven machine learning methods. Especially worth to mention is that this not only holds true in terms of precision and computational performance but also in terms of complexity reduction and maintainability. In this paper we provide an overview about the different stages in the X-ray based imaging pipeline in radiation therapy where machine learning based algorithms show promising results or are already applied in clinical routine.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adamson, P.M., Arrate, F., Jordan, P.: Evaluation of abdominal autosegmentation versus inter-observer variability on a high-speed ring gantry CBCT system. In: AAPM Annual Meeting, San Antonio, TX (2019)

    Google Scholar 

  2. Andersen, A., Kak, A.: Simultaneous algebraic reconstruction technique (SART): a superior implementation of the ART algorithm. Ultrason. Imaging 6(1), 81–94 (1984)

    Google Scholar 

  3. Bae, K.T., Giger, M.L., Chen, C.T., Kahn Jr., C.E.: Automatic segmentation of liver structure in CT images (1993)

    Google Scholar 

  4. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019). https://doi.org/10.1109/tmi.2019.2897538

    Article  Google Scholar 

  5. Benedict, S.H., et al.: Stereotactic body radiation therapy: the report of AAPM Task Group 101. Med. Phys. 37(8), 4078–4101 (2010)

    Google Scholar 

  6. Bertholet, J., Knopf, A., et al.: Real-time intrafraction motion monitoring in external beam radiotherapy. Phys. Med. Biol. 64(15), 15TR01 (2019)

    Google Scholar 

  7. Brehm, M., Paysan, P., Oelhafen, M., Kachelrieß, M.: Artifact-resistant motion estimation with a patient-specific artifact model for motion-compensated cone-beam CT. Med. Phys. 40(10), 101913 (2013)

    Google Scholar 

  8. Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24(4), 325–376 (1992)

    Google Scholar 

  9. Cai, J., Xia, Y., Yang, D., Xu, D., Yang, L., Roth, H.: End-to-end adversarial shape learning for abdomen organ deep segmentation. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) MLMI 2019. LNCS, vol. 11861, pp. 124–132. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32692-0_15

    Chapter  Google Scholar 

  10. Chen, B., Xiang, K., Gong, Z., Wang, J., Tan, S.: Statistical iterative CBCT reconstruction based on neural network. IEEE Trans. Med. Imaging 37(6), 1511–1521 (2018)

    Google Scholar 

  11. Chen, G., et al.: AirNet: fused analytical and iterative reconstruction with deep neural network regularization for sparse-data CT. Med. Phys. (2020). https://doi.org/10.1002/mp.14170

  12. Chen, G., Zhao, Y., Huang, Q., Gao, H.: 4D-AirNet: a temporally-resolved CBCT slice reconstruction method synergizing analytical and iterative method with deep learning. Phys. Med. Biol. (2020). https://doi.org/10.1088/1361-6560/ab9f60

  13. Chun, I., Huang, Z., Lim, H., Fessler, J.: Momentum-Net: fast and convergent iterative neural network for inverse problems. arXiv preprint arXiv:1907.11818, July 2019

  14. Czeizler, E., et al.: Using federated data sources and Varian Learning Portal framework to train a neural network model for automatic organ segmentation. Physica Medica 72, 39–45 (2020)

    Google Scholar 

  15. Dahele, M., Verbakel, W.: Treatment planning, intrafraction monitoring and delivery: linear accelerator-based stereotactic spine radiotherapy. Stereotact Body Radiat Ther Spinal Metastasis Future Medicine Ltd., pp. 37–55 (2014)

    Google Scholar 

  16. De Vos, B.D., Berendsen, F.F., Viergever, M.A., Sokooti, H., Staring, M., Išgum, I.: A deep learning framework for unsupervised affine and deformable image registration. Med. Image Anal. 52, 128–143 (2019)

    Google Scholar 

  17. Ding, Q., Chen, G., Zhang, X., Huang, Q., Ji, H., Gao, H.: Low-dose CT with deep learning regularization via proximal forward backward splitting. Phys. Med. Biol. (2020). https://doi.org/10.1088/1361-6560/ab831a

  18. Dong, X., et al.: Automatic multiorgan segmentation in thorax CT images using U-Net-GAN. Med. Phys. 46, 2157–2168 (2019)

    Google Scholar 

  19. Dunne, E.M., Fraser, I.M., Liu, M.: Stereotactic body radiation therapy for lung, spine and oligometastatic disease: current evidence and future directions. Ann. Transl. Med. 6(14), 283 (2018). https://doi.org/10.21037/atm.2018.06.40

    Article  Google Scholar 

  20. Elbakri, I.A., Fessler, J.A.: Statistical image reconstruction for polyenergetic X-ray computed tomography. IEEE Trans. Med. Imaging 21(2), 89–99 (2002)

    Google Scholar 

  21. Elmahdy, M.S., et al.: Robust contour propagation using deep learning and image registration for online adaptive proton therapy of prostate cancer. Med. Phys. 46, 3329–3343 (2019)

    Google Scholar 

  22. Erath, J., Vöth, T., Maier, J., Kachelrieß, M.: Forward and cross-scatter estimation in dual source CT using the deep scatter estimation (DSE). In: Medical Imaging 2019: Physics of Medical Imaging, vol. 10948, p. 24. International Society for Optics and Photonics (2019). https://doi.org/10.1117/12.2512718

  23. Erdogan, H., Fessler, J.A.: Ordered subsets algorithms for transmission tomography. Phys. Med. Biol. 44(11), 2835–2851 (1999)

    Google Scholar 

  24. Fu, L., De Man, B.: A hierarchical approach to deep learning and its application to tomographic reconstruction. In: 15th International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine, vol. 11072, p. 1107202. International Society for Optics and Photonics (2019)

    Google Scholar 

  25. Fu, Y., Lei, Y., Wang, T., Curran, W.J., Liu, T., Yang, X.: Deep learning in medical image registration: a review. Phys. Med. Biol. (2020). https://doi.org/10.1088/1361-6560/ab843e

  26. Fu, Y., et al.: Pelvic multi-organ segmentation on cone-beam CT for prostate adaptive radiotherapy. Med. Phys. (2020). https://doi.org/10.1002/mp.14196

  27. Gibson, E., et al.: Inter-site variability in prostate segmentation accuracy using deep learning. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 506–514. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_58

    Chapter  Google Scholar 

  28. Gordon, R., Bender, R., Herman, G.: Algebraic Reconstruction Techniques (ART) for three-dimensional electron microscopy and X-ray photography. J. Theor. Biol. 29(3), 471–481 (1970)

    Google Scholar 

  29. Haas, B., Coradi, T., et al.: Automatic segmentation of thoracic and pelvic CT images for radiotherapy planning using implicit anatomic knowledge and organ-specific segmentation strategies. Phys. Med. Biol. 53(6), 1751–1771 (2008)

    Google Scholar 

  30. Hammernik, K., Würfl, T., Pock, T., Maier, A.: A deep learning architecture for limited-angle computed tomography reconstruction. In: Maier-Hein, K.H., Deserno, T.M., Handels, H., Tolxdorff, T. (Hrsg.) Bildverarbeitung für die Medizin 2017. INFORMAT, pp. 92–97. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-662-54345-0_25

  31. Han, Y.S., Yoo, J., Ye, J.C.: Deep residual learning for compressed sensing CT reconstruction via persistent homology analysis. arXiv preprint arXiv:1611.06391, November 2016. http://arxiv.org/abs/1611.06391

  32. Hänsch, A., Dicken, V., Grass, T., Morgas, T., Klein, J., Meine, H.: Deep learning based segmentation of organs of the female pelvis in CBCT scans for adaptive radiotherapy using CT and CBCT data. Comput. Assist. Radiol. Surg. CARS 2018, 133 (2018)

    Google Scholar 

  33. Haytmyradov, M., et al.: Adaptive weighted log subtraction based on neural networks for markerless tumor tracking using dual energy fluoroscopy. Med. Phys. 47(2), 672–680 (2020)

    Google Scholar 

  34. Haytmyradov, M., et al.: Markerless tumor tracking using fast-kV switching dual-energy fluoroscopy on a benchtop system. Med. Phys. 46(7), 3235–3244 (2019)

    Google Scholar 

  35. Hindley, N., Keall, P., Booth, J., Shieh, C.: Real-time direct diaphragm tracking using kV imaging on a standard linear accelerator. Med. Phys. 46(10), 4481–4489 (2019)

    Google Scholar 

  36. Hirai, R., Sakata, Y., Tanizawa, A., Mori, S.: Real-time tumor tracking using fluoroscopic imaging with deep neural network analysis. Physica Medica 59, 22–29 (2019)

    Google Scholar 

  37. Huang, Y., Preuhs, A., Manhart, M., Lauritsch, G., Maier, A.: Data consistent CT reconstruction from insufficient data with learned prior images. arXiv preprint arXiv:2005.10034 (2020)

  38. Radiology Support Devices Inc.: PIXY: Anthropomorphic Phantoms - Radiology Support Devices. http://rsdphantoms.com/radiology/anthropomorphic-phantoms/. Accessed 7 Oct 2019

  39. Jaffray, D.: Image-guided radiotherapy: from current concept to future perspectives. Nat. Rev. Clin. Oncol. 9(12), 688 (2012)

    Google Scholar 

  40. Jégou, S., Drozdzal, M., Vázquez, D., Romero, A., Bengio, Y.: The one hundred layers Tiramisu: fully convolutional DenseNets for semantic segmentation. CoRR abs/1611.09326 (2016). http://arxiv.org/abs/1611.09326

  41. Jeung, A., Zhu, L., Mostafavi, H., van Heteren, J.: What image features are good for correlation-based tracking algorithms used for soft tissue monitoring in X-ray imaging. In: AAPM Annual Meeting, San Antonio, TX (2019)

    Google Scholar 

  42. Jia, X., et al.: Statistical CT reconstruction using region-aware texture preserving regularization learning from prior normal-dose CT image. Phys. Med. Biol. 63(22), 225020 (2018)

    Google Scholar 

  43. Jin, K.H., McCann, M.T., Froustey, E., Unser, M.: Deep convolutional neural network for inverse problems in imaging. IEEE Trans. Image Process. 26(9), 4509–4522 (2017)

    MathSciNet  MATH  Google Scholar 

  44. Kang, E., Koo, H.J., Yang, D.H., Seo, J.B., Ye, J.C.: Cycle-consistent adversarial denoising network for multiphase coronary CT angiography. Med. Phys. 46(2), 550–562 (2019)

    Google Scholar 

  45. Keall, P.J., Hsu, A., Xing, L.: Image-guided adaptive radiotherapy. In: Hoppe, R.T., Phillips, T.L., Roach, M. (eds.) Leibel and Phillips Textbook of Radiation Oncology, 3rd edn., pp. 213–223. W.B. Saunders, Philadelphia (2010)

    Google Scholar 

  46. Kofler, A., Haltmeier, M., Kolbitsch, C., Kachelrieß, M., Dewey, M.: A U-Nets cascade for sparse view computed tomography. In: Knoll, F., Maier, A., Rueckert, D. (eds.) MLMIR 2018. LNCS, vol. 11074, pp. 91–99. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00129-2_11

    Chapter  Google Scholar 

  47. Kristan, M., et al.: A novel performance evaluation methodology for single-target trackers. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2137–2155 (2016)

    Google Scholar 

  48. Lecchi, M., Fossati, P., Elisei, F., Orecchia, R., Lucignani, G.: Current concepts on imaging in radiotherapy. Eur. J. Nucl. Med. Mol. Imaging 35(4), 821–837 (2008)

    Google Scholar 

  49. Li, W., Sahgal, A., Foote, M., Millar, B.A., Jaffray, D.A., Letourneau, D.: Impact of immobilization on intrafraction motion for spine stereotactic body radiotherapy using cone beam computed tomography. Int. J. Radiat. Oncol. Biol. Phys. 84(2), 520–526 (2012)

    Google Scholar 

  50. Liao, H., Lin, W.A., Zhou, S.K., Luo, J.: ADN: artifact disentanglement network for unsupervised metal artifact reduction. IEEE Trans. Med. Imaging 39(3), 634–643 (2020)

    Google Scholar 

  51. Lin, T., et al.: Microsoft COCO: common objects in context (2014). http://arxiv.org/abs/1405.0312

  52. Lin, W.A., et al.: DuDoNet: dual domain network for CT metal artifact reduction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10504–10513 (2020)

    Google Scholar 

  53. Lyu, Y., Lin, W.A., Lu, J., Zhou, S.K.: DuDoNet++: encoding mask projection to reduce CT metal artifacts. arXiv preprint arXiv:2001.00340 (2020)

  54. Maier, J., Sawall, S., Kachelrieß, M.: Deep scatter estimation (DSE): feasibility of using a deep convolutional neural network for real-time x-ray scatter prediction in cone-beam CT. In: Medical Imaging, vol. 10573, pp. 393–398. SPIE (2018). https://doi.org/10.1117/12.2292919

  55. Mansilla, L., Milone, D.H., Ferrante, E.: Learning deformable registration of medical images with anatomical constraints. Neural Netw. 124, 269–279 (2020). https://doi.org/10.1016/j.neunet.2020.01.023

    Article  Google Scholar 

  56. Men, K., et al.: Fully automatic and robust segmentation of the clinical target volume for radiotherapy of breast cancer using big data and deep learning. Physica Medica 50, 13–19 (2018)

    Google Scholar 

  57. Microsoft: COCO - Common Objects in Context. http://cocodataset.org/#detection-eval. Accessed 26 Sept 2019

  58. Nikolov, S., et al.: Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy. CoRR abs/1809.04430 (2018)

    Google Scholar 

  59. Oktay, O., et al.: Anatomically Constrained Neural Networks (ACNN): application to cardiac image enhancement and segmentation. CoRR abs/1705.08302 (2017). http://arxiv.org/abs/1705.08302

  60. Park, H.S., Lee, S.M., Kim, H.P., Seo, J.K., Chung, Y.E.: CT sinogram-consistency learning for metal-induced beam hardening correction. Med. Phys. 45(12), 5376–5384 (2018)

    Google Scholar 

  61. Paysan, P., Munro, P., Scheib, S.G.: CT based simulation framework for motion artifact and ground truth generation of cone-beam CT. In: AAPM Annual Meeting, San Antonio, TX (2019)

    Google Scholar 

  62. Paysan, P., Strzelecki, A., Arrate, F., Munro, P., Scheib, S.G.: Convolutional network based motion artifact reduction in cone-beam CT. In: AAPM Annual Meeting, San Antonio, TX (2019)

    Google Scholar 

  63. Potters, L., et al.: American society for therapeutic radiology and oncology and american college of radiology practice guideline for the performance of stereotactic body radiation therapy. Int. J. Radiat. Oncol. Biol. Phys. 60(4), 1026–1032 (2004)

    Google Scholar 

  64. Ritschl, L., Sawall, S., Knaup, M., Hess, A., Kachelrie, M.: Iterative 4D cardiac micro-CT image reconstruction using an adaptive spatio-temporal sparsity prior. Phys. Med. Biol. 57(6), 1517–1525 (2012)

    Google Scholar 

  65. Rockmore, A.J., Macovski, A.: A maximum likelihood approach to emission image reconstruction from projections. IEEE Trans. Nucl. Sci. 23(4), 1428–1432 (1976)

    Google Scholar 

  66. Roggen, T., Bobic, M., Givehchi, N., Scheib, S.G.: Deep Learning model for markerless tracking in spinal SBRT. Physica Medica Eur. J. Med. Phys. 74, 66–73 (2020)

    Google Scholar 

  67. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. CoRR abs/1505.04597 (2015). http://arxiv.org/abs/1505.04597

  68. Sauppe, S., Hahn, A., Brehm, M., Paysan, P., Seghers, D., Kachelrieß, M.: PO-0934: cardio-respiratory motion compensation for 5D thoracic CBCT in IGRT. Radiother. Oncol. 119, S452–S453 (2016)

    Google Scholar 

  69. Sauppe, S., Kuhm, J., Brehm, M., Paysan, P., Seghers, D., Kachelrieß, M.: Motion vector field phase-to-amplitude resampling for 4D motion-compensated cone-beam CT. Phys. Med. Biol. 63(3), 035032 (2018)

    Google Scholar 

  70. Schnurr, A.K., Chung, K., Russ, T., Schad, L.R., Zöllner, F.G.: Simulation-based deep artifact correction with convolutional neural networks for limited angle artifacts. Zeitschrift fur Medizinische Physik 29(2), 150–161 (2019)

    Google Scholar 

  71. Schreier, J., Attanasi, F., Laaksonen, H.: A full-image deep segmenter for CT images in breast cancer radiotherapy treatment. Front. Oncol. 9, 677 (2019)

    Google Scholar 

  72. Schreier, J., Attanasi, F., Laaksonen, H.: Generalization vs. specificity. In: which cases should a clinic train its own segmentation models? Front. Oncol. 10, 675 (2020)

    Google Scholar 

  73. Schreier, J., Genghi, A., Laaksonen, H.: Clinical evaluation of a full-image deep segmentation algorithm for the male pelvis on cone-beam CT and CT. Radiother. Oncol. 145, 1–6 (2020)

    Google Scholar 

  74. Shojaii, R., Alirezaie, J., Babyn, P.: Automatic lung segmentation in CT images using watershed transform. In: Proceedings of the International Conference on Image Processing, ICIP (2005)

    Google Scholar 

  75. Sonke, J.J., Zijp, L., Remeijer, P., van Herk, M.: Respiratory correlated cone beam CT. Med. Phys. 32(4), 1176–1186 (2005)

    Google Scholar 

  76. Star-Lack, J., et al.: A modified McKinnon-Bates (MKB) algorithm for improved 4D cone-beam computed tomography (CBCT) of the lung. Med. Phys. 45(8), 3783–3799 (2018)

    Google Scholar 

  77. Vishnevskiy, V., Rau, R., Goksel, O.: Deep variational networks with exponential weighting for learning computed tomography. In: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.-T., Khan, A. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 310–318. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_35

    Chapter  Google Scholar 

  78. Wang, A.S., Stayman, J.W., Otake, Y., Vogt, S., Kleinszig, G., Siewerdsen, J.H.: Accelerated statistical reconstruction for C-arm cone-beam CT using Nesterov’s method. Med. Phy. 42(5), 2699–2708 (2015)

    Google Scholar 

  79. Wang, H., et al.: Dosimetric effect of translational and rotational errors for patients undergoing image-guided stereotactic body radiotherapy for spinal metastases. Int. J. Radiat. Oncol. Biol. Phys. 71(4), 1261–1271 (2008)

    Google Scholar 

  80. Wang, J., Liang, J., Cheng, J., Guo, Y., Zeng, L.: Deep learning based image reconstruction algorithm for limited-angle translational computed tomography. PLoS ONE 15(1), e0226963 (2020)

    Google Scholar 

  81. Würfl, T., Ghesu, F.C., Christlein, V., Maier, A.: Deep learning computed tomography. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 432–440. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46726-9_50

    Chapter  Google Scholar 

  82. Zhang, X., Jian, W., Chen, Y., Yang, S.T.: Deform-GAN: an unsupervised learning model for deformable registration. ArXiv abs/2002.11430 (2020)

    Google Scholar 

  83. Zhang, Y., Yu, H.: Convolutional neural network based metal artifact reduction in X-ray computed tomography. IEEE Trans. Med. Imaging 37(6), 1370–1381 (2018)

    Google Scholar 

  84. Zhang, Y., Huang, X., Wang, J.: Advanced 4-dimensional cone-beam computed tomography reconstruction by combining motion estimation, motion-compensated reconstruction, biomechanical modeling and deep learning. Vis. Comput. Ind. Biomed. Art 2(1), 1–15 (2019)

    Google Scholar 

  85. Zhang, Z., Liang, X., Dong, X., Xie, Y., Cao, G.: A sparse-view CT reconstruction method based on combination of DenseNet and deconvolution. IEEE Trans. Med. Imaging 37(6), 1407–1417 (2018)

    Google Scholar 

  86. Zhao, W., et al.: Markerless pancreatic tumor target localization enabled by deep learning. Int. J. Radiat. Oncol. Biol. Phys. 105(2), 432–439 (2019)

    Google Scholar 

  87. Zhu, B., Liu, J.Z., Rosen, B.R., Rosen, M.S.: Image reconstruction by domain transform manifold learning. Nature 555(7697), 487–492 (2018)

    Google Scholar 

  88. ZHu, L., Baturin, P.: Deep neural network image fusion without using training data. In: AAPM ePoster Library (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Pascal Paysan .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Paysan, P. et al. (2020). Deep Learning Methods for Image Guidance in Radiation Therapy. In: Schilling, FP., Stadelmann, T. (eds) Artificial Neural Networks in Pattern Recognition. ANNPR 2020. Lecture Notes in Computer Science(), vol 12294. Springer, Cham. https://doi.org/10.1007/978-3-030-58309-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58309-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58308-8

  • Online ISBN: 978-3-030-58309-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics