Skip to main content
SpringerLink
Log in

A preliminary exploration into top-down and bottom-up deep-learning approaches to localising neuro-interventional point targets in volumetric MRI

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Point localisation is a critical aspect of many interventional planning procedures, specifically representing anatomical regions of interest or landmarks as individual points. This could be seen as analogous to the problem of visual search in cognitive psychology, in which this search is performed either: bottom-up, constructing increasingly abstract and coarse-resolution features over the entire image; or top-down, using contextual cues from the entire image to refine the scope of the region being investigated. Traditional convolutional neural networks use the former, but it is not clear if this is optimal. This article is a preliminary investigation as to how this motivation affects 3D point localisation in neuro-interventional planning.

Methods

Two neuro-imaging datasets were collected: one for cortical point localisation for repetitive transcranial magnetic stimulation and the other for sub-cortical anatomy localisation for deep brain stimulation. Four different frameworks were developed using top-down versus bottom-up paradigms as well as representing points as co-ordinates or heatmaps. These networks were applied to point localisation for transcranial magnetic stimulation and subcortical anatomy localisation. These networks were evaluated using cross-validation and a varying number of training datasets to analyse their sensitivity to quantity of training data.

Results

Each network shows increasing performance as the amount of available training data increases, with the co-ordinate-based top-down network consistently outperforming the others. Specifically, the top-down architectures tend to outperform the bottom-up ones. An analysis of their memory consumption also encourages the top-down co-ordinate based architecture as it requires significantly less memory than either bottom-up architectures or those representing their predictions via heatmaps.

Conclusion

This paper is a preliminary foray into a fundamental aspect of machine learning architectural design: that of the top-down/bottom-up divide from cognitive psychology. Although there are additional considerations within the particular architectures investigated that could affect these results and the number of architectures investigated is limited, our results do indicate that the less commonly used top-down paradigm could lead to more efficient and effective architectures in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Result tables for all experiments are available on request. However, we can not provide sensitive patient data.

Code availability

Our code will be available on request.

Notes

  1. http://www.itksnap.org/pmwiki/pmwiki.php?n=Convert3D.Convert3D, The specific commands used were swapdim, -resample-mm, and -pad-to.

References

  1. He K, Sun J (2015) Convolutional neural networks at constrained time cost. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5353–5360

  2. Qiao S, Lin Z, Zhang J, Yuille A L (2019) Neural rejuvenation: improving deep network training by enhancing computational resource utilization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 61–71

  3. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587

  4. Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp. 1440–1448

  5. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, vol. 28, pp. 91–99

  6. Sugimori H, Kawakami M (2019) Automatic detection of a standard line for brain magnetic resonance imaging using deep learning. In: Applied Sciences, vol. 9, 3849

  7. Yang X, Tang WT, Tjio G, Yeo SY, Su Y (2020) Automatic detection of anatomical landmarks in brain MR scanning using multi-task deep neural networks. In: Neurocomputing, vol. 396, pp. 514–521

  8. Gohel B, Kumar L, Shah D (2023) Deep learning-based automated localisation of anterior commissure and posterior commissure landmarks in 3d space from three-plane 2d mri localiser slices of the brain. In: Procedia Computer Science, vol. 218, pp. 1027–1032

  9. Baxter JS, Bui QA, Maguet E, Croci S, Delmas A, Lefaucheur J-P, Bredoux L, Jannin P (2021) Automatic cortical target point localisation in MRI for transcranial magnetic stimulation via a multi-resolution convolutional neural network. In: International Journal of Computer Assisted Radiology and Surgery, vol. 16, pp.1077–1087

  10. Foulsham T, Chapman C, Nasiopoulos E, Kingstone A (2014) Top-down and bottom-up aspects of active search in a real-world environment. In: Canadian Journal of Experimental Psychology, vol. 68, pp. 8–19

  11. Li S, Gong Q, Li H, Chen S, Liu Y, Ruan G, Zhu L, Liu L, Chen H (2022) Automatic location scheme of anatomical landmarks in 3d head MRI based on the scale attention hourglass network. Comput Methods Progr Biomed 214:106564

    Article  Google Scholar 

  12. Lester H, Arridge SR (1999) A survey of hierarchical non-linear medical image registration. Pattern Recogn 32(1):129–149

    Article  ADS  Google Scholar 

  13. Lefaucheur J-P (2019) Transcranial magnetic stimulation. Handb Clin Neurol 160:559–580

    Article  PubMed  Google Scholar 

  14. Balconi M (2013) Dorsolateral prefrontal cortex, working memory and episodic memory processes: insight through transcranial magnetic stimulation techniques. Neurosci Bull 29:381–389

    Article  PubMed  PubMed Central  Google Scholar 

  15. Hamid P, Malik BH, Hussain ML (2019) Noninvasive transcranial magnetic stimulation (TMS) in chronic refractory pain: a systematic review. Cureus 11(10):e6019

    PubMed  PubMed Central  Google Scholar 

  16. Sparing R, Buelte D, Meister IG, Pauš T, Fink GR (2008) Transcranial magnetic stimulation and the challenge of coil placement: a comparison of conventional and stereotaxic neuronavigational strategies. Hum Brain Mapp 29(1):82–96

    Article  PubMed  Google Scholar 

  17. Siebner HR, Hartwigsen G, Kassuba T, Rothwell JC (2009) How does transcranial magnetic stimulation modify neuronal activity in the brain? Implications for studies of cognition. Cortex 45(9):1035–1042

    Article  PubMed  PubMed Central  Google Scholar 

  18. Middlebrooks E, Domingo R, Vivas-Buitrago T, Okromelidze L, Tsuboi T, Wong J, Eisinger R, Almeida L, Burns M, Horn A et al (2020) Neuroimaging advances in deep brain stimulation: review of indications, anatomy, and brain connectomics. Am J Neuroradiol 41(9):1558–1568

  19. Baxter JS, Jannin P (2023) Validation in the age of machine learning: a framework for describing validation with examples in transcranial magnetic stimulation and deep brain stimulation. Intell. -Based Med. 7:100090

  20. Haegelen C, Coupé P, Fonov V, Guizard N, Jannin P, Morandi X, Collins DL (2013) Automated segmentation of basal ganglia and deep brain structures in MRI of Parkinson’s disease. Int J Comput Assist Radiol Surg 8(1):99–110

    Article  PubMed  Google Scholar 

  21. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431–3440

  22. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention-MICCAI, 18th International Conference, Munich, Proceedings, Part III 18. Springer 2015, pp. 234–241

  23. Ranftl R, Bochkovskiy A, Koltun V (2021) Vision transformers for dense prediction. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 12 179–12 188

  24. Hatamizadeh A, Nath V, Tang Y, Yang D, Roth HR, Xu D (2021) Swin unetr: swin transformers for semantic segmentation of brain tumors in MRI images. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries: 7th international workshop, BrainLes. Held in Conjunction with MICCAI 2021, Virtual Event, Revised Selected Papers. Part I. Springer 2022:272–284

  25. Baxter JS, Jannin P (2022) Combining simple interactivity and machine learning: a separable deep learning approach to subthalamic nucleus localization and segmentation in MRI for deep brain stimulation surgical planning. J Med Imaging 9(4):045001

    Article  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge SYNEIKA (Rennes, France) for the use of their TMS targeting dataset.

Funding

for EG was received from the Institut des Neurosciences Cliniques de Rennes (INCR) and the Allocations de Recherche Doctorale (ARED) initiative from the Région Bretagne.

Author information

Authors and Affiliations

  1. LTSI - INSERM UMR 1099, Université de Rennes, 35000, Rennes, France

    Enora Giffard, Pierre Jannin & John S. H. Baxter

Authors
  1. Enora Giffard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pierre Jannin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John S. H. Baxter
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

EG and JB designed and carried out the experiments as well as wrote the manuscript. JB and PJ supervised the project.

Corresponding author

Correspondence to John S. H. Baxter.

Ethics declarations

Conflicts of interest

We have no Conflicts of interest to declare.

Ethical approval

All patient data were collected retrospectively with informed patient consent and approval from the institutional ethics review board.

Consent for publication

We consent to the publication of this article in IJCARS.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Giffard, E., Jannin, P. & Baxter, J. .H. A preliminary exploration into top-down and bottom-up deep-learning approaches to localising neuro-interventional point targets in volumetric MRI. Int J CARS 19, 283–296 (2024). https://doi.org/10.1007/s11548-023-03023-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-023-03023-9

Keywords

Search

Navigation