Skip to main content

MASSM: An End-to-End Deep Learning Framework for Multi Anatomy Statistical Shape Modeling Directly From Images

  • Conference paper
  • First Online:
Shape in Medical Imaging (ShapeMI 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15275))

Included in the following conference series:

  • 55 Accesses

Abstract

Statistical shape modeling (SSM) effectively analyzes anatomical variations within populations but is limited by the need for manual localization and segmentation, which relies on scarce medical expertise. Recent advances in deep learning have provided a promising approach that automatically generates statistical representations (as point distribution models or PDMs) from unsegmented images. Once trained, these deep learning-based models eliminate the need for manual segmentation for new subjects. Most deep learning methods still require manual pre-alignment of image volumes and bounding box specifications around the target anatomy, leading to a partially manual inference process. Recent approaches facilitate anatomy localization but only estimate population-level statistical representations and cannot directly delineate anatomy in images. Additionally, they are limited to modeling a single anatomy. We introduce MASSM, a novel end-to-end deep learning framework that simultaneously localizes multiple anatomies, estimates population-level statistical representations, and delineates shape representations directly in image space. Our results show that MASSM, which delineates anatomy in image space and handles multiple anatomies through a multitask network, provides superior shape information compared to segmentation networks for medical imaging tasks. Estimating SSM is a stronger task than segmentation because it encodes a more robust statistical prior for the objects to be detected and delineated. MASSMallows for more accurate and comprehensive shape representations, surpassing the capabilities of traditional pixel-wise segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Adams, J., Bhalodia, R., Elhabian, S.: Uncertain-DeepSSM: from images to probabilistic shape models. In: Reuter, M., Wachinger, C., Lombaert, H., Paniagua, B., Goksel, O., Rekik, I. (eds.) ShapeMI 2020. LNCS, vol. 12474, pp. 57–72. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61056-2_5

    Chapter  Google Scholar 

  2. Adams, J., Elhabian, S.: From images to probabilistic anatomical shapes: a deep variational bottleneck approach. arXiv preprint arXiv:2205.06862 (2022)

  3. Adams, J., Iyer, K., Elhabian, S.: Weakly supervised Bayesian shape modeling from unsegmented medical images. arXiv preprint arXiv:2405.09697 (2024)

  4. Almalki, F.A., Cortes, D.H.: Statistical shape modelling of the lumbar spine with reference to gender and principal component analysis. In: ASME International Mechanical Engineering Congress and Exposition, vol. 87622, p. V005T06A065. American Society of Mechanical Engineers (2023)

    Google Scholar 

  5. Atkins, P.R., et al.: Quantitative comparison of cortical bone thickness using correspondence-based shape modeling in patients with cam femoroacetabular impingement. J. Orthop. Res. 35(8), 1743–1753 (2017)

    Article  Google Scholar 

  6. Atkins, P.R., Morris, A., Elhabian, S.Y., Anderson, A.E.: A correspondence-based network approach for groupwise analysis of patient-specific spatiotemporal data. Ann. Biomed. Eng. 51(10), 2289–2300 (2023)

    Article  Google Scholar 

  7. Atkins, P.R., et al.: Which two-dimensional radiographic measurements of cam femoroacetabular impingement best describe the three-dimensional shape of the proximal femur? Clin. Orthop. Relat. Res. 477(1), 242 (2019)

    Article  Google Scholar 

  8. Barshan, E., Fieguth, P.: Stage-wise training: an improved feature learning strategy for deep models. In: Feature Extraction: Modern Questions and Challenges, pp. 49–59. PMLR (2015)

    Google Scholar 

  9. Bhalodia, R., Elhabian, S., Adams, J., Tao, W., Kavan, L., Whitaker, R.: DeepSSM: a blueprint for image-to-shape deep learning models. Med. Image Anal. 91, 103034 (2024)

    Article  Google Scholar 

  10. Bhalodia, R., Elhabian, S.Y., Kavan, L., Whitaker, R.T.: DeepSSM: a deep learning framework for statistical shape modeling from raw images. In: Reuter, M., Wachinger, C., Lombaert, H., Paniagua, B., Lüthi, M., Egger, B. (eds.) ShapeMI 2018. LNCS, vol. 11167, pp. 244–257. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04747-4_23

    Chapter  Google Scholar 

  11. Bhalodia, R., et al.: Deep learning for end-to-end atrial fibrillation recurrence estimation. In: 2018 Computing in Cardiology Conference (CinC), vol. 45, pp. 1–4. IEEE (2018)

    Google Scholar 

  12. Cates, J., Elhabian, S., Whitaker, R.: ShapeWorks: particle-based shape correspondence and visualization software. In: Statistical Shape and Deformation Analysis, pp. 257–298. Elsevier (2017)

    Google Scholar 

  13. Davies, R.H., Twining, C.J., Cootes, T.F., Waterton, J.C., Taylor, C.J.: A minimum description length approach to statistical shape modeling. IEEE Trans. Med. Imaging 21(5), 525–537 (2002)

    Article  Google Scholar 

  14. Durrleman, S., et al.: Morphometry of anatomical shape complexes with dense deformations and sparse parameters. Neuroimage 101, 35–49 (2014)

    Article  Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)

    Google Scholar 

  16. Huang, W., Bridge, C.P., Noble, J.A., Zisserman, A.: Temporal HeartNet: towards human-level automatic analysis of fetal cardiac screening video. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 341–349. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_39

    Chapter  Google Scholar 

  17. Iyer, K., Adams, J., Elhabian, S.Y.: SCorP: statistics-informed dense correspondence prediction directly from unsegmented medical images. arXiv preprint arXiv:2404.17967 (2024)

  18. Jeffery, N., Manson, A.: Postnatal growth and spatial conformity of the cranium, brain, eyeballs and masseter muscles in the macaque (Macaca mulatta). J. Anat. 243(4), 590–604 (2023)

    Article  Google Scholar 

  19. Karanam, M.S.T., Kataria, T., Iyer, K., Elhabian, S.Y.: ADASSM: adversarial data augmentation in statistical shape models from images. In: Wachinger, C., Paniagua, B., Elhabian, S., Li, J., Egger, J. (eds.) ShapeMI 2023. LNCS, vol. 14350, pp. 90–104. Springer, Cham (2023)

    Google Scholar 

  20. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  21. Lamb, A.M., Alias Parth Goyal, A.G., Zhang, Y., Zhang, S., Courville, A.C., Bengio, Y.: Professor forcing: a new algorithm for training recurrent networks. In: Advances in Neural Information Processing Systems, vol. 29 (2016)

    Google Scholar 

  22. Li, Z., Liu, C., Yuille, A., Ni, B., Zhang, W., Gao, W.: Progressive stage-wise learning for unsupervised feature representation enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9767–9776 (2021)

    Google Scholar 

  23. Milletari, F., Rothberg, A., Jia, J., Sofka, M.: Integrating statistical prior knowledge into convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 161–168. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_19

    Chapter  Google Scholar 

  24. Nguyen, H.P., Lee, H.J., Kim, S.: Feasibility study for the automatic surgical planning method based on statistical model. J. Orthop. Surg. Res. 18(1), 398 (2023)

    Article  Google Scholar 

  25. Raju, A., Miao, S., Jin, D., Lu, L., Huang, J., Harrison, A.P.: Deep implicit statistical shape models for 3D medical image delineation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 2135–2143 (2022)

    Google Scholar 

  26. Roda, G.F., Stoneback, J.W., Gimarc, D., Gaffney, B.M.: Above knee socket prosthesis use changes proximal femur morphology. Bone 172, 116752 (2023)

    Article  Google Scholar 

  27. Sarkalkan, N., Weinans, H., Zadpoor, A.A.: Statistical shape and appearance models of bones. Bone 60, 129–140 (2014)

    Article  Google Scholar 

  28. Tao, W., Bhalodia, R., Elhabian, S.: Learning population-level shape statistics and anatomy segmentation from images: a joint deep learning model. arXiv preprint arXiv:2201.03481 (2022)

  29. Ukey, J., Elhabian, S.: Localization-aware deep learning framework for statistical shape modeling directly from images. In: Medical Imaging with Deep Learning (2023)

    Google Scholar 

  30. Ukey, J., Kataria, T., Elhabian, S.Y.: Weakly SSM: on the viability of weakly supervised segmentations for statistical shape modeling. arXiv preprint arXiv:2407.15260 (2024)

  31. Vicory, J., et al.: SlicerSALT: shape AnaLysis toolbox. In: Reuter, M., Wachinger, C., Lombaert, H., Paniagua, B., Lüthi, M., Egger, B. (eds.) ShapeMI 2018. LNCS, vol. 11167, pp. 65–72. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04747-4_6

    Chapter  Google Scholar 

  32. Wasserthal, J., et al.: TotalSegmentator: robust segmentation of 104 anatomic structures in CT images. Radiol. Artif. Intell. 5(5) (2023)

    Google Scholar 

  33. Xie, J., Dai, G., Zhu, F., Wong, E.K., Fang, Y.: DeepShape: deep-learned shape descriptor for 3D shape retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1335–1345 (2016)

    Article  Google Scholar 

  34. Yamamoto, T., Paulus, P., Setliff, J.C., Hogan, M.V., Anderst, W.J.: Influence of talar and calcaneal morphology on subtalar kinematics during walking. Foot Ankle Int. 45(6), 10711007241231981 (2024). https://doi.org/10.1177/10711007241231981. pMID: 38491768

  35. Zheng, Y., Liu, D., Georgescu, B., Nguyen, H., Comaniciu, D.: 3D deep learning for efficient and robust landmark detection in volumetric data. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 565–572. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24553-9_69

    Chapter  Google Scholar 

  36. Zhou, X., Wang, D., Krähenbühl, P.: Objects as points. arXiv preprint arXiv:1904.07850 (2019)

Download references

Acknowledgements

The National Institutes of Health supported this work under grant numbers NIBIB-U24EB029011 and NIAMS-R01AR076120. The content is solely the authors’ responsibility and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shireen Y. Elhabian .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ukey, J., Kataria, T., Elhabian, S.Y. (2025). MASSM: An End-to-End Deep Learning Framework for Multi Anatomy Statistical Shape Modeling Directly From Images. In: Wachinger, C., Paniagua, B., Elhabian, S., Luijten, G., Egger, J. (eds) Shape in Medical Imaging. ShapeMI 2024. Lecture Notes in Computer Science, vol 15275. Springer, Cham. https://doi.org/10.1007/978-3-031-75291-9_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-75291-9_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-75290-2

  • Online ISBN: 978-3-031-75291-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics