Skip to main content
Springer Nature Link
Log in

3D biplanar reconstruction of lower limbs using nonlinear statistical models

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Three-dimensional (3D) reconstruction of lower limbs is of great interest in surgical planning, computer assisted surgery, and for biomechanical applications. The use of 3D imaging modalities such as computed tomography (CT) scan and magnetic resonance imaging (MRI) has limitations such as high radiation and expense. Therefore, three-dimensional reconstruction methods from biplanar X-ray images represent an attractive alternative. In this paper, we present a new unsupervised 3D reconstruction method for the patella, talus, and pelvis using calibrated biplanar (45- and 135-degree oblique) radiographic images and a prior information on the geometric/anatomical structure of these complex bones. A multidimensional scaling (MDS)-based nonlinear dimensionality reduction algorithm is applied to exploit this prior geometric/anatomical information. It represents relevant deformations existing in the training set. Our method is based on a hybrid-likelihood using regions and contours. The edge-based notion represents the relation between the external contours of the bone projections and an edge potential field estimated on the radiographic images. Region-based notion is the non-overlapping ratio between segmented and projected bone regions of interest (RoIs). Our automatic 3D reconstruction model entails stochastically minimizing an energy function allowing an estimation of deformation parameters of the bone shape. This 3D reconstruction method has been successfully tested on 13 biplanar radiographic image pairs, yielding very promising results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. Due to the physical link between the detector source assemblies, the position in space of the sensors and X-ray sources are well known: the radiographic environment is therefore pre-calibrated.

References

  1. Terzopoulos D, Witkin A, Kass M (1988) Constraints on deformable models: recovering 3D shape and nonrigid motion. Artificial Intelligence 36(1):91–123

    Article  Google Scholar 

  2. Benjamin R, Prakoonwit S, Matalas I, Kitney R (1996) Object-based three-dimensional X-ray imaging. Medical and Biological Engineering and Computing 34(6):423–430

    Article  CAS  PubMed  Google Scholar 

  3. Nikkhade-Dehkordi B, Bro-Nielsen M, Darvann T, Gramkow C, Egund N, Hermann K (1996) 3D reconstruction of the femoral bone using two X-ray images from orthogonal views. In: Computer Assisted Radiology, p 1015

  4. Kita Y (1996) Elastic-model driven analysis of several views of a deformable cylindrical object. IEEE Trans Pattern Anal Mach Intell 18(12):1150–1162

    Article  Google Scholar 

  5. Bardinet E, Cohen LD, Ayache N (1998) A parametric deformable model to fit unstructured 3D data. Comp Vision Image Underst. 71(1):39–54

    Article  Google Scholar 

  6. Mitton D, Landry C, Veron S, Skalli W, Lavaste F, De Guise JA (2000) 3D reconstruction method from biplanar radiography using non-stereo corresponding points and elastic deformable meshes. Med Biol Eng Comput 38(2):133–139

    Article  CAS  PubMed  Google Scholar 

  7. Baka N, Kaptein BL, de Bruijne M, van Walsum T, Giphart J, Niessen WJ, Lelieveldt BP (2011) 2D–3D shape reconstruction of the distal femur from stereo X-ray imaging using statistical shape models. Med Image Anal 15(6):840–850

    Article  CAS  PubMed  Google Scholar 

  8. Ehlke M, Ramm H, Lamecker H, Hege HC, Zachow S (2013) Fast generation of virtual X-ray images for reconstruction of 3D anatomy. IEEE Trans Vis Comput Graph 19(12):2673–2682

    Article  PubMed  Google Scholar 

  9. Zhang J, Besier TF (2017) Accuracy of femur reconstruction from sparse geometric data using a statistical shape model. Computer methods in biomechanics and biomedical engineering 20(5):566–576

    Article  PubMed  Google Scholar 

  10. Fotsin TJT, Vázquez C, Cresson T, De Guise J (2019) Shape, pose and density statistical model for 3D reconstruction of articulated structures from X-ray images. In: 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 2748–2751

  11. Gajny L, Girinon F, Bayoud W, Lahkar B, Bonnet-Lebrun A, Rouch P, Lazennec JY, Skalli W (2022) Fast quasi-automated 3D reconstruction of lower limbs from low dose biplanar radiographs using statistical shape models and contour matching. Medical Engineering & Physics 101

  12. Reyneke CJF, Lüthi M, Burdin V, Douglas TS, Vetter T, Mutsvangwa TE (2018) Review of 2-D/3-D reconstruction using statistical shape and intensity models and X-ray image synthesis: toward a unified framework. IEEE Rev Biomed Eng 12:269–286

    Article  PubMed  Google Scholar 

  13. Kim H, Lee K, Lee D, Baek N (2019) 3D reconstruction of leg bones from X-ray images using CNN-based feature analysis. In: International Conference on Information and Communication Technology Convergence (ICTC), pp 669–672

  14. Benameur S, Mignotte M, Labelle H, De Guise JA (2005) A hierarchical statistical modeling approach for the unsupervised 3-D biplanar reconstruction of the scoliotic spine. IEEE Trans Biomed Eng 52(12):2041–2057

    Article  PubMed  Google Scholar 

  15. Benameur S, Mignotte M, Destrempes F, De Guise JA (2005) Three-dimensional biplanar reconstruction of scoliotic rib cage using the estimation of a mixture of probabilistic prior models. IEEE Trans Biomed Eng 52(10):1713–1728

    Article  PubMed  Google Scholar 

  16. Yang CJ, Lin CL, Wang CK, Wang JY, Chen CC, Su FC, Lee YJ, Lui CC, Yeh LR, Fang YHD (2022) Generative adversarial network (GAN) for automatic reconstruction of the 3D spine structure by using simulated bi-planar X-ray images. Diagnostics 12(5):1121

    Article  PubMed  PubMed Central  Google Scholar 

  17. Kasten Y, Doktofsky D, Kovler I (2020) End-to-end convolutional neural network for 3D reconstruction of knee bones from bi-planar X-ray images. In: International Workshop on Machine Learning for Medical Image Reconstruction, pp 123–133

  18. Ahishakiye E, Van Gijzen MB, Tumwiine J, Wario R, Obungoloch J (2021) A survey on deep learning in medical image reconstruction. Intelligent Medicine

  19. Maken P, Gupta A (2022) 2D-to-3D: a review for computational 3D image reconstruction from X-ray images. Archives of Computational Methods in Engineering, p 1–30

  20. Binol H, Niazi MKK, Plotner A, Sopkovich J, Kaffenberger BH, Gurcan MN (2020) A multidimensional scaling and sample clustering to obtain a representative subset of training data for transfer learning-based rosacea lesion identification. In: Medical Imaging: Computer-Aided Diagnosis, International Society for Optics and Photonics 11314:1131415

  21. Cao X, Chen C, Tian L (2021) Supervised multidimensional scaling and its application in MRI-based individual age predictions. Neuroinformatics 19(2):219–231

    Article  PubMed  Google Scholar 

  22. Nguyen DCT, Benameur S, Mignotte M, Lavoie F (2018) Superpixel and multi-atlas based fusion entropic model for the segmentation of X-ray images. Med Image Anal 48:58–74

    Article  CAS  PubMed  Google Scholar 

  23. Zhang YJ (2006) An overview of image and video segmentation in the last 40 years. Advances in Image and Video Segmentation, p 1–16

  24. Cufi X, Munoz X, Freixenet J, Marti J (2006) A review of image segmentation techniques integrating region and boundary information. Advances in imaging and electron physics 120:1–39

    Google Scholar 

  25. Muñoz X, Freixenet J, Cufı X, Martı J (2003) Strategies for image segmentation combining region and boundary information. Pattern recognition letters. 24(1–3):375–392

    Article  Google Scholar 

  26. Faloutsos C, Lin KI (1995) FastMap: a fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets. In: Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, pp 163–174

  27. Huang TC, Zhang G, Guerrero T, Starkschall G, Lin K-P, Forster K (2006) Semi-automated CT segmentation using optic flow and Fourier interpolation techniques. Computer methods and programs in biomedicine 84(2–3):124–134

    Article  PubMed  Google Scholar 

  28. Rusinkiewicz S, Levoy M (2001) Efficient variants of the ICP algorithm. In: Proceedings Third International Conference on 3-D Digital Imaging and Modeling, pp 145–152

  29. Latulippe M (2013) Calage robuste et accéléré de nuages de points en environnements naturels via l’apprentissage automatique

  30. Buades A, Coll B, Morel JM (2011) Non-local means denoising. Image Processing On Line 1:208–212

    Article  Google Scholar 

  31. Canny J (1986) A computational approach to edge detection. IEEE Trans pattern Anal Mach Intell. 6:679–698

  32. Chen H, Bhanu B (2008) Efficient recognition of highly similar 3D objects in range images. IEEE Trans Pattern Anal Mach Intell. 31(1):172–179

  33. Touati R, Mignotte M (2014) MDS-based multi-axial dimensionality reduction model for human action recognition. In: 2014 Canadian Conference on Computer and Robot Vision, pp 262–267

  34. Belhadj LC, Mignotte M (2016) Spatio-temporal fastmap-based mapping for human action recognition. In: 2016 IEEE International Conference on Image Processing (ICIP), pp 3046–3050

  35. Mignotte M (2011) MDS-based multiresolution nonlinear dimensionality reduction model for color image segmentation. IEEE transactions on neural networks 22(3):447–460

    Article  PubMed  Google Scholar 

  36. Mignotte M (2012) MDS-based segmentation model for the fusion of contour and texture cues in natural images. Comput Vis Image Underst. 116(9):981–990

  37. Touati R, Mignotte M (2016) A multidimensional scaling optimization and fusion approach for the unsupervised change detection problem in remote sensing images. In: Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), pp 1–6

  38. Jacobson NP, Gupta MR (2005) Design goals and solutions for display of hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing 43(11):2684–2692

    Article  Google Scholar 

  39. Cox MA, Cox TF (2008) Multidimensional scaling. In: Handbook of Data Visualization, p 315–347

  40. Jain AK, Zhong Y, Lakshmanan S (1996) Object matching using deformable templates. IEEE Trans pattern Anal Mach I ntell. 18(3):267–278

  41. Jáuregui DAG, Horain P (2009) Region-based vs. edge-based registration for 3D motion capture by real time monoscopic vision. In: International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications, pp 344–355

  42. François O (2002) Global optimization with exploration/selection algorithms and simulated annealing. The Annals of Applied Probability 12(1):248–271

    Article  Google Scholar 

  43. Pena-Reyes CA, Sipper M (2000) Evolutionary computation in medicine: an overview. Artif Intell Med. 19(1):1–23

  44. Mignotte M, Meunier J, Tardif JC (2001) Endocardial boundary e timation and tracking in echocardiographic images using deformable template and markov random fields. Pattern Analysis & Applications 4(4):256–271

    Article  Google Scholar 

  45. Aubin CE, Dansereau J, Parent F, Labelle H, de Guise JA (1997) Morphometric evaluations of personalised 3D reconstructions and geometric models of the human spine. Medical and biological engineering and computing 35(6):611–618

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Nicholas Newman MD FRCSC for english language editing.

Author information

Authors and Affiliations

  1. Département d’Informatique et de Recherche Opérationnelle (DIRO), Université de Montréal, Québec, Montréal, Canada

    Dac Cong Tai Nguyen & Max Mignotte

  2. Eiffel Medtech Inc., Québec, Montréal, Canada

    Dac Cong Tai Nguyen, Said Benameur & Frédéric Lavoie

  3. Orthopedic Surgery Department, Centre Hospitalier de l’Université de Montréal (CHUM), Québec, Montréal, Canada

    Frédéric Lavoie

Authors
  1. Dac Cong Tai Nguyen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Said Benameur
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Max Mignotte
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Frédéric Lavoie
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Dac Cong Tai Nguyen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, D.C.T., Benameur, S., Mignotte, M. et al. 3D biplanar reconstruction of lower limbs using nonlinear statistical models. Med Biol Eng Comput 61, 2877–2894 (2023). https://doi.org/10.1007/s11517-023-02882-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02882-3

Keywords