An efficient Riemannian statistical shape model using differential coordinates: With application to the classification of data from the Osteoarthritis Initiative
Graphical abstract
Introduction
Statistical models of shape have been established as one of the most successful methods for understanding the geometric variability of anatomical structures. Shape modeling is of particular interest in image guided diagnosis where morphological changes of anatomies have been hypothesized to be linked to various disorders. Based on a set of training shapes, statistical shape models efficiently parametrize the geometric variability of the biological objects under study. This in turn is not only useful in imposing shape constraints in synthesis and analysis problems but also in understanding the processes behind growth and disease. Hence, statistical shape modeling has moved well beyond its de-facto application of automatic image segmentation (Cootes, Taylor, Cooper, Graham, 1995, Seim, Kainmueller, Lamecker, Bindernagel, Malinowski, Zachow, 2010, Kainmüller, Lamecker, Heller, Weber, Hege, Zachow, 2013, Saito, Nawano, Shimizu, 2016). For example, descriptors based on statistical shape modeling have proven effective for predicting the onset and progression of osteoarthritis (Bredbenner, Eliason, Potter, Mason, Havill, Nicolella, 2010, Neogi, Bowes, Niu, Souza, Vincent, Goggins, Zhang, Felson, 2013, Thomson, O’Neill, Felson, Cootes, 2015, Thomson, O’Neill, Felson, Cootes, 2016) and in anthropometric studies (Robinette, Daanen, Paquet, 1999, Hasler, Stoll, Sunkel, Rosenhahn, Seidel, 2009). An overview of the numerous contributions as well as the wide range of applications can be found in Heimann and Meinzer (2009); Sarkalkan et al. (2014); Brunton et al. (2014); Lamecker and Zachow (2016).
In the simplest case, the space of shapes is represented by an Euclidean space, e.g. by describing a shape ranging from a sparse set of landmarks up to a dense collection of boundary points. This structure then allows linear statistical tools like principal component analysis (PCA) to be applied to the training data resulting in fast and simple algorithms for the analysis and synthesis of shapes. However, the quality of the model depends on the validity of the assumption that instances of the object class being modeled lie in (or are at least well-approximated by) a flat Euclidean space. This assumption is a poor choice for data with a large spread or within regions of high curvature in shape space (Huckemann, Hotz, Munk, 2010, Zhang, Heeren, Rumpf, Smith, 2015) and, thus, is considered a limiting factor for the ability to represent natural biological variability in populations (cf. for example Davis et al., 2010 and the references therein).
More recently, many exciting ideas and new theoretical insights for treating manifold valued data have been presented addressing the growing demand for nonlinear multivariate statistics accounting for the spaces’ intrinsic structure (see Section 1.1). In spite of these developments, linear methods are still the most widely used approaches to 3D statistical shape modeling (Sarkalkan et al., 2014). This hesitant adoption of nonlinear methods has been linked to the lack of numerical robustness (Heimann and Meinzer, 2009). Moreover, while the development of specific fast-converging algorithms for the underlying optimization problems is still an ongoing field of research, many medical applications such as computer-assisted intervention or interactive segmentation require near-realtime response rates which prohibits the use of computationally expensive, nonlinear models.
In this work, we address the challenge of developing a statistical shape model that accounts for the non-Euclidean nature inherent to (anatomical) shapes and at the same time offers fast and numerically robust processing. This work makes three major contributions:
First, we formulate a continuous and physically motivated notion of shape space based on deformation gradients. Other than Euclidean structures that pronounce shape variations with large displacements even if they are near-isometries, our differential representation puts the (local) geometric changes into focus, which is mechanically sound. Furthermore, the continuous formulation allows for discretization that is consistent w.r.t. resampling and is generalizable to other representations of shape.
Second, we endow the differential coordinates with a Riemannian structure that comes with both: Excellent theoretical properties and closed-form expressions yielding simple and efficient algorithms. In particular, the structure stems from a product Lie group of stretches and rotations for which we present a bi-invariant metric. We demonstrate that our model is better able to capture nonlinear deformations, e.g. inherent to articulation and pathological morphology. Specifically, we perform a quantitative comparison to state-of-the-art approaches on datasets of distal femora and human body shapes showing superior performance in terms of specificity and generalization ability.
Third, we derive a statistical shape descriptor that captures systematic differences in shape between normal and diseased subpopulations. We show that these descriptors are better suited for classification tasks in qualitative as well as quantitative evaluation for osteoarthritis of the distal femur.
In medical image processing, the construction and application of statistical models of shape in a linear vector space is a standard technique with long tradition (Cootes, Taylor, Cooper, Graham, 1995, Kelemen, Székely, Gerig, 1999). It is still an active area of research how this technique can be generalized to shape spaces with non-Euclidean structures. One line of work to introduce nonlinearity is based on the so called kernel methods that embed data into a potentially infinite-dimensional feature space and perform linear statistic analysis there (Kirschner et al., 2011). While this approach is widely applicable, the choice of a kernel is however non-trivial and does usually depend on the specific application. Furthermore, there exists in general no (exact) pre-image in shape space so that one can only settle for approximate reconstructions thereof (Mika et al., 1999). This in turn can impede the synthesis of new shapes considerably. A comparison between different kernel methods is given in Rathi et al. (2006).
More sophisticated methods to account for the non-Euclidean structure adopt wider geometrical as well as physical perspectives on shape spaces, e.g. based on the notion of the Hausdorff distance (Charpiat et al., 2006), elastic deformations (Rumpf, Wirth, 2011, von Tycowicz, Schulz, Seidel, Hildebrandt, 2015, Zhang, Heeren, Rumpf, Smith, 2015), and viscous flows (Fuchs, Jüttler, Scherzer, Yang, 2009, Heeren, Rumpf, Wardetzky, Wirth, 2012, Brandt, von Tycowicz, Hildebrandt, 2016). For a discussion of the various concepts we refer to the chapter by Rumpf and Wirth (2011b). The price to pay for such variational approaches is that one has to solve nonlinear, high-dimensional optimization problems that suffer from numerical instability and local minima.
For the case of Riemannian shape spaces the strength of shape variations can be measured in terms of geodesic distances. Averages in these spaces have been presented by Fréchet (1948) and were further analysed by Karcher (1977). There exists also a natural linear representation of shapes in the tangent space of the mean via the geodesic logarithmic map. Fletcher et al. (2004) employ this structure to extend the PCA to the manifold setting which is thus referred to as principal geodesic analysis (PGA). While intrinsic distances between training shapes and the mean are preserved under the logarithmic map this is generally not the case for the distances between shapes. Exact PGA (Sommer et al., 2010) as well as geodesic PCA (Huckemann et al., 2010) both attempt to capture this additional information by employing the true intrinsic distances. In general, geodesic calculus on manifold shape spaces is hard to carry out in practice because most operations do not admit closed-form solutions.
Another line of work, closely related to our framework, alleviates these challenges by explicitly modeling shapes as a collection of primitives equipped with a Lie group structure (Fletcher, Lu, Joshi, 2003, Freifeld, Black, 2012, Hefny, Okada, Hori, Sato, Ellis, 2015). However, contrary to our continuous model, these methods are posed in a purely discrete setting that is not readily applicable to different shape representations. Furthermore, the lack of a continuous counterpart leaves questions as for the consistency and convergence of the discrete model unanswered. These aspects are not only of concern in the refinement limit: Results will depend on the resolution and anisotropy of the sampling.
Section snippets
Differential coordinates
In the following, we regard shapes as boundaries of physical objects belonging to a particular class of anatomical structures so that they can be represented as a collection of (orientation-preserving) embeddings of a common reference where denotes the space of (smooth enough) deformations . While some applications require the interior of the object, e.g. joint shape and appearance analysis, typically only the object’s surface is of importance. For now, we let denote
Statistical framework
We now develop the methodology for statistical analysis of the differential representation presented in the previous section. To this end, we derive the intrinsic mean (Fréchet, 1948) of a set of points in the space of differential coordinates and then apply principal geodesic analysis (Fletcher et al., 2004) to extract the dominant modes of variation. This section assumes familiarity with Lie groups and we refer to O’Neill (1983) for an introduction. Additionally, we provide detailed C++
Numerical treatment
In this section, we present a consistent and convergent finite element based discretization of our framework for volumetric objects together with a dimension reduction to surfaces. In addition, we provide details on the algorithmic aspects involved in the construction of our shape model.
Experiments and results
All experiments are done employing a fixed balancing weight ω ≡ 1 (see Section 3.2) which empirically shows the best performance in our classification experiments.
Conclusion
In this work, we presented a novel approach for nonlinear statistical analysis of shapes. Our formulation employs a differential representation of shape that incorporates a natural (local) measure of deformation in terms of stretch tensors. We furthermore endow this representation with a nonlinear Riemannian structure that provides our shape space with strong theoretical properties. In particular, physically invalid tensors are at an infinite distance from any element in our space. Despite its
Acknowledgements
This work was supported by the DFG project KneeLaxity: Dynamic multi-modal knee joint registration (EH 422/1-1), BMBF project TOKMIS: Treating Osteoarthritis in Knee with Mimicked Interpositional Spacer (01EC1406E), and BMBF MODAL - MedLab. Furthermore we are grateful for the open-access datasets OAI2
References (51)
- et al.
Statistical shape modeling describes variation in tibia and femur surface geometry between control and incidence groups from the osteoarthritis initiative database
J. Biomech.
(2010) - et al.
Review of statistical shape spaces for 3D data with comparative analysis for human faces
Comput. Vis. Image Underst.
(2014) - et al.
Active shape models-their training and application
Comput. Vis. Image Underst.
(1995) - et al.
Morphometry of anatomical shape complexes with dense deformations and sparse parameters
Neuroimage
(2014) - et al.
Statistical shape models for 3d medical image segmentation: a review
Med. Image Anal.
(2009) - et al.
Unbiased diffeomorphic atlas construction for computational anatomy
NeuroImage
(2004) - et al.
Joint optimization of segmentation and shape prior from level-set-based statistical shape model, and its application to the automated segmentation of abdominal organs
Med. Image. Anal.
(2016) - et al.
Statistical shape and appearance models of bones
Bone
(2014) - et al.
Log-Euclidean metrics for fast and simple calculus on diffusion tensors
Magn. Reson. Med.
(2006) - et al.
Overview of the geometries of shape spaces and diffeomorphism groups
J. Math. Imaging Vis.
(2014)
The Helmholtz–Hodge decomposition—a survey
IEEE Trans. Vis. Comput. Graph.
FAUST: Dataset and evaluation for 3D mesh registration
CVPR
Deformation transfer for detail-preserving surface editing
VMV
Geometric flows of curves in shape space for processing motion of deformable objects
Comput. Graph. Forum.
Distance-based shape statistics
ICASSP
Statistical Models of Shape: Optimisation and Evaluation
Population shape regression from random design data
Int. J. Comput. Vis.
Principal geodesic analysis for the study of nonlinear statistics of shape
IEEE Trans. Med. Imaging
Statistics of shape via principal geodesic analysis on lie groups
CVPR
Les éléments aléatoires de nature quelconque dans un espace distancié
Annales de l’institut Henri Poincaré
Lie bodies: A manifold representation of 3d human shape
ECCV
Shape metrics based on elastic deformations
J. Math. Imaging Vis.
Procrustes methods in the statistical analysis of shape
J. Roy. Statist. Soc. Series B (Methodological)
Discrete Shells
ACM SIGGRAPH/Eurographics SCA
A statistical model of human pose and body shape
Cited by (47)
Learning continuous shape priors from sparse data with neural implicit functions
2024, Medical Image AnalysisAn operator-splitting approach for variational optimal control formulations for diffeomorphic shape matching
2023, Journal of Computational PhysicsDynamic multi feature-class Gaussian process models
2023, Medical Image AnalysisPrincipal polynomial shape analysis: A non-linear tool for statistical shape modeling
2022, Computer Methods and Programs in BiomedicineCitation Excerpt :Nevertheless, there are some methods which solve (some of) these problems. For example, Von Tycowicz et al. (2018) presented a promising method for which the inverse problem can be solved at linear cost [29]. They derived a physically motivated shape representation based on differential coordinates to take the deformation within local neighborhoods into account.
Rigid motion invariant statistical shape modeling based on discrete fundamental forms data from the osteoarthritis initiative and the Alzheimer’ disease neuroimaging initiative
2021, Medical Image AnalysisCitation Excerpt :Exploiting the redundancy of such representations present e.g. in articulated motion, these approaches provide low-dimensional encodings of deformations. Considering the limit case of triangle-wise supported polyaffine/-rigid deformations is similar to simplicial maps underlying the construction in (Freifeld and Black, 2012; von Tycowicz et al., 2018; Ambellan et al., 2019b) as well as our setup. However, the latter employ differential characterizations of such maps that remove translational components and put local geometric variability into focus.
- 1
Equal contribution