Spline-based elastic image registration: integration of landmark errors and orientation attributes
Introduction
Image registration based on point landmarks plays a major role in neurosurgery planning and intraoperative navigation as well as other medical applications. While rigid and affine schemes can only describe global geometric differences between images, elastic schemes can additionally cope with local differences. Reasons for local geometric differences are different anatomy (or pathology), scanner-, or patient-induced distortions, as well as intraoperative deformations due to surgical interventions.
An often applied method for point-based elastic image registration is based on thin-plate splines. This approach has been introduced into medical image analysis by Bookstein [5]. Evans et al. [14] applied this scheme to 3D medical images. For an application to 2D aerial image registration see Goshtasby [17]. Thin-plate splines have a physical motivation, are mathematically well-founded, and are moreover computationally efficient. Alternative splines based on the Navier equation, which have been named elastic body splines, have recently been introduced by Davis et al. [12]. Extensions of point-based elastic schemes which allow to include additional attributes at landmarks have been proposed by Bookstein and Green [8] and Mardia and Little [20]. The combination of thin-plate splines with mutual information as similarity measure for the purpose of refining initially coarsely specified landmarks has been proposed by Meyer et al. [27].
In all of these approaches from above the interpolation case has been treated. This means that corresponding landmarks are forced to match exactly and thus it is (implicitly) assumed that the landmark positions are known exactly. This assumption, however, is unrealistic since landmark extraction is always prone to error. Approximation schemes, on the other hand, allow to incorporate landmark errors. The error information is used to control the influence of the landmarks on the registration result, which is important in clinical applications. Also, the resulting computational scheme is more robust in comparison to an interpolation approach. However, it seems that approximation schemes have so far not been a focus of research (but see [6], [10], [29] for exceptions). A more detailed discussion of these schemes is given in Section 2 below.
This contribution is concerned with an approximation scheme for point-based elastic image registration using thin-plate splines. Central to this scheme is a well-defined minimizing functional for which the solution can be stated analytically. Therefore, we yield an efficient computational scheme for determining the transformation between two images. In earlier work, we have introduced an approach that allows incorporation of isotropic as well as anisotropic landmark errors and we have proposed a scheme for estimating landmark localization uncertainties directly from the image data [29], [30], [31], [33]. The term “anisotropic errors” means that the errors are different in different directions, e.g., given an isotropic dataset. In this contribution, we suggest a generalization of our work which allows integration of additional attributes at point landmarks [32]. By this, additional knowledge is used to further improve the registration result without the necessity of specifying additional landmarks. In our case, we consider orientation attributes at corresponding points. Generally, these attributes characterize the local orientation of the contours at the landmarks. In previous work on the incorporation of additional attributes, Bookstein and Green [8] have represented orientations by additional points close to the landmarks, thus they used a finite difference scheme. Mardia and Little [20] have proposed a scheme based on the method of kriging where exact orientations are incorporated. Their scheme requires the orientation vectors to be unit vectors. This imposes constraints which may not be desired. The approach we propose also includes exact orientations, however, in comparison to Ref. [20] the orientation vectors need not to be normalized to unit vectors. This is achieved by representing the constraints due to the orientations through scalar products. Additionally, we treat the interpolation as well as the approximation case. In particular, we propose a combined scheme that integrates isotropic as well as anisotropic errors together with orientation attributes. Also, we extend the domain of application of our scheme to the important case of preserving rigid structures (such as bone) embedded in an otherwise elastic material. It seems that this application has so far not gained much attention in previous work on point-based registration using attributes (but see [20]). In comparison to other schemes such as Little et al. [23] a full segmentation of the rigid structures is not necessary for our approach.
The remainder of this contribution is organized as follows. In the next section, we discuss in more detail related work on approximation schemes for point-based non-rigid image registration. Then, we describe our approach based on thin-plate splines which integrates anisotropic landmark errors and orientation attributes. The applicability of the approach is demonstrated for synthetic data as well as real tomographic images of the human brain.
Section snippets
Related work
In this section, we discuss approximation schemes for point-based non-rigid image registration. For other approaches to medical image registration we refer to recent reviews by Maintz and Viergever [24], Lester and Arridge [19], Toga [36], and Rohr [34]. Note, that in the field of computer vision approximation schemes have previously been considered for the reconstruction of surfaces from sparse depth data (e.g. [4], [9], [18], [35]) or for representing and modifying facial expressions [3].
For
Thin-plate splines with landmark errors and additional attributes
We now describe our approach to elastic image registration based on thin-plate splines. This approach incorporates landmark errors as well as orientation attributes at landmarks. While the landmark errors represent statistical information about the uncertainty of landmark localization, the orientation attributes represent geometric information about the contours at the landmarks. Below, we first briefly review our scheme incorporating anisotropic landmark errors and then describe an extension
Experimental results
We demonstrate the applicability of our approach using synthetic data as well as real tomographic images of the human brain. In the first two experiments we have incorporated either anisotropic landmark errors only or orientation attributes only. For the last two experiments we have integrated both landmark errors (isotropic as well as anisotropic errors) and orientation attributes.
In the first example, we register the 2D MR brain images of different patients displayed in Fig. 2. We have used
Summary and future work
In this contribution, we have proposed an approach to elastic registration of medical images that is based on point landmarks and additional attributes. Our scheme is based on a minimizing functional which covers the full range from interpolation to approximation. Since the solution can be stated analytically we yield an efficient computational scheme. Central to this work is the integration of anisotropic landmark errors and orientation attributes at landmarks. By this we incorporate
Acknowledgements
This work has been supported by Philips Research Hamburg, project IMAGINE (IMage- and Atlas-Guided Interventions in NEurosurgery). We thank R. Sprengel for his contribution to this work. The original MR images have kindly been provided by Philips Research Hamburg and W.P.Th.M. Mali, L. Ramos, and C.W.M. van Veelen (Utrecht University Hospital) via ICS-AD of Philips Medical Systems Best. We thank the reviewers for their constructive comments.
References (40)
- et al.
A method for approximating the density of maximum-likelihood and maximum a posteriori estimates under a Gaussian noise model
Med. Image Anal.
(1998) - et al.
Image warping by radial basis functions: application to facial expressions
Comput. Vision, Graphics, Image Process.
(1994) - et al.
Registration of 3D multi-modality medical images using surfaces and point landmarks
Pattern Recogn. Lett.
(1994) An implementation of a computational theory of visual surface interpolation
Comput. Vision, Graphics, Image Process.
(1983)- et al.
A survey of hierarchical non-linear medical image registration
Pattern Recognition
(1999) - et al.
Deformations incorporating rigid structures
Comput. Vision and Image Understanding
(1997) - et al.
A survey of medical image registration
Med. Image Anal.
(1998) - et al.
Snakes and splines for tracking non-rigid heart motion
- et al.
On three-dimensional surface reconstruction methods
IEEE Trans. Pattern Anal. Mach. Intell.
(1991) Principal warps: thin-plate splines and the decomposition of deformations
IEEE Trans. Pattern Anal. Mach. Intell.
(1989)
Four metrics for image variation
Landmark methods for forms without landmarks: morphometrics of group differences in outline shape
Med. Image Anal.
A feature space for edgels in images with landmarks
J. Math. Imaging Vision
Visual surface reconstruction using sparse depth data
Volumetric transformation of brain anatomy
IEEE Trans. Med. Imaging
A physics-based coordinate transformation for 3-D image matching
IEEE Trans. Med. Imaging
Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces
R.A.I.R.O. Anal. Numér.
Warping of a computerized 3-D atlas to match brain image volumes for quantitative neuroanatomical and functional analysis
Medical Imaging V: Image Processing, 1991, San Jose, CA
Proc. SPIE
Elastic medical image registration using orientation attributes at Landmarks
Finite element approach to warping of brain images
Medical Imaging 1994: Image Processing, 15–18 Feb. 1994, Newport Beach, CA, USA
Proc. SPIE
Cited by (46)
Registration and matching method for directed point set with orientation attributes and local information
2020, Computer Vision and Image UnderstandingCitation Excerpt :Fig. 2(c) shows the transformation results of the TPS function in Rohr et al. (2003) and Fig. 2(d) shows the transformation result of the TPS function in this paper. From the results labeled by the circle, it can be seen that the orientation transformed by the method in Rohr et al. (2003) is opposite from the template point, and the results of the TPS function mapping in this paper are more accurate. Figs. 3–4 show the TPS function transform results in the case of many-to-many correspondences.
Computational pipelines and workflows in bioinformatics
2018, Encyclopedia of Bioinformatics and Computational Biology: ABC of BioinformaticsA robust hybrid method for nonrigid image registration
2011, Pattern RecognitionCitation Excerpt :Typical approaches include the softassign methods [1,2], relaxation labeling methods [18,19], and the registration based on the Expectation-Maximization (EM) algorithm [20,21]. Similar to fuzzy correspondence methods, a group of methods integrate anisotropic landmark errors into thin-plate spline (TPS) based elastic registration, resulting to approximate TPS to cope with non-exact matches [22–24]. Some researchers exploited the intensity features to find correspondence robustly.
Automatic mosaicking and volume assembly for high-throughput serial-section transmission electron microscopy
2010, Journal of Neuroscience MethodsA qualitative and a quantitative analysis of an auto-segmentation module for prostate cancer
2009, Radiotherapy and OncologyCitation Excerpt :This auto-segmentation algorithm, originally developed by Dr. Scholz at the Charité-Hospital (Berlin, Germany), belongs to the category of the rule-based/logic-based algorithms (similar to the work of Archip et al. [7], Betke et al. [8] and Rohr et al. [9]); other approaches have also been proposed by several research groups: holistic methods [10–13], specialised segmentation techniques [14] and semi-automatic methods [15–17].
- 1
Now with Medset Medizintechnik GmbH, Hamburg.