Elsevier

Computers in Biology and Medicine

Volume 69, 1 February 2016, Pages 189-202
Computers in Biology and Medicine

Segmentation and optical flow estimation in cardiac CT sequences based on a spatiotemporal PDM with a correction scheme and the Hermite transform

https://doi.org/10.1016/j.compbiomed.2015.12.021Get rights and content

Highlights

  • A new framework for segmentation and optical flow estimation applied to cardiac CT sequences is proposed.

  • The segmentation is based on two processes: (1) A spatiotemporal point distribution model, and (2) A correction scheme that performs the segmentation of small details.

  • The optical flow estimation uses a bio-inspired model based on the Hermite transform to code local image features.

  • Both algorithms are combined to provide a powerful framework to assist the evaluation of heart mechanical problems.

Abstract

Purpose: The left ventricle and the myocardium are two of the most important parts of the heart used for cardiac evaluation. In this work a novel framework that combines two methods to isolate and display functional characteristics of the heart using sequences of cardiac computed tomography (CT) is proposed. A shape extraction method, which includes a new segmentation correction scheme, is performed jointly with a motion estimation approach.

Methods: For the segmentation task we built a Spatiotemporal Point Distribution Model (STPDM) that encodes spatial and temporal variability of the heart structures. Intensity and gradient information guide the STPDM. We present a novel method to correct segmentation errors obtained with the STPDM. It consists of a deformable scheme that combines three types of image features: local histograms, gradients and binary patterns. A bio-inspired image representation model based on the Hermite transform is used for motion estimation. The segmentation allows isolating the structure of interest while the motion estimation can be used to characterize the movement of the complete heart muscle.

Results: The work is evaluated with several sequences of cardiac CT. The left ventricle was used for evaluation. Several metrics were used to validate the proposed framework. The efficiency of our method is also demonstrated by comparing with other techniques.

Conclusion: The implemented tool can enable physicians to better identify mechanical problems. The new correction scheme substantially improves the segmentation performance. Reported results demonstrate that this work is a promising technique for heart mechanical assessment.

Introduction

Cardiac CT is currently one of the main types of radiological images used for heart analysis. Image slices showing the structural composition of the heart can be obtained with CT scanners [1]. The continued improvement of multidetector CT scanners has increased the potential of cardiac CT as clinical tool for heart imaging [2]. Since heart failure is one of the main health problems in developed and developing countries [3], tasks focused on cardiac analysis are of main concern for physicians. Several benefits of CT systems have been recognized to evaluate heart functions. Quantification of the ejection fraction, left and right ventricular functions, and wall motion evaluation are examples of typical uses of cardiac CT data [4].

The natural movement of the heart implies that its mechanical behavior must be evaluated as well. The spatiotemporal data obtained from cardiac CT studies can be used in computer-aided systems to evaluate the cardiac function, which has become essential over the past few years allowing faster assessments in the diagnosis process [5]. Since the left ventricle is vital for the proper functioning of the heart, it has become of major interest when analyzing cardiac images. In cardiac CT heart is commonly scanned at increments of 10% of the cardiac cycle providing a 4D dataset.

Shape extraction for volume measurement and motion estimation are the most typical tasks for heart evaluation where computer-based algorithms are extensively used [6], [7], [8], [9]. In this sense, development of new and most efficient algorithms, methods and mathematical models to analyze cardiac structures in CT data are activities of great interest for researchers.

In a general sense, basic processes like segmentation and optical flow estimation are primary steps before applying higher processes like image interpretation. Cardiac CT sequences constitute great challenges for segmentation and motion estimation algorithms. A typical problem when working with medical images is that they may vary considerably from one patient to another, from an image processing point of view. These variations are perceived as changes of contrast, size and geometrical shape. Cardiac CT images do not scape from these kinds of problems. Even though development of segmentation and optical flow estimation algorithms imposes issues that researches have tried to solve for several decades, the problem remains open. Recent thorough reviews of segmentation techniques applied to cardiac images [10], [11] conclude that shape extraction in heart images remains a very challenging task.

Active Shape Models (ASM) [12] have gained enormous popularity during the last twenty years and have been extensively used for modeling 2D and 3D data in cardiac imaging [13], [14], [15]. We opted for this approach due to its ability to represent specific shapes of an image. Problems regarding the contrast and shape variability can be easily overcome with ASM-based algorithms. Related literature deals with active shape models as methods to analyze cardiac images [14], [16]. ASM have also been combined with other methods with the aim of segmenting heart images [6]. Modifications of the original approach become necessary for improving the segmentation performance in some cases when the training samples are scarce [14]. However, issues of ASM are evident when the number of training samples is small. It is therefore necessary to design new strategies to overcome these problems.

The dynamic nature of the heart has motivated researchers to design image tracking algorithms to process cardiac images [17], [9], [18]. Tracking heart structures like the left ventricle or the myocardial wall can be performed using optical flow estimation methods which also allow computing the displacements of the cardiac structures in a sequence of images. For this purpose we used a differential approach defined in the Hermite transform (HT) space. The HT is a bio-inspired human vision model that decomposes an image with a set of orthogonal functions defined by the Hermite polynomials. Image patterns and structures relevant to human vision perception such as oriented edges and textures can be efficiently represented with the HT. The proposed optical flow estimation approach using the HT allows defining local image constraints and a multiresolution strategy within differential scheme and are relevant in a perceptual sense as described in [8].

Our main goal is to build a tool that may help physicians evaluate heart mechanical functions. In order to achieve our objective, we implemented a framework with two main processes: (1) A segmentation stage based on a statistical shape model and a new correction scheme, and (2) An optical flow estimation approach based on the Hermite transform. For the first process we have designed a novel correction method that substantially improves the segmentation performance. The goal of the new correction method is to refine the segmentation previously achieved with the statistical shape model. It consists of a deformable scheme that combines three image parameters: histogram, gradient and a binary pattern. These parameters are locally computed for each point of the contour of the segmentation. This work is entirely focused on analyzing sequences of cardiac CT images (2D + time). The algorithms are specifically applied to the left ventricle because it is responsible for some of the most vital functions of the heart. Cardiac CT studies are analyzed using the original axial view. Nevertheless, the method can be extended without major problems to other views. Although short and long axis are the most accepted views used for cardiac analysis, the original axial view is also very important for this task [19]. Combined results of both algorithms are presented. Vectors indicating the motion of the left ventricle are jointly used with contours of the segmentation. Results are evaluated with several image sequences using quantitative and qualitative analysis.

The rest of the paper is organized as follows. Material used in this work is described in Section 2. Methods are depicted in Section 3. Here, segmentation and optical flow approaches are included. Results and discussions are finally presented in Sections 4 and 5 respectively.

Section snippets

Materials

Our dataset consists of 40 sequences of cardiac CT images. Selected sequences used for evaluation show the left ventricle at half of the heart. The tomographic studies were acquired with a SIEMENS 16-slice CT system at 120 kVp of tube voltage and 900 mA. The scanner is composed of 128 detectors and is synchronized with the ECG signal. Each image has a size of 512×512 pixels, quantized to 12 bits per pixel. A contrast agent was also applied to each patient. Each sequence is composed by 10 frames

Statistical model of shape

Active shape models are one of the most powerful segmentation tools for medical image analysis. They consist of a statistical model that can be deformed within a specific range defined by a training set [12]. Here, shapes are represented using discrete points in the spatial domain. These points are commonly called landmarks when they are used to depict anatomical structures. Two main stages must be implemented in ASM algorithms: (1) Training of the statistical model, and (2) Segmentation of new

Segmentation

In this section we present results of the segmentation stage. To carry out the experiments we configured the algorithms as follows. A total of 50 points were used to represent the left ventricle in each frame of the sequence. It means that the spatiotemporal shape was built with 500 landmarks. A maximum of 25 iterations were enough to reach a stable solution in the ASM stage. Similarly, 5 iterations were used for the correction algorithm. The algorithm was initialized using the mean

Discussion and conclusions

We implemented a framework for the analysis of cardiac CT sequences using a shape extraction method and an optical flow estimation approach. The left ventricle was used as object of interest. We firstly performed the corresponding segmentation using a STPDM which consists of a trained statistical model that codes spatial and temporal information of the sequences. Errors of the segmentation were subsequently corrected using an algorithm that incorporates three image parameters for edge

Conflict of interest statement

None declared.

Acknowledgments

This work has been sponsored by the following UNAM Grant: PAPIIT IG100814. Leiner Barba-J thanks CONACYT–245976 for financial support, as well as Colciencias. Ernesto Moya-Albor and Jorge Brieva would like to thank the Faculty of Engineering of Universidad Panamericana (UP) for all support in this work.

References (44)

  • M.J. Black et al.

    The robust estimation of multiple motionsparametric and piecewise-smooth flow fields

    Comput. Vis. Image Underst.

    (1996)
  • B. Escalante-Ramírez et al.

    Advanced modeling of visual information processinga multiresolution directional-oriented image transform based on Gaussian derivatives

    Signal Process.: Image Commun.

    (2005)
  • A.M. Van Dijk et al.

    Image representation and compression with steered Hermite transforms

    Signal Process.

    (1997)
  • J.M. Budoff et al.

    Cardiac CT Imaging, Diagnosis of Cardiovascular Disease

    (2006)
  • J.P. OBrien et al.

    Anatomy of the heart at multidetector CTwhat the radiologist needs to know

    RadioGraph.

    (2007)
  • C. YF.

    The role of 3D wall motion tracking in heart failure

    Nat. Rev. Cardiol.

    (2012)
  • A. Mahnken et al.

    Cardiac CTcoronary arteries and beyond

    Eur. Radiol.

    (2007)
  • M.S. Krishnam et al.

    Left ventricular ejection fraction using 64-slice CT coronary angiography and new evaluation softwareinitial experience

    Br. J. Radiol.

    (2008)
  • O. Ecabert et al.

    Automatic model-based segmentation of the heart in CT images

    IEEE Trans. Med. Imaging

    (2008)
  • S. Faghih Roohi et al.

    4d statistical shape modeling of the left ventricle in cardiac MR images

    Int. J. Comput. Assist. Radiol. Surg.

    (2012)
  • D. Kang, J. Woo, P.J. Slomka, D. Dey, G. Germano, C.-C. Jay Kuo, Heart chambers and whole heart segmentation...
  • C. Petitjean et al.

    A review of segmentation methods in short axis cardiac MR images

    Med. Image Anal.

    (2001)
  • Cited by (13)

    • Optical flow estimation of coronary angiography sequences based on semi-supervised learning

      2022, Computers in Biology and Medicine
      Citation Excerpt :

      The estimated optical flow contains the spatial deformation information and motion information of the image sequences. In recent years, optical flow is widely used in medical image processing, especially in the fields related to cardiac diseases, such as image registration of CTA images [1], motion-based segmentation of CT image sequences [2], motion-compensated reconstruction of rotational angiography [3], temporal super-resolution of echocardiography [4,5] and ejection fraction estimation [6]. Our previous work [7] also analyzed the importance of temporal information for coronary artery segmentation in angiographic videos.

    • Bone SPECT/CT image fusion based on the discrete Hermite transform and sparse representation

      2022, Biomedical Signal Processing and Control
      Citation Excerpt :

      On the other hand, our purpose is to provide a tool for bone SPECT/CT image analysis as well. The DHT is a powerful technique which has been employed in many image processing applications [31], including texture analysis [15], segmentation [3,41,12], optical flow estimation [4], fusion [14] and others [16,6]. One of the main virtues of this tool is its ability to perform directional analysis which is a main issue found in other techniques such as classical wavelet-based methods.

    • Freezing front velocity estimation using image processing techniques

      2020, Measurement: Journal of the International Measurement Confederation
      Citation Excerpt :

      Most used approaches are based on matching primitives in different frames of the sequence and methods more elaborated as optical flow that compute velocity vectors in each pixel of the image [18]. Motion characterization is crucial in many applications such as robot navigation [19,20], security monitoring systems for detecting objects at high speeds [21], video monitoring for industrial applications [22–24] and medical applications ranging from characterization of cardiac motion [25] to the detection of pulse and respiratory rate from a video sequence [26,27]. Several studies have been done with ice samples analyzed with image processing techniques at both macroscopic and microscopic levels.

    • A perceptive watermarking approach applied to covid-19 imaging data

      2020, Proceedings of SPIE - The International Society for Optical Engineering
    View all citing articles on Scopus
    View full text