A benchmark study of automated intra-retinal cyst segmentation algorithms using optical coherence tomography B-scans

https://doi.org/10.1016/j.cmpb.2017.10.010Get rights and content

Highlights

  • The different automated OCT intra-retinal cyst segmentation methods are reviewed and comparatively analysed for quantitative and qualitative benchmarking purposes.

  • A modular approach to standardize the existing cyst segmentation methods is presented for methodological benchmarking purposes.

  • The methods are analysed for their scalability across image acquisition systems using publicly available cyst segmentation challenge dataset.

  • Factors on the automated cyst segmentation are identified and future directions to improve automated detection and diagnosis of retinal pathologies are discussed.

Abstract

(Background and objectives)

Retinal cysts are formed by accumulation of fluid in the retina caused by leakages from inflammation or vitreous fractures. Analysis of the retinal cystic spaces holds significance in detection and treatment of several ocular diseases like age-related macular degeneration, diabetic macular edema etc. Thus, segmentation of intra-retinal cysts and quantification of cystic spaces are vital for retinal pathology and severity detection. In the recent years, automated segmentation of intra-retinal cysts using optical coherence tomography B-scans has gained significant importance in the field of retinal image analysis. The objective of this paper is to compare different intra-retinal cyst segmentation algorithms for comparative analysis and benchmarking purposes.

(Methods)

In this work, we employ a modular approach for standardizing the different segmentation algorithms. Further, we analyze the variations in automated cyst segmentation performances and method scalability across image acquisition systems by using the publicly available cyst segmentation challenge dataset (OPTIMA cyst segmentation challenge).

(Results)

Several key automated methods are comparatively analyzed using quantitative and qualitative experiments. Our analysis demonstrates the significance of variations in signal-to-noise ratio (SNR), retinal layer morphology and post-processing steps on the automated cyst segmentation processes.

(Conclusion)

This benchmarking study provides insights towards the scalability of automated processes across vendor-specific imaging modalities to provide guidance for retinal pathology diagnostics and treatment processes.

Introduction

A retinal cyst is a fluid-filled space in the retina. Medical studies show that visual acuity can be accurately predicted from the volume of retinal cystic fluids and their relative location in the retina  [1]. Also, Cystoid macular edema (CME), caused by cysts in the retinal macular region, is the leading cause of central vision loss in the world today. CME develops when excess fluid accumulates within the retinal macula, which may lead to disruption of the retinal vessel barrier owing to pathologies such as age related macular degeneration, diabetic retinopathy, retinal vein occlusion and ocular inflammation. This process of fluid accumulation in retina can reduce macular retinal function [2] and can lead to irreversible blindness globally in individuals belonging to the economically productive age-group, irrespective of gender and demographics [3]. Further, visual acuity impairment due to CME can be correlated to the volume of the cystic fluid spaces and their location in retinal tissue  [1]. Thus, automated quantification of retinal pathology severity is imperative towards timely retinal diagnostics and treatment.

CME can be clinically characterized using Optical Coherence Tomography (OCT) images [4]. OCT is a noninvasive imaging modality that is widely used for resolving internal structures of biological tissues, and for visualizing cross-sectional high-resolution images of the retina [5]. OCT images are extensively utilized for diagnostic and prognostic purposes for several retinal pathologies with manifestations that impact the intra-retinal micro-structure, such as cysts, exudates and retinal disorganization (see Fig. 1). However, one primary limitation of the OCT images is the manual assessment time required for analyzing the large volumes of image data per patient. This necessitates the development of automated intra-retinal and cyst segmentation/quantification methods to speed up the pathology characterization and diagnostic process. Several automation methodologies have been proposed in the recent past to address automated analysis of OCT images [1], [6], [7], [8], [9], [10]. The key challenges posed by the OCT images to most existing methods for automated segmentation of cystic regions include pixel-level variabilities due to noise, image intensity variations, varied cyst morphology, confounding retinal structures and complex pathologies.

Based on the existing methods for automated cyst segmentation from OCT images, a generic methodological framework is proposed in Fig. 2. This framework consists of four main steps: (1) pre-processing; (2) retinal layer segmentation; (3) cyst segmentation and (4) post-processing. Since OCT images contain varying degrees of additive speckle noise, a pre-processing module is required for quality enhancement and equalization of the OCT images. Finally, the post-processing step is implemented to reduce the incorrectly segmented non-cystic regions (i.e., false positive regions). In this work, existing automated retinal cyst segmentation methods are standardized based on the work-flow shown in Fig. 2 and comparatively analyzed to evaluate the significance of the automated methods with respect to input data and output metrics.

This paper makes three key contributions. First, a modular approach to standardize existing OCT cyst segmentation methods is presented for methodological benchmarking purposes. The methodological contributions from significant automated OCT cyst segmentation methods are reviewed and comparatively discussed. Second, quantitative and qualitative analysis experiments are presented for evaluation of the existing automated OCT cyst segmentation methods. We observe that supervised OCT segmentation methods achieve higher cyst segmentation recall when compared to unsupervised approaches with degradation in segmentation precision across data sets with variable scan qualities. Third, OCT images from two different image acquisition systems are comparatively analyzed for scalability limitations owing to the image-level variabilities introduced by imaging systems. Such exhaustive analysis regarding the scalability of OCT cyst segmentation methods in terms of methodological and input data variations has not been presented so far. This work provides novel insights into the limitations of automated cyst segmentation tasks for retinal diagnostic and screening purposes.

The organization of this paper is as follows. In Section 2, the materials and evaluation metrics used for comparison study are presented. In Section 3, methods considered for comparative study are briefly reviewed. In Section 4, the experimental setup is discussed. In Section 5, the experimental results of the proposed methods are presented. Conclusions and discussion regarding the comparative assessment of the automated intra-retinal cyst segmentation along with future research directions are presented in Section 6.

Section snippets

Dataset

This work comparatively analyzes existing automated intra-retinal OCT cyst segmentation methods on the publicly available OPTIMA cyst challenge OCT dataset [11]. This dataset contains OCT scans obtained from CME subjects using four different imaging systems, namely Zeiss Cirrus, Nidek, Spectralis Heidelberg and Topcon. In this work, OCT scans from Cirrus and Spectralis image acquisition systems are analyzed, since the data sets from these systems demonstrate moderate to severe pathological

Methods

The existing automated OCT cyst segmentation methods can be classified into two categories: semi-automated and fully automated. The semi-automated methods require manual intervention to define initial markers for each cyst. These methods are time and manual labor intensive owing to the large numbers of frames that need to be manually examined to define the markers [13], [14]. Thus, fully automated segmentation methods were developed to overcome these limitations. Fig. 3 highlights this

Experimental setup

Experiments analysis for automated segmentation of cysts using the methods described above is performed on OCT images from Spectralis and Cirrus imaging systems, and the segmentation outcomes are comparatively evaluated. For standardization purposes, the pre-processing module for the removal of additive speckle noise from the OCT B-scans precedes the retinal layer segmentation step. For each method described in Section 3, the automated segmentation algorithm module is followed by

Results and analysis

The quantitative and qualitative comparative assessment of the automated cyst segmentation methods is presented below. Since low precision is indicative of FPs (over-detection) and low recall is indicative of missing patients with abnormalities (under-detection), high precision and high recall values are desired for an ideal automated cyst segmentation method.

Discussion and conclusion

This paper presents a comparative assessment of existing automated intra-retinal cyst segmentation methods on OCT B-scans. Our standardized methodological modules and cyst segmentation experiments demonstrate that variability factors such as pixel intensity variations, noise, blood vessel shadows and retinal layer distortions can impact the automated cyst segmentation accuracies. In addition, pre-processing and post-processing steps are found to play a vital role in automated cyst segmentation

Acknowledgments

This work was supported by the Science and Engineering Research Board (Department of Science and Technology, India) through project funding EMR/2016/002677.

Authors would like to thank Christian Doppler Laboratory for Ophthalmic Image Analysis, Department of Ophthalmology, Medical University of Vienna for providing data and necessary inputs for this paper.

References (29)

  • S. Sivaprasad et al.

    Prevalence of diabetic retinopathy in various ethnic groups: a worldwide perspective

    Surv. Ophthalmol.

    (2012)
  • C. Tomasi et al.

    Bilateral filtering for gray and color images

    Computer Vision, 1998. 6th International Conference on

    (1998)
  • G.R. Wilkins et al.

    Automated segmentation of intraretinal cystoid fluid in optical coherence tomography

    Biomed. Eng. IEEE Trans.

    (2012)
  • S.R. Irvine

    A newly defined vitreous syndrome following cataract surgery

    Am. J. Ophthalmol.

    (1953)
  • P.G. Garg, et al., Cystoid macular edema,...
  • D. Huang et al.

    Optical coherence tomography

    Science

    (1991)
  • W. Wieclawek

    Automatic cysts detection in optical coherence tomography images

    Mixed Design of Integrated Circuits & Systems (MIXDES), 2015 22nd International Conference

    (2015)
  • G.N. Girish et al.

    Automated segmentation of intra-retinal cysts from optical coherence tomography scans using marker controlled watershed transform

    2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

    (2016)
  • S. Roychowdhury et al.

    Automated localization of cysts in diabetic macular edema using optical coherence tomography images

    Engineering in Medicine and Biology Society (EMBC), 2013 35th Annual International Conference of the IEEE

    (2013)
  • A. Lang et al.

    Automatic segmentation of microcystic macular edema in oct

    Biomed. Opt. Express

    (2015)
  • L. Zhang et al.

    Automated segmentation of intraretinal cystoid macular edema for retinal 3D oct images with macular hole

    Biomedical Imaging (ISBI), 2015 IEEE 12th International Symposium on

    (2015)
  • Optima cyst segmentation challenge, (https://optima.meduniwien.ac.at/research/challenges/)...
  • L.R. Dice

    Measures of the amount of ecologic association between species

    Ecology

    (1945)
  • D.C. Fernandez

    Delineating fluid-filled region boundaries in optical coherence tomography images of the retina

    Med. Imaging IEEE Trans.

    (2005)
  • Cited by (13)

    • A cascaded convolutional neural network architecture for despeckling OCT images

      2021, Biomedical Signal Processing and Control
      Citation Excerpt :

      Despeckling is used as a preprocessing step in many OCT image analysis tools. Despeckling improves the signal-to-noise ratio of images and as a result, retinal structures and boundaries between the layers can be seen more clearly [1]. It also improves visualization of pathological conditions like retinal edema, cystoid macular edema, pigment epithelial detachment and other kinds of lesions [2–4].

    • A supervised joint multi-layer segmentation framework for retinal optical coherence tomography images using conditional random field

      2018, Computer Methods and Programs in Biomedicine
      Citation Excerpt :

      Ocular diseases also lead to significant changes in the tissue morphology. In AMD, the drusen deposits in the RPE layer lead to irregularities and undulations in the RPEin boundary [4] as depicted in Fig. 2 b. DME is characterized by the presence of fluid-filled regions [9],[10] in the OPL and INL layers around the macula leading to the swelling of the retinal tissue as shown in Fig. 2c. In this paper we extend our preliminary work in [11] by refining the method, adapting it to images with fluid-filled regions associated with DME and finally presenting a more comprehensive experimental evaluation with cross-testing across datasets acquired using different OCT imaging scanners.

    • Automatic macular edema identification and characterization using OCT images

      2018, Computer Methods and Programs in Biomedicine
      Citation Excerpt :

      In the work of Moura et al. [22], a method for the automatic identification of intraretinal fluid regions was designed based on a set of features that characterize the analyzed regions, including intensity and texture-based features. Girish et al. [23] proposed a benchmark study for the automated intra-retinal CME segmentation. In particular, the authors introduced a modular approach integrating different segmentation algorithms, facilitating the comparative analysis between the obtained quantitative and qualitative results of the experiments.

    • Stack generalized deep ensemble learning for retinal layer segmentation in Optical Coherence Tomography images

      2020, Biocybernetics and Biomedical Engineering
      Citation Excerpt :

      OCT is one of the most popular non-invasive imaging procedures used in ophthalmology [3]. OCT devices allow cross-sectional viewing of retina with localization of level of the lesion within the retina [4]. Most OCT device software allows precise detection of retinal boundaries in normal or minimally distorted retina [3].

    View all citing articles on Scopus
    View full text