DilUnet: A U-net based architecture for blood vessels segmentation

https://doi.org/10.1016/j.cmpb.2022.106732Get rights and content

Highlights

  • Dilated convolutions ensure better feature transfer and accurate classification that result in a sensitivity boost.

  • Dilated convolutions serve as the main constituents of decoder, encoder blocks and skip connections.

  • Dilated convolutions of different rates are more effective at capturing dynamically complex blood vessels.

  • Proposed Weighted multi-output fusion retrieves high fidelity vascular map by extracting critical features from each output block.

Abstract

Background and objective

Retinal image segmentation can help clinicians detect pathological disorders by studying changes in retinal blood vessels. This early detection can help prevent blindness and many other vision impairments. So far, several supervised and unsupervised methods have been proposed for the task of automatic blood vessel segmentation. However, the sensitivity and the robustness of these methods can be improved by correctly classifying more vessel pixels.

Method

We proposed an automatic, retinal blood vessel segmentation method based on the U-net architecture. This end-to-end framework utilizes preprocessing and a data augmentation pipeline for training. The architecture utilizes multiscale input and multioutput modules with improved skip connections and the correct use of dilated convolutions for effective feature extraction. In multiscale input, the input image is scaled down and concatenated with the output of convolutional blocks at different points in the encoder path to ensure the feature transfer of the original image. The multioutput module obtains upsampled outputs from each decoder block that are combined to obtain the final output. Skip paths connect each encoder block with the corresponding decoder block, and the whole architecture utilizes different dilation rates to improve the overall feature extraction.

Results

The proposed method achieved an accuracy: of 0.9680, 0.9694, and 0.9701; sensitivity of 0.8837, 0.8263, and 0.8713; and Intersection Over Union (IOU) of 0.8698, 0.7951, and 0.8184 with three publicly available datasets: DRIVE, STARE, and CHASE, respectively. An ablation study is performed to show the contribution of each proposed module and technique.

Conclusion

The evaluation metrics revealed that the performance of the proposed method is higher than that of the original U-net and other U-net-based architectures, as well as many other state-of-the-art segmentation techniques, and that the proposed method is robust to noise.

Introduction

Various health-related complications can occur in the elderly. Diabetic retinopathy, age-related muscular degeneration, and glaucoma are a few of the diseases that are most prevalent in old age. Vessel segmentation of the fundus image can provide sufficient information for the diagnosis and prognosis of such ailments. Different retinal vessel features, such as length, width, thickness, angles, and branches, correspond to a particular health condition [1,2]. Early symptoms of most of these health conditions are linked to changes in the structure of the retinal blood vessels. These vasculatures are important in terms of medical image diagnostics, as they can be observed through a noninvasive procedure [3]. Therefore, by studying the structural changes in the blood vessels of the fundus, ophthalmologists can detect most of these age-related diseases in their early stages for effective treatment and prevention. However, manual labeling of the blood vessels from a fundus image is quite labor-intensive, as it requires an expert with sufficient knowledge in the field of ophthalmology and many hours of work. Moreover, due to human error and subjective judgment, different experts might label blood vessels according to their perception. Therefore, the traditional method of eye screening is inefficient and time-consuming.

Automatic segmentation of the vasculature of the retina is necessary to reduce costs and save time. However, due to the varying shape, size, and nature of vascular circuitry, its complexity and the low contrast of thin vessels against the background of the retina, automatic segmentation of blood vessels is a challenging task that requires extensive research, experience, and knowledge in the field of computer vision and image processing.

Over the last few years, much work has been conducted on vessel segmentation. However, due to the challenges discussed earlier, it remains a daunting task, and there are many unexplored possibilities. Therefore, in this paper, we propose an overall improved, U-net variant that performs better at correctly classifying vessels than most state-of-the-art, vessel segmentation methods. The following points are the main contributions of this work in the field of biomedical image segmentation:

  • 1.

    An improved U-net architecture that effectively utilizes dilated convolutions as the main form of convolutions that serve as the building blocks along with multi-input modules and multioutput modules to ensure better feature transfer and accurate classification, respectively. Our proposed method of blood vessel segmentation performed better than most of the other existing methods.

  • 2.

    We proposed weighted multioutput fusion, where the feature map from each decoder path is upsampled to image size resolution and then weighted and fused. This weighted output is then passed through a final layer of a 1 × 1 transpose convolution to obtain the final output. The weighted fusion allows us to extract mostly meaningful features from each classifier and to accurately identify the thin blood vessels, which greatly improves the overall sensitivity.

  • 3.

    The decoder, encoder blocks, and skip paths are comprised of dilated convolutions of different rates that are uniquely combined to capture blood vessels of different shapes and sizes, which yields overall better performance.

  • 4.

    Our method achieved the highest sensitivity against many SOTA blood vessel segmentation works and yields fairly competitive results under noisy data conditions, ensuring robustness.

The remainder of the paper is organized as follows: Section 2 discusses certain previous related works Section 3 provides a detailed description of the proposed methodology that is adopted and the architecture of the network Section 4 includes the test results, compares performances, and discusses differences among the related works, including an ablation study of the architecture Section 5 concludes this work.

Section snippets

Related works

The task of automated retinal image segmentation can be divided into two categories namely: supervised learning methods and unsupervised learning methods. Unsupervised methods include rule-based approaches that may utilize a matched filter and morphological operations to detect and segment the vessels. Zardadi et al. [4] applied the combination of adaptive thresholding and different morphological operations to segment blood vessels. In their work, first, images were enhanced in preprocessing;

Materials and methods

The original retinal images are RGB-colored images that are passed through a preprocessing pipeline to obtain the final grayscale, resized image for training. These images are fed to our proposed segmentation network for training, and the results are obtained. The process heavily utilizes image augmentation [33] for effective training and to overcome the problems of data limitation and overfitting. The model outputs the final prediction while taking into account the features from all the

Training

This model utilizes image augmentation to work with a limited amount of data [33]. Aside from the use of traditional data augmentation of horizontal/vertical flips and random rotations, we augmented our data with random addition of Gaussian noise, as well as elastic transformation and grid distortion. These operations of elastic transformation, flips, random rotations, and grid distortion are applied to both the image and the labels. This approach makes our model robust to vessels with

Conclusion

In this work, we investigated the U-net architecture for image segmentation and determined that there is still room for improvement. We designed a new set of modules that can be introduced into the already existing U-net for even better performance than the original U-net and most of its other variants. In summary, the main contributions of our work are presented as follows:

  • The added multioutput block with momentum-based output fusion, multiscale input, optimized implementation of dilated

Acknowledgments

The authors thank the editors and anonymous reviewers for their useful suggestions. This study was supported by the Teaching & Learning Reform of Graduate Education of Central South University, National Natural Science Foundation of China (Grant no. 61502537), Hunan Provincial Natural Science Foundation of China, and Fundamental Research Funds for the Central Universities of Central South University.

References (61)

  • T. Mcreynolds et al.

    “Color, Shading, and Lighting”, in Advanced Graphics Programming Using OpenGL

    (2005)
  • S.M. Pizer

    “Adaptive histogram equalization and its variations"

    Comput. Vision Graph. Image Process.

    (1987)
  • D. Nettleton

    Chapter 6 - Selection of Variables and Factor Derivation,” in Commercial Data M

    (2014)
  • X. You et al.

    “Segmentation of retinal blood vessels using the radial projection and semi-supervised approach”

    Pattern Recognit.

    (2011)
  • G. Azzopardi et al.

    “Trainable COSFIRE filters for vessel delineation with application to retinal images”

    Med. Image Anal.

    (2015)
  • J. Zhang et al.

    “Retinal vessel delineation using a brain-inspired wavelet transform and random forest”

    Pattern Recognit.

    (2017)
  • A. Oliveira et al.

    “Retinal vessel segmentation based on fully convolutional neural networks”

    Expert Syst. Appl.

    (2018)
  • Q. Jin et al.

    “DUNet: a deformable network for retinal vessel segmentation”

    Knowl. Based Syst.

    (2019)
  • P. Tang et al.

    “Multi-proportion channel ensemble model for retinal vessel segmentation”

    Comput. Biol. Med.

    (2019)
  • S. Guo et al.

    “BTS-DSN: deeply supervised neural network with short connections for retinal vessel segmentation”

    Int. J. Med. Inform.

    (2019)
  • Y. Wu et al.

    “NFN+: a novel network followed network for retinal vessel segmentation”

    Neural Netw.

    (2020)
  • M. Zardadi et al.

    “Unsupervised segmentation of retinal blood vessels using the human visual system line detection model”

    J. Inf. Syst. Telecommun.

    (2016)
  • C. Zhou et al.

    “A new robust method for blood vessel segmentation in retinal fundus images based on weighted line detector and hidden Markov model”

    Comput. Methods Programs Biomed.

    (2020)
  • Y. Qian Zhao, X. Hong Wang, X. Fang Wang, and F.Y. Shih, “Retinal vessels segmentation based on level set and region...
  • J. Staal et al.

    “Ridge-based vessel segmentation in color images of the retina”

    IEEE Trans. Med. Imaging

    (2004)
  • M.M. Fraz

    “An ensemble classification-based approach applied to retinal blood vessel segmentation”

    IEEE Trans. Biomed. Eng.

    (2012)
  • A. Hoover

    “Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response”

    IEEE Trans. Med. Imaging

    (2000)
  • V. Shanmugam et al.

    “Retinal blood vessel segmentation using an extreme learning machine approach”

  • C.A. Lupaşcu et al.

    “FABC: retinal vessel segmentation using AdaBoost”

    IEEE Trans. Inf. Technol. Biomed.

    (2010)
  • Y. Jiang et al.

    “Automatic retinal blood vessel segmentation based on fully convolutional neural networks”

    Symmetry

    (2019)
  • Cited by (16)

    View all citing articles on Scopus
    View full text