M3U-CDVAE: Lightweight retinal vessel segmentation and refinement network

https://doi.org/10.1016/j.bspc.2022.104113Get rights and content

Highlights

  • M3U is proposed as a lightweight segmentation network against noise.

  • CDVAE employs variational inference, which is able to correct incorrect segmentations.

  • HFF unit can suppress noise while adding spatial information to features.

  • We design various lightweight network units and reduce feature channels to make the network lightweight.

Abstract

Retinal vessels have high curvature and diverse morphology, making them difficult to segment, especially tiny vessels. At present, the retinal vessels are mainly annotated manually by experts, which is difficult to meet the vast clinical needs. To solve the above problems, we propose an effective network M3U-CDVAE. It adopts the architecture of a segmentation-refinement network to denoise and optimizes segmentation results. Firstly, we design a lightweight segmentation network M3U with an encoder-decoder structure. Then, the Hierarchical Feature Fusion (HFF) unit combines the intermediate features generated by the segmentation network with the pre-segmentation results and connects them to the corresponding layer in the next sub-model. Finally, Convolutional Denoising Variational Auto-Encoder (CDVAE) is used as the refinement network to remove the background noise and optimize segmentation results. We conduct exhaustive ablation experiments to demonstrate the improvement brought by our contribution. At the same time, we carry out comparison experiments on DRIVE, STARE, and HRF datasets to illustrate the effectiveness of the proposed method. Experimental results exhibit that the proposed method is superior to most state-of-art methods.

Introduction

In clinical application, doctors can diagnose whether patients suffer from hypertension, glaucoma, arteriosclerosis, or cardiovascular diseases by analyzing the number, width, length, branches, curvature, and vascular orientation angle of retinal vessels. Retinal vessel structure is highly complex for high curvature and diverse morphology, making retinal vessel segmentation challenging. In addition, non-uniform illumination and diseased regions make vessels more difficult to segment. Therefore, the accuracy and coherence of retinal vessel segmentation are essential. Currently, the vessel of fundus images is mainly manually marked by experts. However, artificial vessel segmentation has a heavy workload and low efficiency, which is hard to meet the vast needs of many patients. Applying an efficient artificial intelligence method to segment retinal vessels in fundus images is extremely important.

Detailed spatial information is critical for the accurate localization of vessel boundaries, especially for tiny vessels [1]. However, details are easily lost when learning contextual information. Additional structures and units are designed in [1], [2], [3] to retain detailed spatial information. The recent research trend is to use less computation to achieve approximate segmentation results. He et al. [4] point out that more complex neural networks are more challenging to train, and the accuracy will get saturation with increased network depth. A lightweight segmentation network is to weigh the number of parameters and segmentation accuracy and uses as few parameters as possible to achieve better segmentation performance.

The results of a single segmentation network are far from flawless and will inevitably contain a small number of incorrect segmentations. And even a single pixel can lead to visual differences, such as isolated pixels or breaks. These errors may put at risk applications that require vessel pathways extraction and characterization [5], as they may lead to relevant differences in the overall blood vessel graph.

To address the above issues, we propose a segmentation-refinement network named M3U-CDVAE for retinal vessel segmentation. The lightweight network M3U is adopted as the segmentation network to achieve effective segmentation with a small number of parameters. Then, the Hierarchical Feature Fusion (HFF) unit combines the intermediate features generated by the segmentation network with the pre-segmentation results and connects them to the corresponding layer in Convolutional Denoising Variational Auto-Encoder (CDVAE). CDVAE is employed as the refinement network to reduce the incorrect segmentations in the pre-segmentation results and make the segmented blood vessels more coherent. The main contributions can be summarized as follows:

M3U is proposed as a lightweight segmentation network, which is composed of lightweight models and can suppress background noise.

CDVAE employs variational inference, which is able to correct many incorrect segmentations by generating a rich but compact latent space.

HFF unit can suppress spatial noise while adding spatial information to features. Then, it connects the segmentation and refinement networks so that CDVAE can reuse the intermediate features of the segmentation network.

The proposed network is tested on three publicly available retina fundus datasets with annotated vessel ground truth labels. M3U-CDVAE achieves new state-of-the-art results on the above three datasets, especially for tiny vessels and complex backgrounds.

Section snippets

Retinal vessel segmentation

Retinal vessel segmentation has always been a research hotspot because of its clinical value. This task has three complex problems: tiny vessels, the high curvature and diverse morphology of retinal vessels, and a complex background environment. Retinal vessels contain many tightly connected tiny vessels with small widths that are often difficult to segment. In addition, the contrast between the tiny blood vessels and the background is relatively low, which further increases the difficulty of

Proposed method

The proposed network comprises a segmentation network and a refinement network. Firstly, the segmentation network employs M3U to produce pre-segmentation results. Then, the HFF unit combines the intermediate features generated by the segmentation network with the pre-segmentation results and connects them to each layer in the CDVAE. Finally, CDVAE is adopted as the refinement network, aiming to improve the segmentation quality. Fig. 1 shows the entire network architecture.

Experiments and discussion

In this section, the proposed M3U-CDVAE is testified on benchmark datasets. We provide a detailed description of the datasets, metrics, the results, and the comparisons.

Conclusions

We propose a two-stage deep neural network architecture for optimizing the results of vessel segmentation with less computational cost. Firstly, the lightweight segmentation network M3U is used to efficiently generate pre-segmentation results and intermediate features. Then, the HFF unit combines the intermediate features generated by the segmentation network with the pre-segmentation results to deliver to the corresponding layer of CDVAE in a progressive fusion manner. Finally, CDVAE is used

Funding

This work was supported by the National Nature Science Foundation of China under Grant 61872143.

CRediT authorship contribution statement

Yang Yu: Software, Methodology, Visualization, Writing – original draft. Hongqing Zhu: Conceptualization, Supervision, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (42)

  • K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proc. IEEE Conf. Comput. Vis. Pattern...
  • Y. Zhao, J. Xie, P. Su, Y. Zheng, Y. Liu, J. Cheng, J. Liu, Retinal artery and vein classification via dominant sets...
  • A. Lou, M. Loew, CFPNET: Channel-wise feature pyramid for real-time semantic segmentation, in Proc. IEEE Int. Conf....
  • B. Wang et al.

    CSU-Net: A context spatial U-Net for accurate blood vessel segmentation in fundus images

    IEEE J. Biomed. Health Inf.

    (2021)
  • C. Guo, M. Szemenyei, Y. Yi, W. Zhou, H. Bian, Residual spatial attention network for retinal vessel segmentation, in...
  • S. Mohan et al.

    Attention W-Net: Improved skip connections for better representations

    (2021)
  • S. Liu et al.

    Light-weight semantic segmentation network for UAV remote sensing images

    IEEE J. Sel. Top Appl. Earth Obs. Remote Sens.

    (2021)
  • H. Sun et al.

    MEAN-SSD: A novel real-time detector for apple leaf diseases using improved light-weight convolutional neural networks

    Comput. Electron. Agric.

    (2021)
  • M. Tan, B. Chen, R. Pang, V. Vasudevan, M. Sandler, A. Howard, Q. V. Le, MnasNet: platform-aware neural architecture...
  • J. H. Shim; S. J. Kang, Neural architecture search for light-weight multi-touch classification, in Proc. Int. Tech....
  • J. Wei, Z. Fan, Genetic U-Net: Automatically designing lightweight U-shaped CNN architectures using the genetic...
  • Cited by (4)

    View full text