M3U-CDVAE: Lightweight retinal vessel segmentation and refinement network
Introduction
In clinical application, doctors can diagnose whether patients suffer from hypertension, glaucoma, arteriosclerosis, or cardiovascular diseases by analyzing the number, width, length, branches, curvature, and vascular orientation angle of retinal vessels. Retinal vessel structure is highly complex for high curvature and diverse morphology, making retinal vessel segmentation challenging. In addition, non-uniform illumination and diseased regions make vessels more difficult to segment. Therefore, the accuracy and coherence of retinal vessel segmentation are essential. Currently, the vessel of fundus images is mainly manually marked by experts. However, artificial vessel segmentation has a heavy workload and low efficiency, which is hard to meet the vast needs of many patients. Applying an efficient artificial intelligence method to segment retinal vessels in fundus images is extremely important.
Detailed spatial information is critical for the accurate localization of vessel boundaries, especially for tiny vessels [1]. However, details are easily lost when learning contextual information. Additional structures and units are designed in [1], [2], [3] to retain detailed spatial information. The recent research trend is to use less computation to achieve approximate segmentation results. He et al. [4] point out that more complex neural networks are more challenging to train, and the accuracy will get saturation with increased network depth. A lightweight segmentation network is to weigh the number of parameters and segmentation accuracy and uses as few parameters as possible to achieve better segmentation performance.
The results of a single segmentation network are far from flawless and will inevitably contain a small number of incorrect segmentations. And even a single pixel can lead to visual differences, such as isolated pixels or breaks. These errors may put at risk applications that require vessel pathways extraction and characterization [5], as they may lead to relevant differences in the overall blood vessel graph.
To address the above issues, we propose a segmentation-refinement network named M3U-CDVAE for retinal vessel segmentation. The lightweight network M3U is adopted as the segmentation network to achieve effective segmentation with a small number of parameters. Then, the Hierarchical Feature Fusion (HFF) unit combines the intermediate features generated by the segmentation network with the pre-segmentation results and connects them to the corresponding layer in Convolutional Denoising Variational Auto-Encoder (CDVAE). CDVAE is employed as the refinement network to reduce the incorrect segmentations in the pre-segmentation results and make the segmented blood vessels more coherent. The main contributions can be summarized as follows:
M3U is proposed as a lightweight segmentation network, which is composed of lightweight models and can suppress background noise.
CDVAE employs variational inference, which is able to correct many incorrect segmentations by generating a rich but compact latent space.
HFF unit can suppress spatial noise while adding spatial information to features. Then, it connects the segmentation and refinement networks so that CDVAE can reuse the intermediate features of the segmentation network.
The proposed network is tested on three publicly available retina fundus datasets with annotated vessel ground truth labels. M3U-CDVAE achieves new state-of-the-art results on the above three datasets, especially for tiny vessels and complex backgrounds.
Section snippets
Retinal vessel segmentation
Retinal vessel segmentation has always been a research hotspot because of its clinical value. This task has three complex problems: tiny vessels, the high curvature and diverse morphology of retinal vessels, and a complex background environment. Retinal vessels contain many tightly connected tiny vessels with small widths that are often difficult to segment. In addition, the contrast between the tiny blood vessels and the background is relatively low, which further increases the difficulty of
Proposed method
The proposed network comprises a segmentation network and a refinement network. Firstly, the segmentation network employs M3U to produce pre-segmentation results. Then, the HFF unit combines the intermediate features generated by the segmentation network with the pre-segmentation results and connects them to each layer in the CDVAE. Finally, CDVAE is adopted as the refinement network, aiming to improve the segmentation quality. Fig. 1 shows the entire network architecture.
Experiments and discussion
In this section, the proposed M3U-CDVAE is testified on benchmark datasets. We provide a detailed description of the datasets, metrics, the results, and the comparisons.
Conclusions
We propose a two-stage deep neural network architecture for optimizing the results of vessel segmentation with less computational cost. Firstly, the lightweight segmentation network M3U is used to efficiently generate pre-segmentation results and intermediate features. Then, the HFF unit combines the intermediate features generated by the segmentation network with the pre-segmentation results to deliver to the corresponding layer of CDVAE in a progressive fusion manner. Finally, CDVAE is used
Funding
This work was supported by the National Nature Science Foundation of China under Grant 61872143.
CRediT authorship contribution statement
Yang Yu: Software, Methodology, Visualization, Writing – original draft. Hongqing Zhu: Conceptualization, Supervision, Writing – review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (42)
- et al.
Contextual information enhanced convolutional neural networks for retinal vessel segmentation in color fundus images
J. Vis. Commun. Image R.
(2021) - et al.
Edge-aware U-Net with gated convolution for retinal vessel segmentation
Biomed. Signal Process. Control
(2022) - et al.
MD-Net: A multi-scale dense network for retinal vessel segmentation
Biomed. Signal Process. Control
(2021) - et al.
Real-time and light-weighted unsupervised video object segmentation network
Pattern Recognition
(2021) - et al.
Salient object detection via light-weight multi-path cascaded networks
Neurocomputing
(2021) - et al.
Aligned variational autoencoder for matching danmaku and video storylines
Neurocomputing
(2021) - et al.
Attention gated networks: Learning to leverage salient regions in medical images
Med. Image Anal.
(2019) - et al.
Bridge-Net: Context-involved U-Net with patch-based loss weight mapping for retinal blood vessel segmentation
Expert Syst. Appl.
(2022) - et al.
Detail-preserving network with high resolution representation for efficient segmentation of retinal vessels
J. Ambient Intell. Humanized Comput.
(2021) - M. Zhang, F. Yu, J. Zhao, L. Zhang, Q. Li, BEFD: Boundary enhancement and feature denoising for vessel segmentation, in...
CSU-Net: A context spatial U-Net for accurate blood vessel segmentation in fundus images
IEEE J. Biomed. Health Inf.
Attention W-Net: Improved skip connections for better representations
Light-weight semantic segmentation network for UAV remote sensing images
IEEE J. Sel. Top Appl. Earth Obs. Remote Sens.
MEAN-SSD: A novel real-time detector for apple leaf diseases using improved light-weight convolutional neural networks
Comput. Electron. Agric.
Cited by (4)
Optimization of retinal artery/vein classification based on vascular topology
2024, Biomedical Signal Processing and ControlRetinal multi-lesion segmentation by reinforcing single-lesion guidance with multi-view learning
2023, Biomedical Signal Processing and ControlTransformer and convolutional based dual branch network for retinal vessel segmentation in OCTA images
2023, Biomedical Signal Processing and ControlCMP-UNet: A Retinal Vessel Segmentation Network Based on Multi-Scale Feature Fusion
2023, Electronics (Switzerland)