Resolution-Invariant Medical Image Segmentation Using Fourier Neural Operators

Abstract

Challenges in medical image segmentation arise from limited labeled datasets, especially in high-resolution scenarios requiring expert annotations. To tackle this, resolution-invariant techniques become crucial, aiming to enhance details in segmentation using models trained on low-resolution images. This study advocates incorporating Fourier neural operators into neural network architectures, leveraging its unique formulation in Fourier space to solve partial differential equations and achieve efficient resolution-invariant results in medical image segmentation. The model’s effectiveness is evaluated across diverse tasks, including pericardium segmentation in low-dose computed tomography (LDCT), left atrium segmentation in mono-model magnetic resonance images (MRI), and HeLa cell segmentation in microscope images. Models trained with images of size $64 \times 64$ are evaluated on images of sizes $32 \times 32$ to $256 \times 256$. The results demonstrate that Fourier neural operator models can achieve accurate and consistent segmentation on image sizes equal to or larger than in the training data, hence nicely generalizing in terms of resolution. Furthermore, we investigate the influence of training data size from 25 to 9000 images on Fourier neural operator models, and experiments show that our model can generate relatively stable segmentation results when training with only 500 images of $64 \times 64$ pixels. Ultimately, we explore the strengths, limitations, and potential research directions regarding the role of the Fourier neural operator in enhancing the accuracy and reliability of medical image segmentation.

A Additional Results

Additional results of the three datasets with FNO models trained with $32 \times 32$ images are given in Table 4. Additional segmentation visualization from FNO models of the pericardium in LDCT (Fig. 6), the left atrium in MRI (Fig. 7), and HeLa cells in microscope images (Fig. 8) are given.

Table 4. Results of the three datasets with FNO models trained with $32 \times 32$ images.

Fig. 6.

Segmentation visualization of the pericardium in LDCT of (a) $32 \times 32$, (b) $64 \times 64$, (c) $128 \times 128$, and (d) $256 \times 256$ with FNO model trained with $64 \times 64$ data.

Fig. 7.

Segmentation visualization of the left atrium in MRI of (a) $32 \times 32$, (b) $64 \times 64$, (c) $128 \times 128$, and (d) $256 \times 256$ with FNO model trained with $64 \times 64$ data.

Fig. 8.

Segmentation visualization of the HeLa cells in microscope images of (a) $32 \times 32$, (b) $64 \times 64$, (c) $128 \times 128$, and (d) $256 \times 256$ with FNO model trained with $64 \times 64$ data.