PAT-Unet: Paired Attention Transformer for Efficient and Accurate Segmentation of 3D Medical Images

Abstract

Due to the remarkable performance of Transformers in 2D medical image segmentation, recent studies have incorporated them into 3D medical segmentation tasks. Compared to convolution operations in CNNs, Transformer-based models possess self-attention, allowing them to capture long-range dependencies among pixels. To address the high computational cost of the Transformer architecture when dealing with volumetric images containing a large number of slices, we propose an efficient hybrid CNN-Transformer architecture for 3D medical image segmentation named PAT-Unet. Firstly, our proposed Paired Attention Transformer (PAT) blocks effectively reduce spatial dimensions while proficiently learning channel and spatial information in 3D feature maps. This leads to improved segmentation performance by reducing parameter count and accelerating computation speed. Secondly, our Deformable Enhanced Skip Connection (DESC) module captures detailed features in irregular lesion areas by learning volume spatial offsets. Finally, we experimentally validate the effectiveness and efficiency of our model on the Synapse and ACDC benchmark datasets. On the Synapse dataset, our model achieves a Dice similarity score of 87.17%, reducing parameters and FLOPs by 67% compared to the best existing methods reported in the literature.