A Fast Stain Normalization Network for Cervical Papanicolaou Images

Abstract

The domain shift between different styles of stain images greatly challenges the generalization of computer-aided diagnosis (CAD) algorithms. To bridge the gap, color normalization is a prerequisite for most CAD algorithms. The existing algorithms with better normalization effect often require more computational consumption, resisting the fast application in large-size medical stain slide images. This paper designs a fast normalization network (FTNC-Net) for cervical Papanicolaou stain images based on learnable bilateral filtering. In our FTNC-Net, explicit three-attribute estimation and spatially adaptive instance normalization are introduced to guide the model to transfer stain color styles in space accurately, and dynamic blocks are adopted to adapt multiple stain color styles. Our method achieves at least 80 fps over 1024$\times$1024 images on our experimental platform, thus it can synchronize with the scanner for image acquisition and processing, and has advantages in visual and quantitative evaluation compared with other methods. Moreover, experiments on our cervical staining image dataset demonstrate that the FTNC-Net improves the precision of abnormal cell detection.

Appendix

Our network (FTNC-Net) is splitted into three branches: the low-resolution splatting (Lsp) branch, the autoencoder (Ae) branch, and the full-resolution guided branch (Fgd). The total process:

$$\begin{aligned} z&= AE(concat(Encoding(I_{c}),Encoding(I_{s})))\end{aligned}$$

(10)

$$\begin{aligned} \varGamma&= Lsp(I_{lc},I_{ls},attr_c,attr_s,z)\end{aligned}$$

(11)

$$\begin{aligned} m&= Fgd(I_{c},z)\end{aligned}$$

(12)

$$\begin{aligned} k,b&= GridSample(m,\varGamma )\end{aligned}$$

(13)

$$\begin{aligned} I_o&= k*SpAdaIn(I_c,I_s,attr_c,attr_s)+b \end{aligned}$$

(14)

The input of Ae is a vector concatenated by two stain style encodings from the full-resolution content image $I_c$ and style image $I_s$. Lsp generates the bilateral grid $\varGamma$ by the low-resolution content image $I_{lc}$, the low-resolution style image $I_{ls}$, 3-attribute estimation ($attr_c$, $attr_s$) of $I_c$ and $I_s$, and the latent variable z. Fgd produces the guided map m by $I_c$ and z. The GrideSample computes the linear transformation weights k and b using m values and pixel locations from $\varGamma$, and this Eq. (13) called slice connects Lsp and Fgd. The output image $I_o$ is obtained by applying the linear weights calculated in Eq. (13) to the output of SpAdaIn, and this process is called apply, where SpAdaIn reduces the influence of the stain color of $I_c$.

The detailed architecture of FTNC-Net is shown in Table 4 and 5:

Table 4. Architecture of Lsp and Fgd

Table 5. Architecture of Ae

Due to the constrain of GPU memory, StainGAN¹ is implemented on images cropped into $320\times 320$ from $1024\times 1024$. As the Fig. 5 shown, the discriminator in GAN guides the generator to generate the nonexistent neutrophils to match the style image, which changes the structure of the content image. And the results of StainGAN appear cross-color since there is no color consistency limits.

Fig. 5.

A sample from StainGAN.