Abstract
Automated nucleus/cell recognition is a very challenging task, especially for differentiating tumor nuclei from non-tumor nuclei in Ki67 immunohistochemistry (IHC) stained images. Convolutional neural networks and their variants have been recently introduced to identify different types of nuclei and have achieved state-of-the-art performance. However, previous nucleus recognition approaches do not explicitly encode contextual information in the images, which can be very helpful for network representation learning. In this paper, we propose a novel multi-field-of-view context encoding method for single-stage nuclei identification in Ki67 IHC stained images. Specifically, we learn a deep structured regression model that takes multi-field of views of images as input and conducts feature aggregation on the fly for representation learning; then, we design a context encoding module to explicitly explore the multi-field-of-view contextual information and enhance the model’s representation power. In order to further improve nucleus recognition, we also introduce a novel deep regression loss that can emphasize specific channels of the prediction map with category-aware channel suppression. The proposed method can be learned in an end-to-end, pixel-to-pixel manner for single-stage nucleus recognition. We evaluate our method on a large-scale pancreatic neuroendocrine tumor image dataset, and the experiments demonstrate the superior performance of our method in nucleus recognition.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bell, S., Zitnick, C.L., Bala, K., Girshick, R.: Inside-outside net: detecting objects in context with skip pooling and recurrent neural networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2874–2883 (2016)
Cireşan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Mitosis detection in breast cancer histology images with deep neural networks. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 411–418. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40763-5_51
Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). In: International Conference on Learning Representations (ICLR), pp. 1–14 (2016)
Csurka, G., Dance, C.R., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: ECCV Workshop on Statistical Learning in Computer Vision, pp. 1–22 (2004)
Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 524–531 (2005)
Graham, S., et al.: Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7132–7141 (2018)
Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (ICLR), pp. 1–15 (2015)
Liu, L., et al.: Deep learning for generic object detection: a survey. Int. J. Comput. Vis. 128, 261–318 (2020). https://doi.org/10.1007/s11263-019-01247-4
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2015)
Nguyen, K., Bredno, J., Knowles, D.A.: Using contextual information to classify nuclei in histology images. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 995–998 (2015)
Qu, H., et al.: Joint segmentation and fine-grained classification of nuclei in histopathology images. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI), pp. 900–904 (2019)
Raza, S.E.A., et al.: Deconvolving convolutional neural network for cell detection. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI), pp. 891–894 (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sirinukunwattana, K., Raza, S.E.A., Tsang, Y.W., Snead, D.R.J., Cree, I.A., Rajpoot, N.M.: Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans. Med. Imag. 35(5), 1196–1206 (2016)
Szegedy, C., et al.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)
Tokunaga, H., Teramoto, Y., Yoshizawa, A., Bise, R.: Adaptive weighting multi-field-of-view CNN for semantic segmentation in pathology. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12589–12598 (2019)
Wang, S., Yao, J., Xu, Z., Huang, J.: Subtype cell detection with an accelerated deep convolution neural network. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 640–648. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_74
Xie, W., Noble, J.A., Zisserman, A.: Microscopy cell counting with fully convolutional regression networks. In: The 1st Workshop on Deep Learning in Medical Image Analysis, pp. 1–8 (2015)
Xie, Y., Xing, F., Shi, X., Kong, X., Su, H., Yang, L.: Efficient and robust cell detection: a structured regression approach. Med. Image Anal. 44, 245–254 (2018)
Xing, F., Cornish, T.C., Bennett, T., Ghosh, D., Yang, L.: Pixel-to-pixel learning with weak supervision for single-stage nucleus recognition in Ki67 images. IEEE Trans. Biomed. Eng. 66(11), 3088–3097 (2019)
Zeng, X., et al.: Crafting GBD-Net for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 40(9), 2109–2123 (2018)
Zhang, H., et al.: Context encoding for semantic segmentation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7151–7160 (2018)
Zhang, H., Xue, J., Dana, K.: Deep TEN: texture encoding network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2896–2905 (2017)
Zhang, M., Li, X., Xu, M., Li, Q.: RBC semantic segmentation for sickle cell disease based on deformable U-Net. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 695–702. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_79
Zhou, Y., Dou, Q., Chen, H., Qin, J., Heng, P.A.: SFCN-OPI: detection and fine-grained classification of nuclei using sibling FCN with objectness prior interaction. In: The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI), pp. 2652–2659 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Bai, T., Xu, J., Xing, F. (2020). Multi-field of View Aggregation and Context Encoding for Single-Stage Nucleus Recognition. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12265. Springer, Cham. https://doi.org/10.1007/978-3-030-59722-1_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-59722-1_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59721-4
Online ISBN: 978-3-030-59722-1
eBook Packages: Computer ScienceComputer Science (R0)
