Abstract
Metastasis detection of lymph nodes in Whole-slide Images (WSIs) plays a critical role in the diagnosis of breast cancer. Automatic metastasis detection is a challenging issue due to the large variance of their appearances and the size of WSIs. Recently, deep neural networks have been employed to detect cancer metastases by dividing the WSIs into small image patches. However, most existing works simply treat these patches independently and do not consider the structural information among them. In this paper, we propose a novel deep neural network, namely Spatially Structured Network (Spatio-Net) to tackle the metastasis detection problem in WSIs. By integrating the Convolutional Neural Network (CNN) with the 2D Long-Short Term Memory (2D-LSTM), our Spatio-Net is able to learn the appearances and spatial dependencies of image patches effectively. Specifically, the CNN encodes each image patch into a compact feature vector, and the 2D-LSTM layers provide the classification results (i.e., normal or tumor), considering its dependencies on other relevant image patches. Moreover, a new loss function is designed to constrain the structure of the output labels, which further improves the performance. Finally, the metastasis positions are obtained by locating the regions with high tumor probabilities in the resulting accurate probability map. The proposed method is validated on hundreds of WSIs, and the accuracy is significantly improved, in comparison with a state-of-the-art baseline that does not have the spatial dependency constraint.
B. Kong and X. Wang—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
The published state-of-the-art results for the cancer metastasis detection competition were reported in our chosen baseline [20]. However, the competition was already closed, and the ground truth of the testing data was no longer available, so we were unable to evaluate our method upon the same testing data. Therefore, for fair comparison, we reimplemented the framework of [20] and evaluated all methods with five-fold cross-validation using the same released dataset.
- 3.
Note that we do not exhaustively tune these parameters, since our goal is to show improvement when using the spatially structured constraint under the same framework for fair comparison.
References
Apou, G., Naegel, B., Forestier, G., Feuerhake, F., Wemmert, C.: Efficient region-based classification for whole slide images. In: Battiato, S., Coquillart, S., Pettré, J., Laramee, R.S., Kerren, A., Braz, J. (eds.) VISIGRAPP 2014. CCIS, vol. 550, pp. 239–256. Springer, Cham (2015). doi:10.1007/978-3-319-25117-2_15
Doyle, S., Agner, S., Madabhushi, A., Feldman, M., Tomaszewski, J.: Automated grading of breast cancer histopathology using spectral clustering with textural and architectural image features. In: 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 496–499. IEEE (2008)
Geçer, B.: Detection and classification of breast cancer in whole slide histopathology images using deep convolutional networks. Ph.D. thesis, Bilkent University (2016)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016, in preparation). http://www.deeplearningbook.org
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Patch-based convolutional neural network for whole slide tissue image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2424–2433 (2016)
Howlader, N., Noone, A., Krapcho, M., Garshell, J., Neyman, N., Altekruse, S., Kosary, C., Yu, M., Ruhl, J., Tatalovich, Z., et al.: SEER Cancer Statistics Review, 1975–2010. National Cancer Institute, Bethesda (2013)
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)
Kalchbrenner, N., Danihelka, I., Graves, A.: Grid long short-term memory. arXiv preprint arXiv:1507.01526 (2015)
Kong, B., Zhan, Y., Shin, M., Denny, T., Zhang, S.: Recognizing end-diastole and end-systole frames via deep temporal regression network. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 264–272. Springer, Cham (2016). doi:10.1007/978-3-319-46726-9_31
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Liang, X., Shen, X., Feng, J., Lin, L., Yan, S.: Semantic object parsing with graph LSTM. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 125–143. Springer, Cham (2016). doi:10.1007/978-3-319-46448-0_8
Liang, X., Shen, X., Xiang, D., Feng, J., Lin, L., Yan, S.: Semantic object parsing with local-global long short-term memory. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3185–3193 (2016)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
van den Oord, A., Kalchbrenner, N., Kavukcuoglu, K.: Pixel recurrent neural networks. arXiv preprint arXiv:1601.06759 (2016)
Peng, Z., Zhang, R., Liang, X., Lin, L.: Geometric scene parsing with hierarchical LSTM. arXiv preprint arXiv:1604.01931 (2016)
Shiraishi, J., Li, Q., Suzuki, K., Engelmann, R., Doi, K.: Computer-aided diagnostic scheme for the detection of lung nodules on chest radiographs: localized search method based on anatomical classification. Med. Phys. 33(7), 2642–2653 (2006)
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Wang, D., Khosla, A., Gargeya, R., Irshad, H., Beck, A.H.: Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). doi:10.1007/978-3-319-10590-1_53
Zhang, X., Liu, W., Dundar, M., Badve, S., Zhang, S.: Towards large-scale histopathological image analysis: hashing-based image retrieval. IEEE Trans. Med. Imaging 34(2), 496–506 (2015)
Zhang, X., Xing, F., Su, H., Yang, L., Zhang, S.: High-throughput histopathological image analysis via robust cell segmentation and hashing. Med. Image Anal. 26(1), 306–315 (2015)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Kong, B., Wang, X., Li, Z., Song, Q., Zhang, S. (2017). Cancer Metastasis Detection via Spatially Structured Deep Network. In: Niethammer, M., et al. Information Processing in Medical Imaging. IPMI 2017. Lecture Notes in Computer Science(), vol 10265. Springer, Cham. https://doi.org/10.1007/978-3-319-59050-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-59050-9_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-59049-3
Online ISBN: 978-3-319-59050-9
eBook Packages: Computer ScienceComputer Science (R0)