Abstract
The automated segmentation of regions of interest (ROIs) in medical imaging is the fundamental requirement for the derivation of high-level semantics for image analysis in clinical decision support systems. Traditional segmentation approaches such as region-based depend heavily upon hand-crafted features and a priori knowledge of the user. As such, these methods are difficult to adopt within a clinical environment. Recently, methods based on fully convolutional networks (FCN) have achieved great success in the segmentation of general images. FCNs leverage a large labeled dataset to hierarchically learn the features that best correspond to the shallow appearance as well as the deep semantics of the images. However, when applied to medical images, FCNs usually produce coarse ROI detection and poor boundary definitions primarily due to the limited number of labeled training data and limited constraints of label agreement among neighboring similar pixels. In this paper, we propose a new stacked FCN architecture with multi-channel learning (SFCN-ML). We embed the FCN in a stacked architecture to learn the foreground ROI features and background non-ROI features separately and then integrate these different channels to produce the final segmentation result. In contrast to traditional FCN methods, our SFCN-ML architecture enables the visual attributes and semantics derived from both the fore- and background channels to be iteratively learned and inferred. We conducted extensive experiments on three public datasets with a variety of visual challenges. Our results show that our SFCN-ML is more effective and robust than a routine FCN and its variants, and other state-of-the-art methods.





Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Musen, M.A., et al.: Clinical decision-support systems. Biomedical informatics. Springer, London (2014)
Doi, K.: Current status and future potential of computer-aided diagnosis in medical imaging. Br. J. Radiol. 78, s3–s19 (2005)
Bi, L., et al.: Automatic detection and classification of regions of FDG uptake in whole-body PET-CT lymphoma studies, Comput. Med. Imaging Graph. (2016). doi:10.1016/j.compmedimag.2016.11.008
Chen, X., et al.: Medical image segmentation by combining graph cuts and oriented active appearance models. IEEE Trans. Image Process. 21, 2035–2046 (2012)
Ahn, E., et al.: Saliency-based Lesion Segmentation via Background Detection in Dermoscopic Images. IEEE J. Biomed. Health Inform. (2017). doi:10.1109/JBHI.2017.2653179
Li, B.N., et al.: Integrating spatial fuzzy clustering with level set methods for automated medical image segmentation. Comput. Biol. Med. 41, 1–10 (2011)
Long, J., et al.: Fully convolutional networks for semantic segmentation, In: Proc. IEEE CVPR, 3431–3440 (2015)
Chen, L.-C., et al.: Semantic image segmentation with deep convolutional nets and fully connected crfs, In: Proc. ICLR (2015)
Zheng, S., et al.: Conditional random fields as recurrent neural networks, In: Proc. IEEE ICCV, 1529–1537 (2015)
Chen, H., et al.: DCAN: Deep contour-aware networks for object instance segmentation from histology images. Med. Image Anal. 36, 135–146 (2017)
Dou, Q., et al.: 3d deeply supervised network for automatic liver segmentation from ct volumes, In: Proc. MICCAI, 149–157 (2016)
Xu, Y., et al.: Gland instance segmentation by deep multichannel side supervision. In: Proc. MICCAI, 496–504 (2016)
BenTaieb, A., et al.: Topology Aware Fully Convolutional Networks for Histology Gland Segmentation. In: Proc. MICCAI, 460–468 (2016)
Paulano, F., et al.: 3D segmentation and labeling of fractured bone from CT images. Vis. Comput. 30, 939–948 (2014)
Lin, L., et al.: Inference with collaborative model for interactive tumor segmentation in medical image sequences. IEEE Trans. Cybern. 46, 2796–2809 (2016)
Bi, L., et al.: Cellular automata and anisotropic diffusion filter based interactive tumor segmentation for positron emission tomography, IEEE EMBC, 5453–5456 (2013)
ONeill, G.T., et al.: Segmentation of cam-type femurs from CT scans. Vis. Comput. 28, 205–218 (2012)
Silveira, M., et al.: Comparison of segmentation methods for melanoma diagnosis in dermoscopy images. IEEE J. Sel. Top. Signal Process. 3, 35–45 (2009)
Emre Celebi, M., et al.: Lesion border detection in dermoscopy images using ensembles of thresholding methods. Skin Res. Technol. 19, e252–e258 (2013)
Li, C., et al.: Minimization of region-scalable fitting energy for image segmentation. IEEE Trans. Image Process. 17, 1940–1949 (2008)
Li, C., et al.: Distance regularized level set evolution and its application to image segmentation. IEEE Trans. Image Process. 19, 3243–3254 (2010)
Wong, A., et al.: Automatic skin lesion segmentation via iterative stochastic region merging. IEEE Trans. Inf. Technol. Biomed. 15, 929–936 (2011)
Somkantha, K., et al.: Boundary detection in medical images using edge following algorithm based on intensity gradient and texture gradient features. IEEE Trans. Biomed. Eng. 58, 567–573 (2011)
Roy, A., et al.: JCLMM: A finite mixture model for clustering of circular–linear data and its application to psoriatic plaque segmentation, Pattern Recognit. (2017)
Nock, R., et al.: Statistical region merging. IEEE Trans. Pattern Anal. Mach. Intel. 26, 1452–1458 (2004)
van Rikxoort, E.M., et al.: Adaptive local multi-atlas segmentation: application to the heart and the caudate nucleus. Med. Image Anal. 14, 39 (2010)
Song, Y., et al.: Similarity guided feature labeling for lesion detection, In Proc. MICCAI, 284–291 (2013)
Li, C., et al.: Automated PET-guided liver segmentation from low-contrast CT volumes using probabilistic atlas. Comput. Method Program Biomed. 107, 164–174 (2012)
Bi, L., et al.: Automatic Descending Aorta Segmentation in Whole-Body PET-CT Studies for PERCIST-Based Thresholding, In: Proc. DICTA, 1–6 (2012)
Li, C., et al.: Joint probabilistic model of shape and intensity for multiple abdominal organ segmentation from volumetric CT images. IEEE J. Biomed. Health Inform. 17, 92–102 (2013)
Isgum, I., et al.: Multi-atlas-based segmentation with local decision fusion: application to cardiac and aortic segmentation in CT scans. IEEE Trans. Med. Imaging 28, 1000–1010 (2009)
Bi, L., et al.: Multi-stage thresholded region classification for whole-body PET–CT lymphoma studies, In: Proc. MICCAI, 569–576 (2014)
Song, Y., et al.: A multistage discriminative model for tumor and lymph node detection in thoracic images. IEEE Trans. Med. Imaging 31, 1061–1075 (2012)
Lartizien, C., et al.: Computer-aided staging of lymphoma patients with FDG PET/CT imaging based on textural information. IEEE J. Biomed. Health Inform. 18, 946–955 (2014)
Ciresan, D., et al.: Deep neural networks segment neuronal membranes in electron microscopy images, In: Proc. NIPS, 2843–2851 (2012)
Cha, K.H., et al.: Urinary bladder segmentation in CT urography using deeplearning convolutional neural network and level sets. Med. Phys. 43, 1882–1896 (2016)
Havaei, M., et al.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017)
Roth, H.R., et al.: Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation, In: Proc. MICCAI, 556–564 (2015)
Farag, A., et al.: A bottom-up approach for pancreas segmentation using cascaded superpixels and (deep) image patch labeling. IEEE Trans. Image Process. 26, 386–399 (2017)
Ronneberger, O., et al.: U-net: convolutional networks for biomedical image segmentation. In: Proc. MICCAI, 234–241 (2015)
Fu, H., et al.: Retinal vessel segmentation via deep learning network and fully-connected conditional random fields, In: Proc. IEEE ISBI, 698–701 (2016)
Lu, F., et al.: Automatic 3D liver location and segmentation via convolutional neural network and graph cut. Int. J. Comput. Assist. Radiol. Surg. 12, 171–182 (2017)
Hu, P., et al.: Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets, Int. J. Comput. Assist. Radiol. Surg. 1–13, (2016)
Chatfield, K., et al.: Return of the devil in the details: delving deep into convolutional nets, BMVC (2014)
Wang, L., et al.: Saliency detection with recurrent fully convolutional networks, In: Proc. ECCV, 825–841 (2016)
Dean, J., et al.: Large scale distributed deep networks, In: NIPS, 1223–1231 (2012)
Hogeweg, L., et al.: Clavicle segmentation in chest radiographs. Med. Image Anal. 16, 1490–1502 (2012)
Sirinukunwattana, K., et al.: A stochastic polygons model for glandular structures in colon histology images. IEEE Trans. Med. Imaging 34, 2366–2378 (2015)
Shin, H.-C., et al.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35, 1285–1298 (2016)
Deng, J., et al.: Imagenet: a large-scale hierarchical image database, In: Proc. CVPR, 248–255, (2009)
Kumar, A., et al.: An ensemble of fine-tuned convolutional neural networks for medical image classification. IEEE J. Biomed. Health Inform. 21, 31–40 (2017)
Vedaldi, A., et al.: Matconvnet: Convolutional neural networks for matlab, In: Proc. ACM MM, 689–692 (2015)
Krizhevsky, A., et al.: Imagenet classification with deep convolutional neural networks, In: NIPS, 1097–1105 (2012)
Li, X., et al.: Contextual hypergraph modeling for salient object detection, In: Proc. ICCV, 3328–3335 (2013)
Chan, T.F., Vese, L.A.: Active contour and segmentation models using geometric PDEs for medical imaging. Geometric methods in bio-medical image processing. Springer, Berlin (2002)
Zhang, Y., et al.: Medical image segmentation using new hybrid level-set method, In: Proc. MEDIVIS, 71–76 (2008)
Shotton, J., et al.: Semantic texton forests for image categorization and segmentation, In: Proc. CVPR, 1–8 (2008)
Bai, X., et al.: Video snapcut: robust video object cutout using localized classifiers, In: ACM Trans. Graph. (2009)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bi, L., Kim, J., Kumar, A. et al. Stacked fully convolutional networks with multi-channel learning: application to medical image segmentation. Vis Comput 33, 1061–1071 (2017). https://doi.org/10.1007/s00371-017-1379-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-017-1379-4