Guoxiang Qu
ABSTRACT
In this work, we propose to study a special semantic segmentation problem where the targets are long and continuous strip patterns. Strip patterns widely exist in medical images and natural photos, such as retinal layers in OCT images and lanes on the roads, and segmentation of them has practical significance. Traditional pixel-level segmentation methods largely ignore the structure prior of strip patterns and thus easily suffer from the topological inconformity problem, such as holes and isolated islands in segmentation results. To tackle this problem, we design a novel deep framework, StripNet, that leverages the strong end-to-end learning ability of CNNs to predict the structured outputs as a sequence of boundary locations of the target strips. Specifically, StripNet decomposes the original segmentation problem into more easily solved local boundary-regression problems, and takes account of the topological constraints on the predicted boundaries. Moreover, our framework adopts a coarse-to-fine strategy and uses carefully designed heatmaps for training the boundary localization network. We examine StripNet on two challenging strip pattern segmentation tasks, retinal layer segmentation and lane detection. Extensive experiments demonstrate that StripNet achieves excellent results and outperforms state-of-the-art methods in both tasks.
- Joao Carreira, Rui Caseiro, Jorge Batista, and Cristian Sminchisescu. 2012. Semantic segmentation with second-order pooling. In Proc. ECCV. Google Scholar
Digital Library
- Dengfeng Chai, Wolfgang Förstner, and Florent Lafarge. 2013. Recovering line-networks in images by junction-point processes. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on. IEEE, 1894--1901. Google ScholarDigital Library
- Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Return of the Devil in the Details: Delving Deep into Convolutional Nets. (2014).Google Scholar
- Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. 2017. Rethinking Atrous Convolution for Semantic Image Segmentation. CoRR, Vol. abs/1706.05587 (2017). arxiv: 1706.05587 http://arxiv.org/abs/1706.05587Google Scholar
- Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2015. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs. In Proc. ICLR.Google Scholar
- Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2016. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. arXiv preprint arXiv:1606.00915 (2016).Google Scholar
- KY Chiu and SF Lin. 2005. Lane detection using color-based segmentation. WOS:000235518700117 (2005). https://ir.nctu.edu.tw/handle/11536/17998Google Scholar
- Francc ois Chollet. 2016. Xception: Deep Learning with Depthwise Separable Convolutions. CoRR, Vol. abs/1610.02357 (2016). arxiv: 1610.02357 http://arxiv.org/abs/1610.02357Google Scholar
- Xiao Chu, Wanli Ouyang, Hongsheng Li, and Xiaogang Wang. 2016. Structured Feature Learning for Pose Estimation. CoRR, Vol. abs/1603.09065 (2016). arxiv: 1603.09065 http://arxiv.org/abs/1603.09065Google Scholar
- Clement Farabet, Camille Couprie, Laurent Najman, and Yann LeCun. 2013. Learning hierarchical features for scene labeling. TPAMI, Vol. 35, 8 (2013), 1915--1929. Google ScholarDigital Library
- Mona Kathryn Garvin, Michael David Abramoff, Xiaodong Wu, Stephen R Russell, Trudy L Burns, and Milan Sonka. 2009. Automated 3-D intraretinal layer segmentation of macular spectral-domain optical coherence tomography images. IEEE transactions on medical imaging, Vol. 28, 9 (2009), 1436--1447.Google Scholar
- Raghuraman Gopalan, Tsai Hong, Michael Shneier, and Rama Chellappa. 2012. A Learning Approach Towards Detection and Tracking of Lane Markings. Technical Report. IEEE Transactions on Intelligent Transportation Systems. Google ScholarDigital Library
- Bei He, Rui Ai, Yang Yan, and Xianpeng Lang. 2016a. Accurate and robust lane detection based on dual-view convolutional neutral network. In Intelligent Vehicles Symposium (IV), 2016 IEEE. IEEE, 1041--1046.Google Scholar
- Kaiming He, Georgia Gkioxari, Piotr Dollá r, and Ross B. Girshick. 2017b. Mask R-CNN. CoRR, Vol. abs/1703.06870 (2017). arxiv: 1703.06870 http://arxiv.org/abs/1703.06870Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016b. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.Google ScholarCross Ref
- Yufan He, Aaron Carass, Yeyi Yun, Can Zhao, Bruno M. Jedynak, Sharon D. Solomon, Shiv Saidha, Peter A. Calabresi, and Jerry L. Prince. 2017a. Towards Topological Correct Segmentation of Macular OCT from Cascaded FCNs. (2017).Google Scholar
- Brody Huval, Tao Wang, Sameep Tandon, Jeff Kiske, Will Song, Joel Pazhayampallil, Mykhaylo Andriluka, Pranav Rajpurkar, Toki Migimatsu, Royce Cheng-Yue, et almbox. 2015. An empirical evaluation of deep learning on highway driving. arXiv preprint arXiv:1504.01716 (2015).Google Scholar
- Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv preprint arXiv:1408.5093 (2014).Google Scholar
- Claudio Rosito Jung and Christian Roberto Kelber. 2004. A robust linear-parabolic model for lane following. In Computer Graphics and Image Processing, 2004. Proceedings. 17th Brazilian Symposium on. IEEE, 72--79. Google ScholarDigital Library
- Byungsoo Kim, Oliver Wang, A. Cengiz Öztireli, and Markus Gross. 2018. Semantic Segmentation for Line Drawing Vectorization Using Neural Networks. Computer Graphics Forum (Proc. Eurographics), Vol. 37, 2 (2018), 329--338.Google ScholarCross Ref
- Jihun Kim and Minho Lee. 2014. Robust lane detection based on convolutional neural network and random sample consensus. In International Conference on Neural Information Processing. Springer, 454--461.Google ScholarCross Ref
- Andrew Lang, Carass Aaron, Hauser Matthew, Elias S Sotirchos, Peter A Calabresi, Howard S Ying, and Jerry L Prince. 2013. Retinal layer segmentation of macular OCT images using boundary classification. Biomedical Optics Express, Vol. 4, 7 (2013), 1133--1152.Google ScholarCross Ref
- Seokju Lee, Junsik Kim, Jae Shin Yoon, Seunghak Shin, Oleksandr Bailo, Namil Kim, Tae-Hee Lee, Hyun Seok Hong, Seung-Hoon Han, and In So Kweon. 2017. VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition. In The IEEE International Conference on Computer Vision (ICCV).Google Scholar
- Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen-Change Loy, and Xiaoou Tang. 2015. Semantic image segmentation via deep parsing network. In Proc. ICCV. Google ScholarDigital Library
- Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431--3440.Google ScholarCross Ref
- Agata Mosinska, Pablo Marquez-Neila, Mateusz Kozinski, and Pascal Fua. 2017. Beyond the Pixel-Wise Loss for Topology-Aware Delineation. arXiv preprint arXiv:1712.02190 (2017).Google Scholar
- Jelena Novosel, Koenraad A. Vermeer, Gijs Thepass, Hans G. Lemij, and Lucas J. Van Vliet. 2003. Loosely coupled level sets for simultaneous 3D retinal layer segmentation in optical coherence tomography. In Simulation Conference, 2003. Proceedings of the. 59--65.Google Scholar
- Tomas Pfister, James Charles, and Andrew Zisserman. 2015. Flowing ConvNets for Human Pose Estimation in Videos. CoRR, Vol. abs/1506.02897 (2015). arxiv: 1506.02897 http://arxiv.org/abs/1506.02897 Google ScholarDigital Library
- Pedro H. O. Pinheiro and Ronan Collobert. 2014. Recurrent Convolutional Neural Networks for Scene Labeling. In Proc. ICML. Google ScholarDigital Library
- Xiaojuan Qi, Jianping Shi, Shu Liu, Renjie Liao, and Jiaya Jia. 2015. Semantic Segmentation With Object Clique Potential. In Proc. ICCV. Google ScholarDigital Library
- Fabian Rathke, Stefan Schmidt, and Christoph Schnörr. 2014. Probabilistic intra-retinal layer segmentation in 3-D OCT images using global shape regularization. Medical image analysis, Vol. 18, 5 (2014), 781--794.Google Scholar
- Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In Proc. NIPS. Google ScholarDigital Library
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In MICCAI. Springer, 234--241.Google Scholar
- Abhijit Guha Roy, Sailesh Conjeti, Sri Phani Krishna Karri, Debdoot Sheet, Amin Katouzian, Christian Wachinger, and Nassir Navab. 2017. ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomedical optics express, Vol. 8, 8 (2017), 3627--3642.Google Scholar
- Alexander G Schwing and Raquel Urtasun. 2015. Fully connected deep structured networks. arXiv preprint arXiv:1503.02351 (2015).Google Scholar
- Abhishek Sharma, Oncel Tuzel, and David W Jacobs. 2015. Deep Hierarchical Parsing for Semantic Segmentation. Proc. CVPR (2015).Google ScholarCross Ref
- Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- Ben Southall and Camillo J Taylor. 2001. Stochastic road shape estimation. In Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, Vol. 1. IEEE, 205--212.Google ScholarCross Ref
- Zhu Teng, Jeong-Hyun Kim, and Dong-Joong Kang. 2010. Real-time Lane detection by using multiple cues. In Control Automation and Systems (ICCAS), 2010 International Conference on. IEEE, 2334--2337.Google Scholar
- Chuang Wang, Yaxing Wang, Djibril Kaba, Zidong Wang, Xiaohui Liu, and Yongmin Li. 2015. Automated Layer Segmentation of 3D Macular Images Using Hybrid Methods. In International Conference on Image and Graphics. 614--628.Google ScholarCross Ref
- Jan D Wegner, Javier A Montoya-Zegarra, and Konrad Schindler. 2013. A higher-order CRF model for road network extraction. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on. IEEE, 1698--1705. Google ScholarDigital Library
- Shih En Wei, Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh. 2016. Convolutional Pose Machines. In Computer Vision and Pattern Recognition. 4724--4732.Google Scholar
- Pan Xingang, Shi Jianping, Luo Ping, Wang Xiaogang, and Tang Xiaoou. 2018. Spatial As Deep: Spatial CNN for Traffic Scene Understanding. In AAAI Conference on Artificial Intelligence (AAAI).Google Scholar
- Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. 2017. Pyramid scene parsing network. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). 2881--2890.Google ScholarCross Ref
Index Terms
- StripNet: Towards Topology Consistent Strip Structure Segmentation
Recommendations
Joint Boundary-Enhanced and Topology-Preserving Dual-Path Network for Retinal Layer Segmentation in OCT Images with Pigment Epithelial Detachment
Pattern Recognition and Computer VisionAbstractAutomatic retinal layer segmentation methods are currently successful in normal Optical Coherence Tomography (OCT) images, but they face great challenges for eyes with Pigment Epithelial Detachment (PED), where the morphology and structure of the ...
SD-LayerNet: Semi-supervised Retinal Layer Segmentation in OCT Using Disentangled Representation with Anatomical Priors
Medical Image Computing and Computer Assisted Intervention – MICCAI 2022AbstractOptical coherence tomography (OCT) is a non-invasive 3D modality widely used in ophthalmology for imaging the retina. Achieving automated, anatomically coherent retinal layer segmentation on OCT is important for the detection and monitoring of ...
VLDNet: Vision-based lane region detection network for intelligent vehicle system using semantic segmentation
AbstractDetection of lane region under the road boundary is an imperative module for intelligent vehicle system. Lane markings provide separate regions on the road for the vehicles to avoid the possibility of accidents. Existing methods in lane detection ...
Comments