skip to main content
research-article

Superpixel Region Merging Based on Deep Network for Medical Image Segmentation

Published: 31 May 2020 Publication History

Abstract

Automatic and accurate semantic segmentation of pathological structures in medical images is challenging because of noisy disturbance, deformable shapes of pathology, and low contrast between soft tissues. Classical superpixel-based classification algorithms suffer from edge leakage due to complexity and heterogeneity inherent in medical images. Therefore, we propose a deep U-Net with superpixel region merging processing incorporated for edge enhancement to facilitate and optimize segmentation. Our approach combines three innovations: (1) different from deep learning--based image segmentation, the segmentation evolved from superpixel region merging via U-Net training getting rich semantic information, in addition to gray similarity; (2) a bilateral filtering module was adopted at the beginning of the network to eliminate external noise and enhance soft tissue contrast at edges of pathogy; and (3) a normalization layer was inserted after the convolutional layer at each feature scale, to prevent overfitting and increase the sensitivity to model parameters. This model was validated on lung CT, brain MR, and coronary CT datasets, respectively. Different superpixel methods and cross validation show the effectiveness of this architecture. The hyperparameter settings were empirically explored to achieve a good trade-off between the performance and efficiency, where a four-layer network achieves the best result in precision, recall, F-measure, and running speed. It was demonstrated that our method outperformed state-of-the-art networks, including FCN-16s, SegNet, PSPNet, DeepLabv3, and traditional U-Net, both quantitatively and qualitatively. Source code for the complete method is available at https://github.com/Leahnawho/Superpixel-network.

References

[1]
Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurélien Lucchi, Pascal Fua, and Sabine Süsstrunk. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 11 (2012), 2274--2282.
[2]
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. 2017. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 12 (2017), 2481--2495.
[3]
Robin Brügger, Christian F. Baumgartner, and Ender Konukoglu. 2019. A partially reversible U-net for memory-efficient volumetric image segmentation. In Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI’19). 429--437.
[4]
Mahaman Sani Chaibou, Pierre-Henri Conze, Karim Kalti, Basel Solaiman, and Mohamed Ali Mahjoub. 2018. Adaptive strategy for superpixel-based region-growing image segmentation. arXiv:1803.06541.
[5]
Jiansheng Chen, Zhengqin Li, and Bo Huang. 2017. Linear spectral clustering superpixel. IEEE Transactions on Image Processing 26, 7 (2017), 3317--3330.
[6]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. 2018. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 4 (2018), 834--848.
[7]
Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. 2017. Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587.
[8]
Hao Dong, Guang Yang, Fangde Liu, Yuanhan Mo, and Yike Guo. 2017. Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks. In Proceedings of the 21st Conferrence on Medical Image Understanding and Analysis (MIUA’17). 506--517.
[9]
Michal Drozdzal, Gabriel Chartrand, Eugene Vorontsov, Mahsa Shakeri, Lisa Di-Jorio, An Tang, Adriana Romero, Yoshua Bengio, Chris Pal, and Samuel Kadoury. 2018. Learning normalized inputs for iterative estimation in medical image segmentation. Medical Image Analysis 44 (2018), 1--13.
[10]
Robert M. Haralick and Linda G. Shapiro. 1985. Image segmentation techniques. Computer Vision, Graphics, and Image Processing 29, 1 (1985), 100--132.
[11]
Bulat Ibragimov, Robert Korez, Bostjan Likar, Franjo Pernus, Lei Xing, and Tomaz Vrtovec. 2017. Segmentation of pathological structures by landmark-assisted deformable models. IEEE Transactions on Medical Imaging 36, 7 (2017), 1457--1469.
[12]
Bulat Ibragimov and Lei Xing. 2016. Segmentation of organs-at-risks in head and neck CT images using convolutional neural networks. Medical Physics 44 (2016), 547--557.
[13]
Bo Li, Wiro J. Niessen, Stefan Klein, Marius de Groot, Mohammad Arfan Ikram, Meike W. Vernooij, and Esther E. Bron. 2019. A hybrid deep learning framework for integrated segmentation and registration: Evaluation on longitudinal white matter tract changes. In Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI’19). 645--653.
[14]
Zun Li, Congyan Lang, Jiashi Feng, Yidong Li, Tao Wang, and Songhe Feng. 2019. Co-saliency detection with graph matching. ACM Transactions on Intelligent Systems and Technology 10, 3 (2019), Article 22, 22 pages.
[15]
Geert J. S. Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen A. W. M. van der Laak, Bram van Ginneken, and Clara I. Sánchez. 2017. A survey on deep learning in medical image analysis. Medical Image Analysis 42 (2017), 60--88.
[16]
Hong Liu, Baoxi Liu, Hao Zhang, Liang Li, Xin Qin, and Guijuan Zhang. 2018. Crowd evacuation simulation approach based on navigation knowledge and two-layer control mechanism. Information Sciences 436-437 (2018), 247--267.
[17]
Hong Liu, Bin Xu, Dianjie Lu, and Guijuan Zhang. 2018. A path planning approach for crowd evacuation in buildings based on improved artificial bee colony algorithm. Applied Soft Computing 68 (2018), 360--376.
[18]
Ming-Yu Liu, Oncel Tuzel, Srikumar Ramalingam, and Rama Chellappa. 2011. Entropy rate superpixel segmentation. In Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). 2097--2104.
[19]
Yong-Jin Liu, Minjing Yu, Bing-Jun Li, and Ying He. 2018. Intrinsic manifold SLIC: A simple and efficient method for computing content-sensitive superpixels. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 3 (2018), 653--666.
[20]
Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 3431--3440.
[21]
Alastair Philip Moore, Simon Prince, Jonathan Warrell, Umar Mohammed, and Graham Jones. 2008. Superpixel lattices. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’08). 1--8.
[22]
Dong Nan, Duyan Bi, Shiping Ma, Xiaolong Lou, and Jiacheng Ni. 2014. An adaptive bilateral filtering method based on parameter estimation. Journal of Central South University 45, 11 (2014), 3840--3845.
[23]
Xiao Pan, Yuanfeng Zhou, Zhonggui Chen, and Caiming Zhang. 2019. Texture relative superpixel generation with adaptive parameters. IEEE Transactions on Multimedia 21, 8 (2019), 1997--2011.
[24]
Sérgio Pereira, Adriano Pinto, Victor Alves, and Carlos A. Silva. 2016. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging 35, 5 (2016), 1240--1251.
[25]
David M. W. Powers. 2008. Evaluation evaluation. In Proceedings of the 18th European Conference on Artificial Intelligence (ECAI’08). 843--844.
[26]
Xiaofeng Ren and Jitendra Malik. 2003. Learning a classification model for segmentation. In Proceedings of the 9th International Conference on Computer Vision (ICCV’03). 10--17.
[27]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI’15). 234--241.
[28]
Tim Salimans and Diederik P. Kingma. 2016. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In Advances in Neural Information Processing Systems (NIPS’16). 901--909.
[29]
Jo Schlemper, Jose Caballero, Joseph V. Hajnal, Anthony N. Price, and Daniel Rueckert. 2018. A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Transactions on Medical Imaging 37, 2 (2018), 491--503.
[30]
Jianbo Shi and Jitendra Malik. 2000. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 8 (2000), 888--905.
[31]
Brahim Ait Skourt, Abdelhamid El Hassani, and Aicha Majda. 2018. Lung CT image segmentation using deep neural networks. Procedia Computer Science 127 (2018), 109--113.
[32]
David Stutz, Alexander Hermans, and Bastian Leibe. 2018. Superpixels: An evaluation of the state-of-the-art. Computer Vision and Image Understanding 166 (2018), 1--27.
[33]
John E. Vargas-Muñoz, Ananda S. Chowdhury, Eduardo B. Alexandre, Felipe L. Galvão, Paulo A. Vechiatto Miranda, and Alexandre X. Falcão. 2019. An iterative spanning forest framework for superpixel segmentation. IEEE Transactions on Image Processing 28, 7 (2019), 3477--3489.
[34]
Rubén Vera-Rodríguez, Julian Fiérrez, and Aythami Morales. 2019. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications:23rd Iberoamerican Congress, CIARP 2018, Madrid, Spain, November 19-22, 2018, Proceedings. Lecture Notes in Computer Science, Vol. 11401. Springer.
[35]
Lei Xing, Elizabeth A. Krupinski, and Jing Cai. 2018. Artificial intelligence will soon revolutionize medical physics research and practice. Medical Physics 45, 5 (2018), 1791--1793.
[36]
Yongxia Zhang, Xuemei Li, Xifeng Gao, and Caiming Zhang. 2017. A simple algorithm of superpixel segmentation with boundary constraint. IEEE Transactions on Circuits and Systems for Video Technology 27, 7 (2017), 1502--1514.
[37]
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. 2017. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 6230--6239.
[38]
Chunbiao Zhu, Wenhao Zhang, Thomas H. Li, Shan Liu, and Ge Li. 2019. Exploiting the value of the center-dark channel prior for salient object detection. ACM Transactions on Intelligent Systems and Technology 10, 3 (2019), Article 32, 20 pages.

Cited By

View all
  • (2024)DDNet: Depth Dominant Network for Semantic Segmentation of RGB-D ImagesSensors10.3390/s2421691424:21(6914)Online publication date: 28-Oct-2024
  • (2024)Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid ApproachesBioengineering10.3390/bioengineering1110103411:10(1034)Online publication date: 16-Oct-2024
  • (2024)CGKPN: Cross-Graph Knowledge Propagation Network with Adaptive Connection for Reasoning-Based Machine Reading ComprehensionACM Transactions on Intelligent Systems and Technology10.1145/365867315:4(1-24)Online publication date: 17-Apr-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology
ACM Transactions on Intelligent Systems and Technology  Volume 11, Issue 4
Survey Paper and Regular Paper
August 2020
358 pages
ISSN:2157-6904
EISSN:2157-6912
DOI:10.1145/3401889
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2020
Online AM: 07 May 2020
Accepted: 01 March 2020
Revised: 01 February 2020
Received: 01 December 2019
Published in TIST Volume 11, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Medical image segmentation
  2. bilateral filtering
  3. deep U-Net
  4. normalization layer
  5. superpixel-based classification algorithm

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)86
  • Downloads (Last 6 weeks)2
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)DDNet: Depth Dominant Network for Semantic Segmentation of RGB-D ImagesSensors10.3390/s2421691424:21(6914)Online publication date: 28-Oct-2024
  • (2024)Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid ApproachesBioengineering10.3390/bioengineering1110103411:10(1034)Online publication date: 16-Oct-2024
  • (2024)CGKPN: Cross-Graph Knowledge Propagation Network with Adaptive Connection for Reasoning-Based Machine Reading ComprehensionACM Transactions on Intelligent Systems and Technology10.1145/365867315:4(1-24)Online publication date: 17-Apr-2024
  • (2024)HMSAM-UNet: A Hierarchical Multi-Scale Attention Module-Based Convolutional Neural Network for Improved CT Image SegmentationIEEE Access10.1109/ACCESS.2024.340166912(79415-79427)Online publication date: 2024
  • (2024)Few-shot segmentation based on multi-level and cross-scale clusteringConnection Science10.1080/09540091.2023.228797236:1Online publication date: 29-Feb-2024
  • (2024)When deep learning is not enough: artificial life as a supplementary tool for segmentation of ultrasound images of breast cancerMedical & Biological Engineering & Computing10.1007/s11517-024-03026-xOnline publication date: 18-Mar-2024
  • (2024)Learning Hierarchical Visual Transformation for Domain Generalizable Visual Matching and RecognitionInternational Journal of Computer Vision10.1007/s11263-024-02106-7132:11(4823-4849)Online publication date: 1-Nov-2024
  • (2024)A systematic survey of fuzzy deep learning for uncertain medical dataArtificial Intelligence Review10.1007/s10462-024-10871-757:9Online publication date: 5-Aug-2024
  • (2024)Efficient artificial intelligence approaches for medical image processing in healthcare: comprehensive review, taxonomy, and analysisArtificial Intelligence Review10.1007/s10462-024-10814-257:8Online publication date: 29-Jul-2024
  • (2024)An improved non-local means algorithm for CT image denoisingMultimedia Systems10.1007/s00530-024-01283-230:2Online publication date: 13-Mar-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media