skip to main content
10.1145/3338533.3366558acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Efficient Dense Modules of Asymmetric Convolution for Real-Time Semantic Segmentation

Published: 10 January 2020 Publication History

Abstract

Real-time semantic segmentation plays an important role in practical applications such as self-driving and robots. Most semantic segmentation research focuses on improving estimation accuracy with little consideration on efficiency. Several previous studies that emphasize high-speed inference often fail to produce high-accuracy segmentation results. In this paper, we propose a novel convolutional network named Efficient Dense modules with Asymmetric convolution (EDANet), which employs an asymmetric convolution structure and incorporates dilated convolution and dense connectivity to achieve high efficiency at low computational cost and model size. EDANet is 2.7 times faster than the existing fast segmentation network, ICNet, while it achieves a similar mIoU score without any additional context module, post-processing scheme, and pretrained model. We evaluate EDANet on Cityscapes and CamVid datasets, and compare it with the other state-of-art systems. Our network can run with the high-resolution inputs at the speed of 108 FPS on one GTX 1080Ti.

References

[1]
V. Badrinarayanan, A. Kendall, and R. Cipolla. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. In TPAMI, 2017.
[2]
G. J. Brostow, J. Shotton, J. Fauqueur, and R. Cipolla. Segmentation and recognition using structure from motion point clouds. In ECCV, 2008.
[3]
L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Semantic image segmentation with deep convolutional nets and fully connected crfs. In ICLR, 2015.
[4]
L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic imae segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. In TPAMI, 2017.
[5]
L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587, 2017.
[6]
L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV, 2018.
[7]
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016.
[8]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
[9]
M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. In IJCV, 2010.
[10]
J. Fu, J. Liu, Y. Wang, and H. Lu. Stacked deconvolutional network for semantic segmentation. arXiv preprint arXiv:1708.04943, 2017.
[11]
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016.
[12]
K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In ECCV, 2016.
[13]
H. Gao, L. Zhuang, and Q. W. Kilian. Densely connected convolutional networks. In CVPR, 2017.
[14]
S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
[15]
S. Jégou, M. Drozdzal, D. Vazquez, A. Romero, and Y. Bengio. The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In CVPRW, 2017.
[16]
D. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
[17]
A. Krizhevsky, I. Sutskever, and G.E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
[18]
J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015.
[19]
S. Mehta, M. Rastegari, A. Caspi, L. Shapiro, and H. Hajishirzi. Espnet:Efficient spatial pyramid of dilated convolutions for semantic segmentation. In ECCV, 2018.
[20]
H. Noh, S. Hong, and B. Han. Learning deconvolution network for semantic segmentation. In ICCV, 2015.
[21]
A. Paszke, A. Chaurasia, S. Kim, and E. Culurciello. Enet: A deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv: 1606.02417, 2016.
[22]
R. P. K. Poudel, U. Bonde, S. Liwicki, and C. Zach. ContextNet: Exploring context and detail for semantic segmentation in real-time. In BMVC, 2018.
[23]
E. Romera, J. M. Alvarez, L. M. Bergasa, and R. Arroyo. Efficient convnet for real-time semantic segmentation. In IEEE Intelligent Vehicles Symposium, 2017.
[24]
O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015.
[25]
F. Shen, R. Gan, S. Yan, and G. Zeng. Semantic segmentation via structured patch prediction, context crf and guidance crf. In CVPR, 2017.
[26]
M. Siam; M. Gamal, M. Abdel-Razek, S. Yogamani, and M. Jagersand. Rtseg: Real-time semantic segmentation comparative study. In ICIP, 2018.
[27]
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
[28]
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from over-fitting. In JMLR, 2014.
[29]
C. Szegedy. V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. In CVPR, 2016.
[30]
M. Treml, J. Arjona-Medina, T. Unterthiner, R. Durgesh, F. Friedmann, P. Schuberth, A. Mayr, M. Heusel, M. Hofmarcher, M. Widrich, B. Nessler, and S. Hochreiter. Speeding up semantic segmentation for autonomous driving. In NIPSW, 2016.
[31]
C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang. Bisenet: Bilateral segmentation network for real-time semantic segmentation. In ECCV, 2018.
[32]
F. Yu and V. Koltun. Multi-scale context aggregation by dilated convolutions. In ICLR, 2016.
[33]
H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia. Pyramid scene parsing network. In CVPR, 2017.
[34]
H. Zhao, X. Qi, X. Shen, J. Shi, and J. Jia. Icnet for real-time semantic segmentation on high-resolution images. In ECCV, 2018.

Cited By

View all
  • (2025)RTLinearFormer: Semantic segmentation with lightweight linear attentionsNeurocomputing10.1016/j.neucom.2025.129489625(129489)Online publication date: Apr-2025
  • (2025)Lightweight and efficient feature fusion real-time semantic segmentation networkImage and Vision Computing10.1016/j.imavis.2024.105408154(105408)Online publication date: Feb-2025
  • (2025)LDN-SNP: SNP-based lightweight deep network for CT image segmentation of COVID-19Expert Systems with Applications10.1016/j.eswa.2024.125793263(125793)Online publication date: Mar-2025
  • Show More Cited By

Index Terms

  1. Efficient Dense Modules of Asymmetric Convolution for Real-Time Semantic Segmentation
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          MMAsia '19: Proceedings of the 1st ACM International Conference on Multimedia in Asia
          December 2019
          403 pages
          ISBN:9781450368414
          DOI:10.1145/3338533
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 10 January 2020

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. Semantic segmentation
          2. fast network design
          3. real-time

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Conference

          MMAsia '19
          Sponsor:
          MMAsia '19: ACM Multimedia Asia
          December 15 - 18, 2019
          Beijing, China

          Acceptance Rates

          MMAsia '19 Paper Acceptance Rate 59 of 204 submissions, 29%;
          Overall Acceptance Rate 59 of 204 submissions, 29%

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)27
          • Downloads (Last 6 weeks)1
          Reflects downloads up to 17 Feb 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2025)RTLinearFormer: Semantic segmentation with lightweight linear attentionsNeurocomputing10.1016/j.neucom.2025.129489625(129489)Online publication date: Apr-2025
          • (2025)Lightweight and efficient feature fusion real-time semantic segmentation networkImage and Vision Computing10.1016/j.imavis.2024.105408154(105408)Online publication date: Feb-2025
          • (2025)LDN-SNP: SNP-based lightweight deep network for CT image segmentation of COVID-19Expert Systems with Applications10.1016/j.eswa.2024.125793263(125793)Online publication date: Mar-2025
          • (2025)An enhanced tree-seed algorithm for global optimization and neural architecture search optimization in medical image segmentationBiomedical Signal Processing and Control10.1016/j.bspc.2024.107457104(107457)Online publication date: Jun-2025
          • (2025)An Asymmetric Semantic Segmentation Model via Lightweight Attention-Guided Feature Enhancement and FusionCognitive Computation10.1007/s12559-025-10407-317:1Online publication date: 21-Jan-2025
          • (2024)SPNet: Dual-Branch Network with Spatial Supplementary Information for Building and Water Segmentation of Remote Sensing ImagesRemote Sensing10.3390/rs1617316116:17(3161)Online publication date: 27-Aug-2024
          • (2024)A Lightweight Network Based on Multi-Scale Asymmetric Convolutional Neural Networks with Attention Mechanism for Ship-Radiated Noise ClassificationJournal of Marine Science and Engineering10.3390/jmse1201013012:1(130)Online publication date: 9-Jan-2024
          • (2024)Real-Time Semantic Segmentation Algorithm for Street Scenes Based on Attention Mechanism and Feature FusionElectronics10.3390/electronics1318369913:18(3699)Online publication date: 18-Sep-2024
          • (2024)LMANet: A Lightweight Asymmetric Semantic Segmentation Network Based on Multi-Scale Feature ExtractionElectronics10.3390/electronics1317336113:17(3361)Online publication date: 23-Aug-2024
          • (2024)A Lightweight and High-Accuracy Model for Pavement Crack SegmentationApplied Sciences10.3390/app14241163214:24(11632)Online publication date: 12-Dec-2024
          • Show More Cited By

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media