AMNet: Convolutional Neural Network embeded with Attention Mechanism for Semantic Segmentation
Authors: 
Baiyi Shu, Jiong Mu, Yu ZhuAuthors Info & Claims
Baiyi Shu, Jiong Mu, Yu ZhuAuthors Info & ClaimsPages 261 - 266
Abstract
DeepLabv3+ is one of the most accurate algorithms in semantic segmentation. CBAM is an attention mechanism proposed to improve the performance of obect detection model which can be used in a convolutional neural network. Given an intermediate feature map, CBAM sequentially infers attention maps along two separate dimensions, channel and spatial, then the attention maps are multiplied to the input feature map for adaptive feature refinement. In the image segmentation tasks, in order to achieve the goal of enhancing feature representation and improving segmentation accuracy without extra overheads. In this paper, we proposed AMNet which is an end-to-end semantic segmentation network based on DeepLabv3+ which is embeded with CBAM. Further, CBAM activates when the input image passes through CNN.Channel attention module in CBAM focues on 'what' is meaningful given an input image and spatial attention module focus on 'where'. Our network acheives 77.66% mIoU on the PASCAL VOC2012, which is a 2.73% better mIoU than DeepLabv3+ with 6 batchsize using only one single Nvidia 2080 GPU. Beyond that, for getting a faster segmentation model, we also embed the attention mechanism into ENet, one of the fastest lightweight networks. After our evaluation on the Cityscapes dataset, we got a better performance in the case of fast training speed. The feasibility that attention mechanism can be integrated into semantic segmentaion network is proved.
References
[1]
Alex Krizhevsky, I Sutskever, G Hinton.ImageNet Classification with Deep Convolutional Neural Networks{J}.
[2]
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation{J}. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 39(4):640--651.
[3]
Badrinarayanan V, Kendall A, Cipolla R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Scene Segmentation{J}. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, PP(99):1--1.
[4]
Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs{J}. 2015.
[5]
Noh H, Hong S, Han B. Learning Deconvolution Network for Semantic Segmentation{J}. 2015.
[6]
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation{J}. 2015.
[7]
Huang G, Liu Z, Laurens V D M, et al. Densely Connected Convolutional Networks{J}. 2016.
[8]
G. J. Brostow, J. Shotton, J. Fauqueur, and R. Cipolla, "Segmentation and recognition using structure from motion point clouds," in ECCV, 2008, pp. 44--57.
[9]
Tsai D. Large-scale image annotation using visual synset{C}// IEEE International Conference on Computer Vision. IEEE, 2011.
[10]
Paszke A, Chaurasia A, Kim S, et al. ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation{J}. 2016.
[11]
Woo S, Kim D, Cho D, et al. LinkNet: Relational Embedding for Scene Graph{J}. 2018.
[12]
Woo S, Park J, Lee J Y, et al. CBAM: Convolutional Block Attention Module{J}. 2018.
[13]
Everingham M, Winn J. The PASCAL2 Visual Object Classes Challenge 2012 (VOC2012) Development Kit{J}. International Journal of Computer Vision, 2018.
[14]
Chen L C, Zhu Y, Papandreou G, et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation{J}. 2018.
[15]
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, "The cityscapes dataset for semantic urban scene understanding," in CVPR, 2016
[16]
Lin G, Milan A, Shen C, et al. RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation{J}. 2016.
[17]
Yu F, Koltun V Multi-Scale Context Aggregation by Dilated Convolutions{J}. 2015.
[18]
Krähenbühl, Philipp, Koltun V. Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials{J}. 2012.
[19]
Zheng S, Jayasumana S, Romera-Paredes B, et al. Conditional Random Fields as Recurrent Neural Networks{J}. 2015.
[20]
Geman S, Geman D. Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images{J}. Readings in Computer Vision, 1984, 20(5-6):25--62.
[21]
Ke T W, Hwang J J, Liu Z, et al. Adaptive Affinity Field for Semantic Segmentation{J}. 2018.
[22]
Zhao H, Shi J, Qi X, et al. Pyramid Scene Parsing Network{J}. 2016.
[23]
Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs{J}. 2016.
[24]
Chen L C, Papandreou G, Schroff F, et al. Rethinking Atrous Convolution for Semantic Image Segmentation{J}. 2017.
[25]
He K, Zhang X, Ren S, et al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition{J}. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 37(9):1904--16.
[26]
Hu J, Shen L, Albanie S, et al. Squeeze-and-Excitation Networks{J}. 2017
[27]
Park J, Woo S, Lee J Y, et al. BAM: Bottleneck Attention Module{J}. 2018.
[28]
He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition{J}. 2015.
[29]
Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition{J}. Computer Science, 2014.
[30]
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Computer Vision and Pattern Recognition(CVPR), 2016 IEEE Conference on, IEEE (2016) 2921--2929.
[31]
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. arXiv preprint arXiv:1709.01507 (2017).
[32]
Garcia-Garcia A, Orts-Escolano S, Oprea S, et al. A Review on Deep Learning Techniques Applied to Semantic Segmentation{J}. 2017.
[33]
Howard A G, Zhu M, Chen B, et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications{J}. 2017.
Recommendations
Dynamic deformable attention network (DDANet) for COVID-19 lesions semantic segmentation
Graphical abstractDisplay Omitted
Highlights- Proposed Novel Deformable attention module learns sparse attention filter offsets.
AbstractDeep learning based medical image segmentation is an important step within diagnosis, which relies strongly on capturing sufficient spatial context without requiring too complex models that are hard to train with limited labelled data. ...
PEA-U-Net: Parallel Embedded Attention for Liver and Tumor Segmentation in CT Volumes
Advanced Intelligent Computing Technology and ApplicationsAbstractThe automatic segmentation of liver and tumor on CT is an urgent need in clinical practice. Currently existing high-precision algorithms based on deep learning have complex structures and configurations, or rely heavily on prior knowledge and ...
Semantic segmentation of high-resolution remote sensing images based on a class feature attention mechanism fused with Deeplabv3+
AbstractAiming at solving the problems of inaccurate segmentation of edge targets, inconsistent segmentation of different types of targets, and slow prediction efficiency on semantic segmentation of high-resolution remote sensing images by ...
Highlights- The class feature attention mechanism and Deeplabv3+ are combined for high-resolution remote sensing image segmentation.
Comments
Information & Contributors
Information
Published In
June 2019
293 pages
ISBN:9781450371858
DOI:10.1145/3341069
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 June 2019
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
HPCCT 2019
HPCCT 2019: 2019 The 3rd High Performance Computing and Cluster Technologies Conference
June 22 - 24, 2019
Guangzhou, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 166Total Downloads
- Downloads (Last 12 months)11
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in