Abstract
Capsule network, which performs feature presentations for classification tasks via novel capsule forms, has attracted more and more attention. However, its performance on complex datasets has not been fully utilized. Through an in-depth exploration of Dense Convolutional Network (DenseNet), we propose a novel dense capsule network based on dense capsule layers, named DenseCaps. As far as we know, this is the first attempt to achieve a cross-capsule feature concatenations. This architecture enhances feature reuse by realizing dense connections at capsule-level, and captures different levels of detailed features to improve the performance on color datasets. Extensive experiments and ablation studies prove the proposed model achieves competitive results on multiple benchmark datasets (MNIST, Fashion-MNIST, CIFAR-10, and SVHN).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60(6):84–90
Zoph B, Vasudevan V, Shlens J, Le QV (2018) Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Salt Lake City, pp. 8697–8710
Han H, Li Y, Zhu X (2019) Convolutional neural network learning for generic data classification. Inf Sci 477:448–465
Shang R, He J, Wang J, Xu K, Jiao L, Stolkin R (2020) Dense connection and depthwise separable convolution based CNN for polarimetric SAR image classification. Knowl-Based Syst 194:105542
Wang J, Ma Y, Huang Z, Xue R, Zhao R (2019) Performance analysis and enhancement of deep convolutional neural network. Bus Inf Syst Eng 61(3):311–326
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas, pp. 770–778
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Honolulu, pp. 4700–4708
Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In: Proceedings of the 31st International conference on neural information processing systems. California, pp 3859-3869
Patrick MK, Adekoya AF, Mighty AA, Edward BY (2019) Capsule networks–a survey. J King Saud Univ-Comput Inf Sci 477:448–465. https://doi.org/10.1016/j.ins.2018.10.053
Saqur R, Vivona S (2019) Capsgan: using dynamic routing for generative adversarial networks. In: Science and Information Conference. Las Vegas, pp. 511–525
Sun G, Ding S, Sun T, Zhang C (2021) SA-CapsGAN: using capsule networks with embedded self-attention for generative adversarial network. Neurocomputing 423:399–406
Chao H, Dong L, Liu Y, Lu B (2019) Emotion recognition from multiband EEG signals using CapsNet. Sensors 19(9):2212
Panigrahi S, Das J, Swarnkar T (2020) Capsule network based analysis of histopathological images of oral squamous cell carcinoma. J King Saud Univ-Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2020.11.003
Afshar P, Naderkhani F, Oikonomou A, Rafiee MJ, Mohammadi A, Plataniotis KN (2021) Mixcaps: a capsule network-based mixture of experts for lung nodule malignancy prediction. Pattern Recogn 116:107942
Peer D, Stabinger S, Rodríguez-Sánchez A (2021) Limitation of capsule networks. Pattern Recogn Lett 144:68–67
Phaye SSR, Sikka A, Dhall A, Bathula DR (2018) Multi-level dense capsule networks. In: Asian Conference on Computer Vision. Perth, pp. 577–592
Do Rosario VM, Borin E, Breternitz M (2019) The multi-lane capsule network. IEEE Signal Process Lett 26(7):1006–1010
Cheng X, He J, He J, Xu H (2019) Cv-CapsNet: complex-valued capsule network. IEEE Access 7:85492–85499
Rajasegaran J, Jayasundara V, Jayasekara S, Jayasekara H, Seneviratne S, Rodrigo R (2019) Deepcaps: going deeper with capsule networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, pp. 10725–10733
Choi J, Seo H, Im S, Kang M (2019) Attention routing between capsules. In: Proceedings of the IEEE international conference on computer vision workshops. Seoul, pp. 1981–1989
Zhang S, Zhou Q, Wu X (2018) Fast dynamic routing based on weighted kernel density estimation. In: International Symposium on Artificial Intelligence and Robotics. Nanjing, pp. 301–309
Sabour S, Frosst N, Hinton G (2018) Matrix capsules with EM routing. In: 6th international conference on learning representations. Vancouver, pp 1-15
Deliege A, Cioppa A, Van Droogenbroeck M (2018) Hitnet: a neural network with capsules embedded in a hit-or-miss layer, extended with hybrid data augmentation and ghost capsules. arXiv preprint arXiv:1806.06519
Wu Y, Li J, Wu J, Chang J (2020) Siamese capsule networks with global and local features for text classification. Neurocomputing 390:88–98
Xi E, Bing S, Jin Y (2017). Capsule network performance on complex data. arXiv preprint arXiv:1712.03480
Yang Z, Wang X (2019). Reducing the dilution: an analysis of the information sensitiveness of capsule network with a practical improvement method. arXiv preprint arXiv:1903.10588
LaLonde R, Xu Z, Irmakci I, Jain S, Bagci U (2021) Capsules for biomedical image segmentation. Med Image Anal 68:101889
Deng L (2012) The mnist database of handwritten digit images for machine learning research. IEEE Signal Process Mag 29(6):141–142
Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning. Granada, p.5. http://ufldl.stanford.edu/housenumbers
Nair P, Doshi R, Keselj S (2021) Pushing the limits of capsule networks. arXiv preprint arXiv:2103.08074
Xiong Y, Su G, Ye S, Sun Y, Sun Y (2019) Deeper capsule network for complex data. In: 2019 international joint conference on neural networks. Budapest, pp 1-8
Huang W, Zhou F (2020) DA-CapsNet: dual attention mechanism capsule network. Sci Rep 10(1):1–13
Xiang C, Zhang L, Tang Y, Zou W, Xu C (2018) MS-CapsNet: a novel multi-scale capsule network. IEEE Signal Process Lett 25(12):1850–1854
Acknowledgements
This work is supported by the National Natural Science Foundation of China (No.61976216 and No.61672522).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sun, G., Ding, S., Sun, T. et al. A novel dense capsule network based on dense capsule layers. Appl Intell 52, 3066–3076 (2022). https://doi.org/10.1007/s10489-021-02630-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02630-w