Skip to main content
SpringerLink
Log in

A tiny deep capsule network

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The capsule network (CapsNet) is a novel network model that can learn spatial information in images. However, the performance of CapsNet on complex datasets (such as CIFAR10) is limited and it requires a large number of parameters. These disadvantages make CapsNet less useful, especially in some resource-constrained devices. To solve this problem, we propose a novel tiny deep capsule architecture (CapsInfor), which consists of many fast tensor capsule layers (FastCaps) with a novel routing process. CapsInfor requires only a few parameters to achieve satisfactory performance. For example, on CIFAR10, the accuracy of CapsInfor is 9.32% higher than that of CapsNet, but the parameters are reduced by 97.53%. CapsInfor is composed of multiple pipelines each of which processes a kind of image information. To achieve information interaction between pipelines, a novel cross node is proposed to implement pipeline-level capsule routing. A new decision maker is used to analyze the predicted values of pipelines and gives the final classification result. Using these proposed methods, CapsInfor achieves competitive results on CIFAR10, CIFAR100, FMNIST, and SVHN. Besides, it is proved that CapsInfor has satisfactory affine robustness on affNIST. To alleviate the problem that the parameter explosion with increasing the number of classes, a novel two-level classification method is proposed. This method can effectively reduce the parameters of the model on the 10 categories and 100 categories tasks. The experimental results confirm that CapsInfor is a tiny deep capsule model with satisfactory classification accuracy and affine robustness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Available at http://www.cs.toronto.edu/~tijmen/affNIST/.

References

  1. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/. Accessed 8 Sept 2020

  2. Bhamidi SBS, El-Sharkawy M (2020) 3-level residual capsule network for complex datasets. In: IEEE 11th Latin American symposium on circuits and systems, pp 1–4. https://doi.org/10.1109/LASCAS45839.2020.9068990

  3. Chang S, Yang J, Park S, Kwak N (2018) Broadcasting convolutional network for visual relational reasoning. In: European conference on computer vision, pp 780–796. https://doi.org/10.1007/978-3-030-01267-0_46

  4. Chen J, Liu Z (2020) Mask dynamic routing to combined model of deep capsule network and u-net. IEEE Trans Neural Netw Learn Syst 31(7):2653–2664. https://doi.org/10.1109/TNNLS.2020.2984686

    Article  Google Scholar 

  5. Cheng X, He J, He J, Xu H (2019) Cv-capsnet: complex-valued capsule network. IEEE Access 7:85492–85499. https://doi.org/10.1109/ACCESS.2019.2924548

    Article  Google Scholar 

  6. Choi J, Seo H, Im S, Kang M (2019) Attention routing between capsules. In: IEEE/CVF international conference on computer vision workshop, pp 1981–1989. https://doi.org/10.1109/ICCVW.2019.00247

  7. Deliège A, Cioppa A, Droogenbroeck MV (2018) Hitnet: a neural network with capsules embedded in a hit-or-miss layer, extended with hybrid data augmentation and ghost capsules. arXiv preprint arXiv:1806.06519

  8. Dong Y, Fu Y, Wang L, Chen Y, Dong Y, Li J (2020) A sentiment analysis method of capsule network based on bilstm. IEEE Access 8:37014–37020. https://doi.org/10.1109/ACCESS.2020.2973711

    Article  Google Scholar 

  9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Process Syst 27:2672–2680

    Google Scholar 

  10. Gu J, Tresp V (2020) Improving the robustness of capsule networks to image affine transformations. In: IEEE/CVF conference on computer vision and pattern recognition, pp 7285–7293. https://doi.org/10.1109/CVPR42600.2020.00731

  11. Han T, Sun R, Shao F, Sui Y (2020) Feature and spatial relationship coding capsule network. J Electron Imaging 29(2):23004. https://doi.org/10.1117/1.JEI.29.2.023004

    Article  Google Scholar 

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  13. Hinton GE, Sabour S, Frosst N (2018) Matrix capsules with em routing. In: International conference on learning representations

  14. Hsu JT, Kuo CH, Chen DW (2020) Image super-resolution using capsule neural networks. IEEE Access 8:9751–9759. https://doi.org/10.1109/ACCESS.2020.2964292

    Article  Google Scholar 

  15. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. Int Conf Mach Learn 37:448–456

    Google Scholar 

  16. Jeong T, Lee Y, Kim H (2019) Ladder capsule network. Int Conf Mach Learn 97:3071–3079

    Google Scholar 

  17. Kakillioglu B, Ren A, Wang Y, Velipasalar S (2020) 3d capsule networks for object classification with weight pruning. IEEE Access 8:27393–27405. https://doi.org/10.1109/ACCESS.2020.2971950

    Article  Google Scholar 

  18. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  19. Krizhevsky A (2009) Learning multiple layers of features from tiny images. Technical report, University of Toronto, Toronto

  20. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  21. Lei K, Fu Q, Yang M, Liang Y (2020) Tag recommendation by text classification with attention-based capsule network. Neurocomputing 391:65–73. https://doi.org/10.1016/J.NEUCOM.2020.01.091

    Article  Google Scholar 

  22. Lenssen JE, Fey M, Libuschewski P (2018) Group equivariant capsule networks. Adv Neural Inf Process Syst 31:8844–8853

    Google Scholar 

  23. Li HC, Wang WY, Pan L, Li W, Du Q, Tao R (2020) Robust capsule network based on maximum correntropy criterion for hyperspectral image classification. IEEE J Sel Top Appl Earth Obs Remote Sens 13:738–751. https://doi.org/10.1109/JSTARS.2020.2968930

    Article  Google Scholar 

  24. Marchisio A, Bussolino B, Colucci A, Hanif MA, Martina M, Masera G, Shafique M (2019) X-traincaps: Accelerated training of capsule nets through lightweight software optimizations. arXiv preprint arXiv:1905.10142

  25. Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning. In: NIPS workshop deep learning unsupervised feature learning

  26. Paik I, Kwak T, Kim I (2019) Capsule networks need an improved routing algorithm. Asian Conf Mach Learn 101:489–502

    Google Scholar 

  27. Peer D, Stabinger S, Rodriguez-Sanchez A (2019) Limitations of routing-by-agreement based capsule networks. arXiv preprint arXiv:1905.08744

  28. Phaye SSR, Sikka A, Dhall A, Bathula DR (2018) Dense and diverse capsule networks: making the capsules learn better. arXiv preprint arXiv:1805.04001

  29. Pucci R, Micheloni C, Foresti GL, Martinel N (2020) Deep interactive encoding with capsule networks for image classification. Multimed Tools Appl 79(43):32243–32258. https://doi.org/10.1007/s11042-020-09455-8

    Article  Google Scholar 

  30. Rajasegaran J, Jayasundara V, Jayasekara S, Jayasekara H, Seneviratne S, Rodrigo R (2019) Deepcaps: Going deeper with capsule networks. In: IEEE/CVF conference on computer vision and pattern recognition, pp 10725–10733. https://doi.org/10.1109/CVPR.2019.01098

  31. Ren Q, Shang S, He L (2019) Adaptive routing between capsules. arXiv preprint arXiv:1911.08119

  32. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention, pp 234–241. https://doi.org/10.1007/978-3-319-24574-4_28

  33. Rosario VMd, Borin E, Breternitz M (2019) The multi-lane capsule network. IEEE Signal Process Lett 26(7):1006–1010. https://doi.org/10.1109/LSP.2019.2915661

    Article  Google Scholar 

  34. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. Adv Neural Inf Process Syst 30:3856–3866

    Google Scholar 

  35. Sun K, Zhao Y, Jiang B, Cheng T, Xiao B, Liu D, Mu Y, Wang X, Liu W, Wang J (2019) High-resolution representations for labeling pixels and regions. arXiv preprint arXiv:1904.04514

  36. Sun K, Yuan L, Xu H, Wen X (2020) Deep tensor capsule network. IEEE Access 8:96920–96933. https://doi.org/10.1109/ACCESS.2020.2996282

    Article  Google Scholar 

  37. Xi E, Bing S, Jin Y (2017) Capsule network performance on complex data. arXiv preprint arXiv:1712.03480

  38. Xiang C, Zhang L, Tang Y, Zou W, Xu C (2018) Ms-capsnet: a novel multi-scale capsule network. IEEE Signal Process Lett 25(12):1850–1854. https://doi.org/10.1109/LSP.2018.2873892

    Article  Google Scholar 

  39. Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747

  40. Yang S, Lee F, Miao R, Cai J, Chen L, Yao W, Kotani K, Chen Q (2020) Rs-capsnet: an advanced capsule network. IEEE Access 8:85007–85018. https://doi.org/10.1109/ACCESS.2020.2992655

    Article  Google Scholar 

  41. Zhang X, Sun Y, Wang Y, Li Z, Li N, Su J (2019) A novel effective and efficient capsule network via bottleneck residual block and automated gradual pruning. Comput Electr Eng 80:106481. https://doi.org/10.1016/j.compeleceng.2019.106481

    Article  Google Scholar 

  42. Zhao J, Li J, Zhao F, Nie X, Chen Y, Yan S, Feng J (2017) Marginalized cnn: learning deep invariant representations. In: British machine vision conference, pp 127.1–127.12. https://doi.org/10.5244/C.31.127

  43. Zhao Z, Kleinhans A, Sandhu G, Patel I, Unnikrishnan KP (2019) Capsule networks with max-min normalization. arXiv preprint arXiv:1903.09662

Download references

Acknowledgements

The work was supported by the National Natural Science Foundation of China under Grant 61472278, and Major project of Tianjin under Grant 18ZXZNGX00150, and the Key Project of Natural Science Foundation of Tianjin University under Grant 2017ZD13, and the Research Project of Tianjin Municipal Education Commission under Grant 2017KJ255.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, 300384, China

    Kun Sun, Haixia Xu, Liming Yuan & Xianbin Wen

  2. Key Laboratory of Computer Vision and System, Ministry of Education, Tianjin, 300384, China

    Kun Sun, Haixia Xu, Liming Yuan & Xianbin Wen

  3. Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin, 300384, China

    Kun Sun, Haixia Xu, Liming Yuan & Xianbin Wen

Authors
  1. Kun Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haixia Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liming Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xianbin Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianbin Wen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, K., Xu, H., Yuan, L. et al. A tiny deep capsule network. Int. J. Mach. Learn. & Cyber. 13, 989–1004 (2022). https://doi.org/10.1007/s13042-021-01431-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01431-4

Keywords

Search

Navigation