Skip to main content
SpringerLink
Log in

Response index: quantitative evaluation index of translational equivariance

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Translational equivariance, one of the properties of Convolutional neural networks(CNNs), directly reflects the coherence of the influence of input at each position on the output. By looking for changes in variability such as translational equivariance, it is possible to determine whether the direction of model fit is correct. A controllable location target is designed to verify the translationlal equivariance of a CNN and then the effect of the CNN’s parameters on positioning errors was investigated. Furthermore, A quantitative method called response index(ResIndex) is proposed in this paper. When the parameters of a CNN are determined, the distribution of the input signal response at each position in the heatmap can be obtained via simple algebraic calculations. Here we demonstrate that translational equivariance is primarily affected by the convolution boundary effect,which can be quantitatively assessed by the ResIndex. Experimental evidence for the Pearson correlation coefficient between the MSE and ResIndex demonstrates that our ResIndex is strongly negatively correlated with the MSE, with the mean Pearson correlation coefficient is -0.9282 on the CIFAR-10 and -0.7837 on COCO. For the first time, a unified quantitative evaluation index called the ResIndex is proposed to measure the translational equivariance of CNN. A complete mathematical derivation and a time-saving calculation method are given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The data come from benchmark datasets and do not raise any ethical issues. COCO and CIFAR-10 are publicly available datasets and have been cited in the paper.

References

  1. Acharjya DP, Mitra A, Zaman N (2022) Deep learning in data analytics. Springer

    Book  Google Scholar 

  2. AlAfandy KA, Omara H, Lazaar M et al (2020) Using classic networks for classifying remote sensing images: Comparative study. Adv Scie Technol Eng Syst J 5(5):770–780

    Article  Google Scholar 

  3. Alguacil A, Pinto WG, Bauerheim M, et al (2021) Effects of boundary conditions in fully convolutional networks for learning spatio-temporal dynamics. In: Joint european conference on machine learning and knowledge discovery in databases, Springer, pp 102–117

  4. Araujo A, Norris W, Sim J (2019) Computing receptive fields of convolutional neural networks. Distill 4(11):e21

    Article  Google Scholar 

  5. Bhatt D, Patel C, Talsania H et al (2021) Cnn variants for computer vision: History, architecture, application, challenges and future scope. Electron 10(20):2470

    Article  Google Scholar 

  6. Cao Z, Simon T, Wei SE, et al (2017) Realtime multi-person 2d pose estimation using part affinity fields. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7291–7299

  7. Chaman A, Dokmanic I (2021) Truly shift-invariant convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 3773–3783

  8. Cheng B, Xiao B, Wang J, et al (2020) Higherhrnet: Scale-aware representation learning for bottom-up human pose estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5386–5395

  9. Chu X, Yang W, Ouyang W, et al (2017) Multi-context attention for human pose estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1831–1840

  10. Cohen T, Welling M (2016) Group equivariant convolutional networks. In: International conference on machine learning, PMLR, pp 2990–2999

  11. Deng J, Dong W, Socher R, et al (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp 248–255

  12. Geng Z, Sun K, Xiao B, et al (2021) Bottom-up human pose estimation via disentangled keypoint regression. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 14676–14686

  13. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT press

  14. Groos D, Ramampiaro H, Ihlen EA (2021) Efficientpose: Scalable single-person pose estimation. Appl Intell 51:2518–2533

    Article  Google Scholar 

  15. Hamey LG (2015) A functional approach to border handling in image processing. In: 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA), IEEE, pp 1–8

  16. Han J, Ding J, Xue N, et al (2021) Redet: A rotation-equivariant detector for aerial object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2786–2795

  17. He K, Zhang X, Ren S, et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  18. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  19. Innamorati C, Ritschel T, Weyrich T et al (2020) Learning on the edge: Investigating boundary filters in cnns. Int J Comput Vis 128(4):773–782

    Article  MATH  Google Scholar 

  20. Islam MA, Kowal M, Jia S, et al (2021) Global pooling, more than meets the eye: Position information is encoded channel-wise in cnns. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 793–801

  21. Kayhan OS, Gemert JCv (2020) On translation invariance in cnns: Convolutional layers can exploit absolute spatial location. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 14274–14285

  22. Kaymak Ç, Uçar A (2019) A brief survey and an application of semantic image segmentation for autonomous driving. Handbook of Deep Learning Applications, pp 161–200

  23. Knuth DE (1992) Two notes on notation. The American Mathematical Monthly 99(5):403. https://doi.org/10.2307/2325085

    Article  MathSciNet  MATH  Google Scholar 

  24. Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases 1(4)

  25. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  26. Law H, Deng J (2018) Cornernet: Detecting objects as paired keypoints. In: Proceedings of the European conference on computer vision (ECCV), pp 734–750

  27. Lenc K, Vedaldi A (2015) Understanding image representations by measuring their equivariance and equivalence. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 991–999

  28. Li B, Wu W, Wang Q, et al (2019) Evolution of siamese visual tracking with very deep networks. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, Long Beach, CA, USA, pp 16–20

  29. Lin TY, Maire M, Belongie S et al (2014) Microsoft coco: Common objects in context. In: Fleet D, Pajdla T, Schiele B et al (eds) Computer Vision - ECCV 2014. Springer International Publishing, Cham, pp 740–755

  30. Lin TY, Maire M, Belongie S, et al (2014) Microsoft coco: Common objects in context. In: European conference on computer vision, Springer, pp 740–755

  31. Liu JJ, Hou Q, Cheng MM, et al (2020) Improving convolutional networks with self-calibrated convolutions. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10096–10105

  32. Liu R, Jia J (2008) Reducing boundary artifacts in image deconvolution. In: 2008 15th IEEE International conference on image processing, IEEE, pp 505–508

  33. Luo W, Li Y, Urtasun R, et al (2016) Understanding the effective receptive field in deep convolutional neural networks. Advances in neural information processing systems, p 29

  34. Manfredi M, Wang Y (2020) Shift equivariance in object detection. In: European Conference on Computer Vision, Springer, pp 32–45

  35. Mouton C, Myburgh JC, Davel MH (2020) Stride and translation invariance in cnns. In: Gerber A (ed) Artificial Intelligence Research. Springer International Publishing, Cham, pp 267–281

  36. Nguyen AD, Choi S, Kim W, et al (2019) Distribution padding in convolutional neural networks. In: 2019 IEEE International Conference on Image Processing (ICIP), IEEE, pp 4275–4279

  37. Shannon CE (1949) Communication in the presence of noise. Proc IRE 37(1):10–21

    Article  MathSciNet  Google Scholar 

  38. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  39. Sun G, Ding S, Sun T et al (2022) A novel dense capsule network based on dense capsule layers. Appl Intell 52(3):3066–3076

    Article  Google Scholar 

  40. Sun K, Xiao B, Liu D, et al (2019) Deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5693–5703

  41. Takahashi R, Matsubara T, Uehara K (2018) Ricap: Random image cropping and patching data augmentation for deep cnns. In: Asian conference on machine learning, PMLR, pp 786–798

  42. Tian Z, Shen C, Chen H, et al (2019) Fcos: Fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9627–9636

  43. Wang TH, Huang HJ, Lin JT, et al (2018) Omnidirectional cnn for visual place recognition and navigation. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE, pp 2341–2348

  44. Wang Z, Chen J, Hoi SC (2020) Deep learning for image super-resolution: A survey. IEEE Trans Pattern Anal Mach Intell 43(10):3365–3387

    Article  Google Scholar 

  45. Weiler M, Hamprecht FA, Storath M (2018) Learning steerable filters for rotation equivariant cnns. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 849–858

  46. Wohlberg B (2016) Boundary handling for convolutional sparse representations. In: 2016 IEEE International Conference on Image Processing (ICIP), IEEE, pp 1833–1837

  47. Worrall DE, Garbin SJ, Turmukhambetov D, et al (2017) Harmonic networks: Deep translation and rotation equivariance. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5028–5037

  48. Xie E, Sun P, Song X, et al (2020) Polarmask: Single shot instance segmentation with polar representation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12193–12202

  49. Xie S, Girshick R, Dollár P, et al (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500

  50. Zaman N, Gaur L, Humayun M (2022) Approaches and applications of deep learning in virtual medical care. IGI Global

  51. Zhang R (2019) Making convolutional networks shift-invariant again. In: International conference on machine learning, PMLR, pp 7324–7334

  52. Zhou H, Zhang C, Zhang X et al (2023) Image classification based on quaternion-valued capsule network. Appl Intell 53(5):5587–5606

    Google Scholar 

  53. Zou N, Xiang Z, Chen Y et al (2019) Boundary-aware cnn for semantic segmentation. IEEE Access 7:114520–114528

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Optics and Photonics, Beijing Institute of Technology, No. 5, Zhongguancun South St., Haidian Dist., Beijing, 100081, China

    Peng Yang, Lingqin Kong, Ming Liu, Ge Tang, Liquan Dong, Yuejin Zhao, Xuhong Chu & Mei Hui

  2. Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, Beijing, 100081, China

    Lingqin Kong, Ming Liu, Ge Tang, Liquan Dong, Yuejin Zhao & Xuhong Chu

  3. Yangtze Delta Region Academy of Beijing Institute of Technology, School of Optics and Photonics, Jiaxing, 314019, Zhejiang, China

    Lingqin Kong, Liquan Dong & Xuhong Chu

Authors
  1. Peng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lingqin Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ge Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liquan Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuejin Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xuhong Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Mei Hui
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Peng Yang and Lingqin Kong conceived of the presented idea. Peng Yang, Ming Liu and Ge Tang developed the theory and performed the computations. Yuejin Zhao encouraged PengYang to investigate boundary effects and supervised the findings of this work. Ming Liu, Liquan Dong, Xuhong Chu, and Hui Mei verified the analytical methods. All authors discussed the results and contributed to the final manuscript.

Corresponding author

Correspondence to Ming Liu.

Ethics declarations

Conflicts of interest

The authors declare that they have no known competing financial interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, P., Kong, L., Liu, M. et al. Response index: quantitative evaluation index of translational equivariance. Appl Intell 53, 28642–28654 (2023). https://doi.org/10.1007/s10489-023-05021-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05021-5

Keywords

Search

Navigation