Skip to main content
SpringerLink
Log in

SRK-Augment: A self-replacement and discriminative region keeping augmentation scheme for better classification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Data augmentation has been proved to be a critical and effective way to alleviate over-fitting of deep learning model. Region-level removal is one of state-of-the-art solutions, which can not only synthesize vicinity samples, but also improve generalization of model. However, region removing using random strategy tends to make the training samples suffer from excessive information loss and the introduction of negative noise. In this paper, we propose a novel data augmentation scheme called Self-Replacement-and-Keeping-Augment (SRK-Augment), which exploits self-deformation data as the replacement template and keeps discriminative parts guided by Class Activation Map (CAM) in input image. Concretely, we firstly exploit Grad-CAM++ algorithm to calculate the CAM mask of the input image, and design a patch-shuffling mechanism (PS-operator) to obtain the structural self-deformation template. Then, we utilize the self-deformation template to fill the information removal area, as well as we apply the binary CAM mask to recover the discriminative regions. Finally, these augmented data will be randomly used for model training. The proposed method is simple to implement and can be incorporated with existing augmentation strategies with low computational cost. Extensive experiments are conducted on the challenging datasets. With the help of the SRK-Augment strategy, the performances of DCNNs have achieved obvious improvements. On CIFAR-10 dataset, the Top-1 error rate is dropped by 2.07% at most; On CIFAR-100 dataset, the Top-1 error rate is decreased by up to 3.73%; On Mini-ImageNet dataset, the maximum decline of the Top-1 error rate is 3.38%; On Pascal VOC dataset, the mean Average Precision increases by a maximum of 1.38%. Experimental results manifest the effectiveness and generality of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Han C (2021) Pathology-aware generative adversarial networks for medical image augmentation. arXiv preprint arXiv:2106.01915

  2. Wang M, Deng W (2021) Deep face recognition: a survey. Neurocomputing 429:215–244

    Article  Google Scholar 

  3. Liu X, Wang L, Han X (2022) Transformer with peak suppression and knowledge guidance for fine-grained image recognition. Neurocomputing 492:137–149

    Article  Google Scholar 

  4. Song Y, Li Q, Feng D et al (2016) Texture image classification with discriminative neural networks. Comput Vis Media 2(4):367–377

    Article  Google Scholar 

  5. Meng L, Xu J, Tan X et al (2021) MixSpeech: data augmentation for low-resource automatic speech recognition. In: ICASSP 2021-2021 IEEE International conference on acoustics, speech and signal processing (ICASSP). IEEE, 7008–7012

  6. Park DS, Chan W, Zhang Y et al (2019) Specaugment: a simple data augmentation method for automatic speech recognition. arXiv preprint arXiv:1904.08779

  7. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1–48

    Article  Google Scholar 

  8. Ghiasi G, Cui Y, Srinivas A et al (2021) Simple copy-paste is a strong data augmentation method for instance segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2918–2928

  9. Zhang S, Huang J, Metaxas D et al (2010) Discriminative sparse representations for cervigram image segmentation[C]//2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. IEEE, pp 133–136

  10. Srivastava N, Hinton G, Krizhevsky A et al (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  11. Naveed H (2021) Survey: image mixing and deleting for data augmentation. arXiv preprint arXiv:2106.07085

  12. Wu Z, Meng M, Wu J (2020) Visual sentiment prediction with attribute augmentation and multi-attention mechanism. Neural Process Lett 51(3):2403–2416

    Article  Google Scholar 

  13. Dai D, Wang J, Chen Z et al (2021) Image guidance based 3D vehicle detection in traffic scene. Neurocomputing 428:1–11

    Article  Google Scholar 

  14. Fabian Z, Heckel R, Soltanolkotabi M (2021) Data augmentation for deep learning based accelerated MRI reconstruction with limited data. In: International conference on machine learning. PMLR pp 3057-3067

  15. Li R, Bastiani M, Auer D et al (2021) Image augmentation using a task guided generative adversarial network for age estimation on brain MRI. In: Annual conference on medical image understanding and analysis. Springer, Cham, pp 350–360

  16. Thurnhofer-Hemsi K, Dominguez E (2021) A convolutional neural network framework for accurate skin cancer detection. Neural Process Lett 53(5):3073–3093

    Article  Google Scholar 

  17. Shankar VG, Sisodia DS, Chandrakar P (2021) A novel discriminant feature selection based mutual information extraction from MR brain images for Alzheimer’s stages detection and prediction. Int J Imaging Syst Tech

  18. Nirthika R, Manivannan S, Ramanan A et al (2022) Pooling in convolutional neural networks for medical image analysis: a survey and an empirical study. Neural Comput Appl, 1–27

  19. Zhong Z, Zheng L, Kang G et al (2020) Random erasing data augmentation. In: Proceedings of the AAAI conference on artificial intelligence 34(07):13001–13008

  20. DeVries T, Taylor GW (2017) Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552

  21. Chen P, Liu S, Zhao H et al (2020) Gridmask data augmentation. arXiv preprint arXiv:2001.04086

  22. Singh KK, Lee YJ (2017) Hide-and-seek: forcing a network to be meticulous for weakly-supervised object and action localization. In: 2017 IEEE international conference on computer vision (ICCV). IEEE, pp 3544–3553

  23. Gong C, Wang D, Li M et al (2021) KeepAugment: a simple information-preserving data augmentation approach. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1055–1064

  24. Wei L, Xiao A, Xie L et al (2020) Circumventing outliers of autoaugment with knowledge distillation. In: Computer vision-ECCV, 16th european conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16. Springer International Publishing 2020:608–625

  25. Gontijo-Lopes R, Smullin SJ, Cubuk ED et al (2020) Affinity and diversity: quantifying mechanisms of data augmentation. arXiv preprint arXiv:2002.08973

  26. Uddin AFM, Monira M, Shin W et al (2020) Saliencymix: a saliency guided data augmentation strategy for better regularization. arXiv preprint arXiv:2006.01791

  27. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. PMLR, pp 448–456

  28. Salimans T, Kingma DP (2016) Weight normalization: a simple reparameterization to accelerate training of deep neural networks. Adv Neural Inf Process Syst 29:901–909

    Google Scholar 

  29. Wu Y, He K (2018) Group normalization. In: Proceedings of the European conference on computer vision (ECCV). pp 3-19

  30. Kim B, Ko Y, Seo J (2022) Novel regularization method for the class imbalance problem. Expert Syst Appl 188:115974

    Article  Google Scholar 

  31. Lin D, Li Y, Prasad S et al (2021) CAM-guided multi-path decoding U-net with triplet feature regularization for defect detection and segmentation. Knowl-Based Syst 228:107272

    Article  Google Scholar 

  32. Hinton GE, Srivastava N, Krizhevsky A et al (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580

  33. Wan L, Zeiler M, Zhang S et al (2013) Regularization of neural networks using dropconnect. In: International conference on machine learning. PMLR, pp 1058–1066

  34. Liang S, Khoo Y, Yang H (2021) Drop-activation: implicit parameter reduction and harmonious regularization. Comm App Math Comp Sci 3(2):293–311

    Article  MathSciNet  MATH  Google Scholar 

  35. Wu L, Li J, Wang Y et al (2021) R-drop: regularized dropout for neural networks. Adv Neural Inf Process Syst 34:10890–10905

    Google Scholar 

  36. Cubuk ED, Zoph B, Mane D et al (2019) Autoaugment: learning augmentation strategies from data. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 113–123

  37. Lemley J, Bazrafkan S, Corcoran P (2017) Smart augmentation learning an optimal data augmentation strategy. IEEE Access 5:5858–5869

    Article  Google Scholar 

  38. Lim S, Kim I, Kim T et al (2019) Fast autoaugment. Adv Neural Inf Process Syst 32:6665–6675

    Google Scholar 

  39. Cubuk ED, Zoph B, Shlens J et al (2020) Randaugment: Practical automated data augmentation with a reduced search space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 702–703

  40. May P (2019) Improved image augmentation for convolutional neural networks by copyout and copypairing. arXiv preprint arXiv:1909.00390

  41. Summers C, Dinneen MJ (2019) Improved mixed-example data augmentation. In: 2019 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1262–1270

  42. Zhang H, Cisse M, Dauphin YN et al (2017) mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412

  43. Guo H, Mao Y, Zhang R (2019) Mixup as locally linear out-of-manifold regularization. In: Proceedings of the AAAI conference on artificial intelligence. 33(01):3714–3722

  44. Takahashi R, Matsubara T, Uehara K (2019) Data augmentation using random image cropping and patching for deep CNNs. IEEE Trans Circuits Syst Video Technol 30(9):2917–2931

    Article  Google Scholar 

  45. Chattopadhay A, Sarkar A, Howlader P et al (2018) Grad-cam++: generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 839–847

  46. Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images

  47. Peng Z, Li Z, Zhang J et al (2019) Few-shot image recognition with knowledge transfer. In: Proceedings of the IEEE/CVF international conference on computer vision. pp 441–449

  48. Everingham M, Van Gool L, Williams CKI et al (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  49. Lin TY, Goyal P, Girshick R et al (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  50. Ma N, Zhang X, Zheng HT et al (2018) Shufflenet v2: Practical guide-lines for efficient cnn architecture design. In: Proceedings of the European conference on computer vision (ECCV), pp 116–131

  51. Howard A, Sandler M, Chu G et al (2019) Searching for mobilenetv3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1314–1324

Download references

Acknowledgements

This work was supported by the National Natural Science Found of China (Grant No. 62103393).

Author information

Authors and Affiliations

  1. Department of Automation, University of Science and Technology of China, Huangshan Road, Hefei, 230026, Anhui, China

    Hao Zhao, Jikai Wang, Zonghai Chen, Shiqi Lin & Peng Bao

  2. School of Information Engineering, Southwest University of Science and Technology, Qinglong Avenue, Mianyang, 610000, Sichuan, China

    Hao Zhao

Authors
  1. Hao Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jikai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zonghai Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shiqi Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peng Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zonghai Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, H., Wang, J., Chen, Z. et al. SRK-Augment: A self-replacement and discriminative region keeping augmentation scheme for better classification. Neural Process Lett 55, 3533–3549 (2023). https://doi.org/10.1007/s11063-022-11022-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-11022-1

Keywords

Search

Navigation