Skip to main content
SpringerLink
Log in

EAAE: A Generative Adversarial Mechanism Based Classfication Method for Small-scale Datasets

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

When used for small-scale datasets classification tasks, deep neural networks are difficult to train, which results in the network not extracting useful features and the low accuracy of network. This paper proposes a method called Enhanced Adversarial Autoencoders (EAAE) to improve the accuracy of classification in small-scale datasets scenarios. We reuse the decoder of the Adversarial Autoencoders to generate data for the specified category, which expands the training space of the samples. In order to capture the features of the generated data, we also reuse the encoder to generate the corresponding latent label variables and train them in a supervised manner. To maintain and improve the generative power of the decoder, we impose conditional adversarial training on the generated data. Finally, we use the encoder with latent feature extraction capability to complete the classification by transfer learning. To evaluate the performance of the proposed method, we validate on a real Gastric Cancer dataset and five widely used UCI public datasets. Extensive experiments show that EAAE improves significantly in classification accuracy on small-scale datasets and performs better on large-scale datasets compared to popular classification methods. Furthermore, we evaluate the time cost of EAAE and conduct ablation experiments to demonstrate the effectiveness of the various modules of EAAE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  2. Shinde PP, Shah S (2018) A review of machine learning and deep learning applications. In: Proceedings of the Fourth International Conference on Computing Communication Control and Automation, pp 1–6

  3. Pandey SK, Janghel RR (2019) Recent deep learning techniques, challenges and its applications for medical healthcare system: a review. Neural Process Lett 50(2):1907–1935

    Article  Google Scholar 

  4. Yu J, Tan M, Zhang H et al (2019) Hierarchical deep click feature prediction for fine-grained image recognition. IEEE Trans Pattern Anal Mach Intell 44(2):563–578

    Article  Google Scholar 

  5. Yu J, Zhu C, Zhang J et al (2019) Spatial pyramid-enhanced netvlad with weighted triplet loss for place recognition. IEEE Transactions on Neural Networks and Learning Systems 31(2):661–674

    Article  Google Scholar 

  6. Zhao ZQ, Zheng P, Xu S et al (2019) Object detection with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems 30(11):3212–3232

    Article  Google Scholar 

  7. Ayachi R, Afif M, Said Y et al (2020) Traffic signs detection for realworld application of an advanced driving assisting system using deep learning. Neural Process Lett 51(1):837–851

    Article  Google Scholar 

  8. Hong C, Yu J, Zhang J et al (2018) Multimodal face-pose estimation with multitask manifold deep learning. IEEE Trans Industr Inf 15(7):3952–3961

    Article  Google Scholar 

  9. Wang P, Di J (2018) Deep learning-based object classification through multimode fiber via a cnn-architecture specklenet. Appl Opt 57(28):8258–8263

    Article  Google Scholar 

  10. Sharma S, Rana V, Kumar V (2021) Deep learning based semantic personalized recommendation system. International Journal of Information Management Data Insights 1(2):100028

    Article  Google Scholar 

  11. Choi H, Cho K, Bengio Y (2018) Fine-grained attention mechanism for neural machine translation. Neurocomputing 284:171–176

    Article  Google Scholar 

  12. Sun C, Shrivastava A, Singh S, et al (2017) Revisiting unreasonable effectiveness of data in deep learning era. In: Proceedings of the IEEE International Conference on Computer Vision, pp 843–852

  13. Wang Y, Yao Q, Kwok JT et al (2020) Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys (CSUR) 53(3):1–34

    Article  Google Scholar 

  14. Tan C, Sun F, Kong T, et al (2018) A survey on deep transfer learning. In: Proceedings of the Conference on Artificial Neural Networks, pp 270–279

  15. He Y, Ding G (2020) Deep transfer learning for image emotion analysis: Reducing marginal and joint distribution discrepancies together. Neural Process Lett 51(3):2077–2086

    Article  Google Scholar 

  16. Hospedales T, Antoniou A, Micaelli P, et al (2021) Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.05439

  17. Snell J, Swersky K, Zemel R (2017) Prototypical networks for few-shot learning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp 4080–4090

  18. Wang YX, Hebert M (2016) Learning to learn: Model regression networks for easy small sample learning. In: Proceedings of the European Conference on Computer Vision, pp 616–634

  19. Salamon J, Bello JP (2017) Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Signal Process Lett 24(3):279–283

    Article  Google Scholar 

  20. Cubuk ED, Zoph B, Mane D, et al (2019) AutoAugment: Learning augmentation strategies from data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 113–123

  21. Sung F, Yang Y, Zhang L, et al (2018) Learning to compare: Relation network for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1199–1208

  22. Gao R, Hou X, Qin J et al (2020) Zero-VAE-GAN: Generating unseen features for generalized and transductive zero-shot learning. IEEE Trans Image Process 29:3665–3680

    Article  MATH  Google Scholar 

  23. Hong Y, Niu L, Zhang J, et al (2020) F2GAN: Fusing-and-filling gan for few-shot image generation. In: Proceedings of the 28th ACM International Conference on Multimedia, pp 2535–2543

  24. Hong Y, Niu L, Zhang J, et al (2020) MatchingGAN: Matching-based few-shot image generation. In: Proceedings of the IEEE International Conference on Multimedia and Expo, pp 1–6

  25. Goodfellow IJ, Pouget-Abadie J, Mirza M, et al (2014) Generative adversarial nets. In: Proceedings of the Conference on Neural Information Processing Systems, pp 2672–2680

  26. Larsen ABL, Sønderby SK, Larochelle H, et al (2016) Autoencoding beyond pixels using a learned similarity metric. In: Proceedings of the Conference on Machine Learning, pp 1558–1566

  27. Kim JY, Cho SB (2021) Deep cnn transferred from vae and gan for classifying irritating noise in automobile. Neurocomputing 452:395–403

    Article  Google Scholar 

  28. Yu S, Principe JC (2019) Understanding autoencoders with information theoretic concepts. Neural Netw 117:104–123

    Article  MATH  Google Scholar 

  29. Kingma DP, Welling M (2019) An introduction to variational autoencoders. Foundations and Trends in Machine Learning 12(4):307–392

    Article  MATH  Google Scholar 

  30. Hoffman MD, Blei DM, Wang C et al (2013) Stochastic variational inference. J Mach Learn Res 14(1):1303–1347

  31. Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784

  32. Madani A, Moradi M, Karargyris A, et al (2018) Semi-supervised learning with generative adversarial networks for chest x-ray classification with ability of data domain adaptation. In: Proceedings of the International Symposium on Biomedical Imaging, pp 1038–1042

  33. Makhzani A, Shlens J, Jaitly N, et al (2016) Adversarial autoencoders. arXiv preprint arXiv:1511.05644

  34. Li J, Liu W, Zhou Y et al (2020) Domain adaptation with few labeled source samples by graph regularization. Neural Process Lett 51(1):23–39

    Article  Google Scholar 

  35. Liu W, Li J, Liu B et al (2021) Unified crossdomain classification via geometric and statistical adaptations. Pattern Recogn 110:107658

    Article  Google Scholar 

  36. Kornblith S, Shlens J, Le QV (2019) Do better imagenet models transfer better? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2661–2671

  37. Girshick R, Donahue J, Darrell T, et al (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 580–587

  38. Long M, Cao Y, Wang J, et al (2015) Learning transferable features with deep adaptation networks. In: Proceedings of the International Conference on Machine Learning, pp 97–105

  39. Sharif RA, Azizpour H, Sullivan J, et al (2014) CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 806–813

  40. Guo Y, Shi H, Kumar A, et al (2019) SpotTune: transfer learning through adaptive fine-tuning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4805–4814

  41. Blake CL, Merz CJ (1998) Uci repository of machine learning databases

  42. Abu Alfeilat HA, Hassanat AB, Lasassmeh O et al (2019) Effects of distance measure choice on k-nearest neighbor classifier performance: A review. Big Data 7(4):221–248

    Article  Google Scholar 

  43. Grandini M, Bagli E, Visani G (2020) Metrics for multi-class classification: an overview. arXiv preprint arXiv:2008.05756

  44. Elaidi H, Benabbou Z, Abbar H (2018) A comparative study of algorithms constructing decision trees: Id3 and c4. 5. In: Proceedings of the International Conference on Learning and Optimization Algorithms: Theory and Applications, pp 1–5

  45. Patel HH, Prajapati P (2018) Study and analysis of decision tree based classification algorithms. International Journal of Computer Sciences and Engineering 6(10):74–78

    Article  Google Scholar 

  46. Zhou ZH, Feng J (2017) Deep forest: Towards an alternative to deep neural networks. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp 1038–1042

  47. Nalepa J, Kawulok M (2019) Selecting training sets for support vector machines: a review. Artif Intell Rev 52(2):857–900

    Article  Google Scholar 

  48. Rumelhart DE, Hinton GE, Williams RJ (1985) Learning internal representations by error propagation. Parallel Distributed Processing: Explorations in the Microstructure of Cognition 1:318–362

    Google Scholar 

  49. He K, Zhang X, Ren S, et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778

  50. Arık SO, Pfister T (2021) Tabnet: Attentive interpretable tabular learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 6679–6687

  51. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of the International Conference on Machine Learning, pp 448–456

  52. Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Proceedings of the Conference on Learning Representations

  53. Mukkamala MC, Hein M (2017) Variants of rmsprop and adagrad with logarithmic regret bounds. In: Proceedings of the International Conference on Machine Learning, pp 2545–2553

  54. Müller R, Kornblith S, Hinton G (2019) When does label smoothing help? In: Proceedings of the Conference on Neural Information Processing Systems, pp 4696–4705

Download references

Acknowledgements

This work is sponsored by the Science and Technology Planning Project of Guangzhou under Grant No.202103000036, the Guangdong Basic and Applied Basic Research Foundation under Grant No.2021B1515120048, the National Natural Science Foundation of China under Grant No.62072214, the Industry-University-Research Collaboration Project of Zhuhai under Grant No.ZH22017001210048PWC, the International Cooperation Project of Guangdong Province under Grant No.2020A0505100040, and the Open Project Program of Wuhan National Laboratory for Optoelectronics No.2020WNLOKF006. The corresponding author of this paper is Yuhui Deng.

Author information

Authors and Affiliations

  1. Department of Computer Science, Jinan University, Guangzhou, 510632, China

    Ping Chen, Yuhui Deng & Hong Li

  2. Department of Computer Science, Southwest University, Chongqing, 400715, China

    Qiang Zou

  3. School of Business Administration, South China University of Technology, Guangzhou, 510640, China

    Lijuan Lu

Authors
  1. Ping Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuhui Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiang Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lijuan Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuhui Deng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, P., Deng, Y., Zou, Q. et al. EAAE: A Generative Adversarial Mechanism Based Classfication Method for Small-scale Datasets. Neural Process Lett 55, 969–987 (2023). https://doi.org/10.1007/s11063-022-10921-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10921-7

Keywords

Search

Navigation