Skip to main content
Log in

Unsupervised discriminative feature representation via adversarial auto-encoder

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Feature representation is generally applied to reducing the dimensions of high-dimensional data to accelerate the process of data handling and enhance the performance of pattern recognition. However, the dimensionality of data nowadays appears to be a rapidly increasing trend. Existing unsupervised feature representation methods are susceptible to the rapidly increasing dimensionality of data, which may result in learning a meaningless feature that in turn affect their performance in other applications. In this paper, an unsupervised adversarial auto-encoder network is studied. This network is a probability model that combines generative adversarial networks and variational auto-encoder to perform variational inference and aims to generate reconstructed data similar to original data as much as possible. Due to its adversarial training, this model is relatively robust in feature learning compared with other methods. First, the architecture and training strategy of adversarial auto-encoder are presented. We attempt to learn a discriminative feature representation for high-dimensional image data via adversarial auto-encoder and take its advantage into image clustering, which has become a difficult computer vision task recently. Then amounts of comparative experiments are carried out. The comparison contains eight feature representation methods and two recently proposed deep clustering methods performed on eight different publicly available image data sets. Finally, to evaluate their performance, we utilize a K-means clustering on the low-dimensional feature learned from each feature representation algorithm, and select three evaluation metrics including clustering accuracy, adjusted rand index and normalized mutual information, to provide a comparison. Comprehensive experiments prove the usefulness of the learned discriminative feature via adversarial auto-encoder in the tested data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://www.csie.ntu.edu.tw/∼cjlin/libsvmtools/datasets/multiclass.html

  2. http://www.ee.surrey.ac.uk/CVSSP/demos/chars74k/

  3. http://vision.ucsd.edu/∼leekc/ExtYaleDatabase/ExtYaleB.html

  4. http://www.uk.research.att.com/facedatabase.html

  5. http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php

  6. http://www.cs.toronto.edu/∼kriz/cifar.html

  7. https://github.com/zalandoresearch/fashion-mnist

  8. http://www.idiap.ch/resource/gestures/

References

  1. Balasubramanian M, Schwartz EL (2002) The isomap algorithm and topological stability. Science 295 (5552):7–7

    Article  Google Scholar 

  2. Bao S, Song X, Hu G, Yang X, Wang C (2017) Colour face recognition using fuzzy quaternion-based discriminant analysis. International Journal of Machine Learning and Cybernetics, pp 1–11

  3. Barone AVM (2016) Towards cross-lingual distributed representations without parallel text trained with adversarial autoencoders. ACL 2016:121

    Google Scholar 

  4. Belkin M, Niyogi P (2002) Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Advances in Neural Information Processing Systems, pp 585–591

  5. Cai D, He X, Han J, Zhang HJ (2006) Orthogonal laplacianfaces for face recognition. IEEE Trans Image Process 15(11):3608–3614

    Article  Google Scholar 

  6. Cai D, He X, Hu Y, Han J, Huang T (2007) Learning a spatially smooth subspace for face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  7. Cui Z, Li F, Zhang W (2018) Bat algorithm with principal component analysis. International Journal of Machine Learning and Cybernetics, pp 1–20

  8. De Campos TE, Babu BR, Varma M, et al. (2009) Character recognition in natural images. International Conference on Computer Vision Theory and Applications 7(2):273–280

    Google Scholar 

  9. Denton EL, Chintala S, Fergus R, et al. (2015) Deep generative image models using a laplacian pyramid of adversarial networks. In: Advances in Neural Information Processing Systems, pp 1486–1494

  10. Estévez PA, Tesmer M, Perez CA, Zurada JM (2009) Normalized mutual information feature selection. IEEE Trans Neural Netw 20(2):189–201

    Article  Google Scholar 

  11. Floyd RW (1962) Algorithm 97: shortest path. Commun ACM 5(6):345

    Article  Google Scholar 

  12. Georghiades A, Belhumeur P, Kriegman D (2001) From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

    Article  Google Scholar 

  13. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 580–587

  14. Gogna A, Majumdar A, Ward R (2017) Semi-supervised stacked label consistent autoencoder for reconstruction and analysis of biomedical signals. IEEE Trans Biomed Eng 64(9):2196– 2205

    Article  Google Scholar 

  15. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp 2672–2680

  16. Guo G, Li SZ, Chan K (2000) Face recognition by support vector machines. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, IEEE, pp 196–201

  17. Guo X, Gao L, Liu X, Yin J (2017) Improved deep embedded clustering with local structure preservation. In: International Joint Conference on Artificial Intelligence, pp 1753–1759

  18. Hartigan JA, Wong MA (1979) Algorithm as 136: a k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics) 28(1):100–108

    MATH  Google Scholar 

  19. He X, Cai D, Yan S, Zhang HJ (2005) Neighborhood preserving embedding. In: Tenth IEEE International Conference on Computer Vision, vol 2, IEEE, pp 1208–1213

  20. He X, Niyogi P (2004) Locality preserving projections. In: Advances in Neural Information Processing Systems, pp 153– 160

  21. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  Google Scholar 

  22. Huang JZ, Ng MK, Rong H, Li Z (2005) Automated variable weighting in k-means type clustering. IEEE Trans Pattern Anal Mach Intell 27(5):657–668

    Article  Google Scholar 

  23. Joachims T (1998) Text categorization with support vector machines: Learning with many relevant features. In: European Conference on Machine Learning, Springer, pp 137–142

  24. Johnson DB (1973) A note on dijkstra’s shortest path algorithm. Journal of the ACM (JACM) 20(3):385–388

    Article  MathSciNet  Google Scholar 

  25. Kadurin A, Aliper A, Kazennov A, Mamoshina P, Vanhaelen Q, Khrabrov K, Zhavoronkov A (2017) The cornucopia of meaningful leads: Applying deep adversarial autoencoders for new molecule development in oncology. Oncotarget 8(7):10883

    Article  Google Scholar 

  26. Khan J, Alam A, Hussain J, Lee YK (2019) Enswf: effective features extraction and selection in conjunction with ensemble learning methods for document sentiment classification. Appl Intell, pp 1–23

  27. Krizhevsky A, Hinton GE (2009) Learning multiple layers of features from tiny images. Tech. rep., Citeseer

  28. Makhzani A, Shlens J, Jaitly N, Goodfellow I, Frey B (2015) Adversarial autoencoders. arXiv:1511.05644

  29. Marcel S, Bernier O (1999) Hand posture recognition in a body-face centered space. In: International Gesture Workshop, Springer, pp 97–100

    Chapter  Google Scholar 

  30. Mika S, Ratsch G, Weston J, Scholkopf B, Mullers KR (1999) Fisher discriminant analysis with kernels. In: Neural Networks for Signal Processing IX. Proceedings of the 1999 IEEE Signal Processing Society Workshop, IEEE, pp 41–48

  31. Mojarad M, Nejatian S, Parvin H, Mohammadpoor M (2019) A fuzzy clustering ensemble based on cluster clustering and iterative fusion of base clusters. Appl Intell 49(7):2567–2581

    Article  Google Scholar 

  32. Moore B (1981) Principal component analysis in linear systems: Controllability, observability, and model reduction. IEEE Trans Autom Control 26(1):17–32

    Article  MathSciNet  Google Scholar 

  33. Nene SA, Nayar SK, Murase H et al (1996) Columbia object image library (coil-20)

  34. Park HS, Jun CH (2009) A simple and fast algorithm for k-medoids clustering. Expert Syst Appl 36 (2):3336–3341

    Article  Google Scholar 

  35. Proedrou K, Nouretdinov I, Vovk V, Gammerman A (2002) Transductive confidence machines for pattern recognition. In: European Conference on Machine Learning, . Springer, pp 381–390

  36. Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recogn 43(1):331–341

    Article  Google Scholar 

  37. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290 (5500):2323–2326

    Article  Google Scholar 

  38. Samko O, Marshall AD, Rosin PL (2006) Selection of the optimal parameter value for the isomap algorithm. Pattern Recogn Lett 27(9):968–979

    Article  Google Scholar 

  39. Steinley D (2004) Properties of the hubert-arable adjusted rand index. Psychol Methods 9(3):386

    Article  Google Scholar 

  40. Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, pp 586–591

  41. Wang S, Guo W (2017) Robust co-clustering via dual local learning and high-order matrix factorization. Knowl-Based Syst 138:176–187

    Article  Google Scholar 

  42. Wang S, Guo W (2017) Sparse multi-graph embedding for multimodal feature representation. IEEE Transactions on Multimedia 99:1–1

    Google Scholar 

  43. Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemometr Intell Lab Syst 2 (1-3):37–52

    Article  Google Scholar 

  44. Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist:, a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747

  45. Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, pp 478–487

  46. Yu W, Teng X, Liu C (2006) Face recognition using discriminant locality preserving projections. Image Vis Comput 24(3):239–248

    Article  Google Scholar 

  47. Zhang Z, Song Y, Qi H (2017) Age progression/regression by conditional adversarial autoencoder. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5810–5818

  48. Zheng Y, Doermann D (2006) Robust point matching for nonrigid shapes by preserving local neighborhood structures. IEEE Trans Pattern Anal Mach Intell 28(4):643–649

    Article  Google Scholar 

  49. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv:1703.10593

Download references

Acknowledgments

This work was partially supported by the Technology Innovation Platform Project of Fujian Province under Grant (Nos. 2014H2005 and 2009J1007), the National Natural Science Foundation of China (Nos. 61502104 and 61672159), the Fujian Collaborative Innovation Center for Big Data Application in Governments, the Fujian Engineering Research Center of Big Data Analysis and Processing.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shiping Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, W., Cai, J. & Wang, S. Unsupervised discriminative feature representation via adversarial auto-encoder. Appl Intell 50, 1155–1171 (2020). https://doi.org/10.1007/s10489-019-01581-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01581-7

Keywords

Navigation