Abstract
Feature representation is generally applied to reducing the dimensions of high-dimensional data to accelerate the process of data handling and enhance the performance of pattern recognition. However, the dimensionality of data nowadays appears to be a rapidly increasing trend. Existing unsupervised feature representation methods are susceptible to the rapidly increasing dimensionality of data, which may result in learning a meaningless feature that in turn affect their performance in other applications. In this paper, an unsupervised adversarial auto-encoder network is studied. This network is a probability model that combines generative adversarial networks and variational auto-encoder to perform variational inference and aims to generate reconstructed data similar to original data as much as possible. Due to its adversarial training, this model is relatively robust in feature learning compared with other methods. First, the architecture and training strategy of adversarial auto-encoder are presented. We attempt to learn a discriminative feature representation for high-dimensional image data via adversarial auto-encoder and take its advantage into image clustering, which has become a difficult computer vision task recently. Then amounts of comparative experiments are carried out. The comparison contains eight feature representation methods and two recently proposed deep clustering methods performed on eight different publicly available image data sets. Finally, to evaluate their performance, we utilize a K-means clustering on the low-dimensional feature learned from each feature representation algorithm, and select three evaluation metrics including clustering accuracy, adjusted rand index and normalized mutual information, to provide a comparison. Comprehensive experiments prove the usefulness of the learned discriminative feature via adversarial auto-encoder in the tested data sets.
Similar content being viewed by others
Notes
References
Balasubramanian M, Schwartz EL (2002) The isomap algorithm and topological stability. Science 295 (5552):7–7
Bao S, Song X, Hu G, Yang X, Wang C (2017) Colour face recognition using fuzzy quaternion-based discriminant analysis. International Journal of Machine Learning and Cybernetics, pp 1–11
Barone AVM (2016) Towards cross-lingual distributed representations without parallel text trained with adversarial autoencoders. ACL 2016:121
Belkin M, Niyogi P (2002) Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Advances in Neural Information Processing Systems, pp 585–591
Cai D, He X, Han J, Zhang HJ (2006) Orthogonal laplacianfaces for face recognition. IEEE Trans Image Process 15(11):3608–3614
Cai D, He X, Hu Y, Han J, Huang T (2007) Learning a spatially smooth subspace for face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
Cui Z, Li F, Zhang W (2018) Bat algorithm with principal component analysis. International Journal of Machine Learning and Cybernetics, pp 1–20
De Campos TE, Babu BR, Varma M, et al. (2009) Character recognition in natural images. International Conference on Computer Vision Theory and Applications 7(2):273–280
Denton EL, Chintala S, Fergus R, et al. (2015) Deep generative image models using a laplacian pyramid of adversarial networks. In: Advances in Neural Information Processing Systems, pp 1486–1494
Estévez PA, Tesmer M, Perez CA, Zurada JM (2009) Normalized mutual information feature selection. IEEE Trans Neural Netw 20(2):189–201
Floyd RW (1962) Algorithm 97: shortest path. Commun ACM 5(6):345
Georghiades A, Belhumeur P, Kriegman D (2001) From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 580–587
Gogna A, Majumdar A, Ward R (2017) Semi-supervised stacked label consistent autoencoder for reconstruction and analysis of biomedical signals. IEEE Trans Biomed Eng 64(9):2196– 2205
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp 2672–2680
Guo G, Li SZ, Chan K (2000) Face recognition by support vector machines. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, IEEE, pp 196–201
Guo X, Gao L, Liu X, Yin J (2017) Improved deep embedded clustering with local structure preservation. In: International Joint Conference on Artificial Intelligence, pp 1753–1759
Hartigan JA, Wong MA (1979) Algorithm as 136: a k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics) 28(1):100–108
He X, Cai D, Yan S, Zhang HJ (2005) Neighborhood preserving embedding. In: Tenth IEEE International Conference on Computer Vision, vol 2, IEEE, pp 1208–1213
He X, Niyogi P (2004) Locality preserving projections. In: Advances in Neural Information Processing Systems, pp 153– 160
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507
Huang JZ, Ng MK, Rong H, Li Z (2005) Automated variable weighting in k-means type clustering. IEEE Trans Pattern Anal Mach Intell 27(5):657–668
Joachims T (1998) Text categorization with support vector machines: Learning with many relevant features. In: European Conference on Machine Learning, Springer, pp 137–142
Johnson DB (1973) A note on dijkstra’s shortest path algorithm. Journal of the ACM (JACM) 20(3):385–388
Kadurin A, Aliper A, Kazennov A, Mamoshina P, Vanhaelen Q, Khrabrov K, Zhavoronkov A (2017) The cornucopia of meaningful leads: Applying deep adversarial autoencoders for new molecule development in oncology. Oncotarget 8(7):10883
Khan J, Alam A, Hussain J, Lee YK (2019) Enswf: effective features extraction and selection in conjunction with ensemble learning methods for document sentiment classification. Appl Intell, pp 1–23
Krizhevsky A, Hinton GE (2009) Learning multiple layers of features from tiny images. Tech. rep., Citeseer
Makhzani A, Shlens J, Jaitly N, Goodfellow I, Frey B (2015) Adversarial autoencoders. arXiv:1511.05644
Marcel S, Bernier O (1999) Hand posture recognition in a body-face centered space. In: International Gesture Workshop, Springer, pp 97–100
Mika S, Ratsch G, Weston J, Scholkopf B, Mullers KR (1999) Fisher discriminant analysis with kernels. In: Neural Networks for Signal Processing IX. Proceedings of the 1999 IEEE Signal Processing Society Workshop, IEEE, pp 41–48
Mojarad M, Nejatian S, Parvin H, Mohammadpoor M (2019) A fuzzy clustering ensemble based on cluster clustering and iterative fusion of base clusters. Appl Intell 49(7):2567–2581
Moore B (1981) Principal component analysis in linear systems: Controllability, observability, and model reduction. IEEE Trans Autom Control 26(1):17–32
Nene SA, Nayar SK, Murase H et al (1996) Columbia object image library (coil-20)
Park HS, Jun CH (2009) A simple and fast algorithm for k-medoids clustering. Expert Syst Appl 36 (2):3336–3341
Proedrou K, Nouretdinov I, Vovk V, Gammerman A (2002) Transductive confidence machines for pattern recognition. In: European Conference on Machine Learning, . Springer, pp 381–390
Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recogn 43(1):331–341
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290 (5500):2323–2326
Samko O, Marshall AD, Rosin PL (2006) Selection of the optimal parameter value for the isomap algorithm. Pattern Recogn Lett 27(9):968–979
Steinley D (2004) Properties of the hubert-arable adjusted rand index. Psychol Methods 9(3):386
Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, pp 586–591
Wang S, Guo W (2017) Robust co-clustering via dual local learning and high-order matrix factorization. Knowl-Based Syst 138:176–187
Wang S, Guo W (2017) Sparse multi-graph embedding for multimodal feature representation. IEEE Transactions on Multimedia 99:1–1
Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemometr Intell Lab Syst 2 (1-3):37–52
Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist:, a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747
Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, pp 478–487
Yu W, Teng X, Liu C (2006) Face recognition using discriminant locality preserving projections. Image Vis Comput 24(3):239–248
Zhang Z, Song Y, Qi H (2017) Age progression/regression by conditional adversarial autoencoder. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5810–5818
Zheng Y, Doermann D (2006) Robust point matching for nonrigid shapes by preserving local neighborhood structures. IEEE Trans Pattern Anal Mach Intell 28(4):643–649
Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv:1703.10593
Acknowledgments
This work was partially supported by the Technology Innovation Platform Project of Fujian Province under Grant (Nos. 2014H2005 and 2009J1007), the National Natural Science Foundation of China (Nos. 61502104 and 61672159), the Fujian Collaborative Innovation Center for Big Data Application in Governments, the Fujian Engineering Research Center of Big Data Analysis and Processing.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Guo, W., Cai, J. & Wang, S. Unsupervised discriminative feature representation via adversarial auto-encoder. Appl Intell 50, 1155–1171 (2020). https://doi.org/10.1007/s10489-019-01581-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-019-01581-7