Abstract
A well-designed graph plays a fundamental role in graph-based semi-supervised learning; however, the topological structure of a constructed neighborhood is unstable in most current approaches, since they are very sensitive to the high dimensional, sparse and noisy data. This generally leads to dramatic performance degradation. To deal with this issue, we developed a relative manifold based semi-supervised dimensionality reduction (RMSSDR) approach by utilizing the relative manifold to construct a better neighborhood graph with fewer short-circuit edges. Based on the relative cognitive law and manifold distance, a relative transformation is used to construct the relative space and the relative manifold. A relative transformation can improve the ability to distinguish between data points and reduce the impact of noise such that it may be more intuitive, and the relative manifold can more truly reflect the manifold structure since data sets commonly exist in a nonlinear structure. Specifically, RMSSDR makes full use of pairwise constraints that can define the edge weights of the neighborhood graph by minimizing the local reconstruction error and can preserve the global and local geometric structures of the data set. The experimental results on face data sets demonstrate that RMSSDR is better than the current state of the art comparing methods in both performance of classification and robustness.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Change history
28 October 2015
Figure 8 of this article shows YaleB and CMU PIE with incorrect legend titles: YaleB (Tr=1900, Te=514, NOC=100) should be YaleB (Tr=1900, Te=514, d=100) (Fig. 8(a)); TIE (Tr=1200, Te=2880, d=100) should be PIE (Tr=1200, Te=2880, d=100) (Fig. 8(b)).
28 October 2015
Figure 8 of this article shows YaleB and CMU PIE with incorrect legend titles: YaleB (Tr=1900, Te=514, NOC=100) should be YaleB (Tr=1900, Te=514, d=100) (Fig. 8(a)); TIE (Tr=1200, Te=2880, d=100) should be PIE (Tr=1200, Te=2880, d=100) (Fig. 8(b)).
References
Duda R O, Hart P E, Stork D G. Pattern Classification. New York: John Wiley & Sons, 2001, 566–581
Jolliffe I T. Principal Component Analysis (Springer Series in Statistics). 2nd ed. Springer, 2002
Martinez A M, Kak A C. PCA versus LDA. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(2): 228–233
Roweis S, Saul L. Nonlinear dimensionality reduction by locally linear embedding. Science, 2000, 290: 2323–2326
Belkin M, Niyogi P. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 2003, 15(6): 1373–1396
He X F, Niyogi P. Locality preserving projections. Advances in Neural Information Processing Systems, 2004, 16: 153–160
Tenenbaum J B, Silva V D, Langford J C. A global geometric framework for nonlinear dimensionality reduction. Science, 2000, 290(5500): 2319–2323
Zhang Z Y, Zha H Y. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM Journal of Scientific Computing, 2004, 26(1): 313–338
Zhu X. Semi-Supervised Learning Literature. Technique Report 1530, Department of Computer Sciences, University of Wisconsin-Madison. 2008
Klein D, Kamvar S D, Manning C D. From instance-level constraints to space-level constraints: making the most of prior knowledge in data clustering. In: Proceedings of the 19th International Conference on Machine Learning. 2002, 307–314
Zhang D Q, Zhou Z H, Chen S C. Semi-supervised dimensionality reduction. In: Proceedings of the 7th SIAM International Conference on Data Mining. 2007, 629–634
Cevikalp H, Verbeek J, Jurie F, Klaser A. Semi-supervised dimensionality reduction using pairwise equivalence constraints. In: Proceedings of the 3rd International Conference on Computer Vision Theory and Applications. 2008
Wei J, Peng H. Neighbourhood preserving based semi-supervised dimensionality reduction. Electronics Letters, 2008, 44(20): 1190–1192
Yan S C, Xu D, Zhang B Y, Zhang H J, Yang Q, Lin S. Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Transactions on Pattern Analysis and Machine Intelligent, 2007, 28(1): 40–51
Cai D, He X F, Han J W. Semi-supervised discriminate analysis. In: Proceedings of the 11th IEEE International Conference on Computer Vision. 2007
Belkin M, Niyogi P. Semi-supervised learning on riemannian manifolds. Machine Learning, 2004, 56(1): 209–239
Zhang Z Y, Zha H Y, Zhang M. Spectral methods for semi-supervised manifold learning. In: Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition. 2008
Wen G, Lu T, Jiang L, Wen J. Locally linear embedding based on relative manifold. Journal of Software, 2009, 20(9): 2376–2386
Georghiades A S, Belhumeur P N, Kriegman D J. From few to many: illumination cone models for face recognition under variable lighting and rose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(6): 643–660
Sim T, Barker S, Bsat M. The CMU pose, illumination, and expression database. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(12): 1615–1618
Wen G H, Jiang L J, Wen J, Shadbolt N R. Performing locally linear embedding with adaptable neighborhood size on manifold. In: Proceedings of the 9th Pacific Rim International Conference on Artificial Intelligence. 2006, 985–989
Zhang H, Berg A C, Maire M, Malik J. SVM-KNN: discriminative nearest neighbor classification for visual category recognition. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2006, 2: 2126–2136
Bergman T J, Beehner J C, Cheney D L, Seyfarth R M. Hierarchical classification by rank and kinship in baboons. Science, 2003, 302(5648): 1234–1236
Li DY, Liu C Y, Du Y, Han X. Artificial intelligence with uncertainty. Journal of Software, 2004, 15(11): 1583–1594 (in Chinese)
Wen G H, Jiang L J, Wen J, Wen J, Yu ZW. Perceptual relativity-based local hyperplane classification. Neurocomputing, 2012, 97: 155–163
Wen G H, Pan X J, Jiang L J, Wen J. Modeling Gestalt laws for classification. In: Proceedings of the 9th IEEE International Conference on Cognitive Informatics. 2010, 914–918
Guihua W, Jun W, Li J. Perceptual nearest neighbors for classification. In: Proceedings of the 5th IEEE International Conference on Computing, Theories and Applications. 2010, 118–122
Wen G, Jiang L, Wen J. Local relative transformation with application to isometric embedding. Pattern Recognition Letters, 2009, 30(3): 203–211
WEI Lai, WANG S J. Semi-supervised discriminant analysis based on manifold distance. Journal of Software, 2010, 21(10): 2445–2453
Guo G, Dyer, C R, Learning from examples in the small sample case: face expression recognition. IEEE Transactions on Systems, Man and Cybernetics, 2005, 35: 477–488
Wen G H, Wei J, Wang J B, Zhou T G, Chen L. Cognitive gravitation model for classification on small noisy data. Neurocomputing, 2013, 118: 245–252
Author information
Authors and Affiliations
Corresponding author
Additional information
Xianfa Cai is an associate professor of the School of Medical Information Engineering, Guangdong Pharmaceutical University. Cai received his BS and MS degrees from Jiangxi Normal university and SunYat-sen University in 2002 and 2006, respectively, and obtained his PhD degree from South China University of Technology in 2013. His research focuses on machine learning, pattern recognition, and bioinformatics.
Guihua Wen is a professor and doctoral supervisor. In 2005 and 2006, he did visiting research on machine learning and semantic web in School of Electronics and Computer, University of Southampton, UK. His main research interests are computational creativity, machine learning, knowledge discovery and cognitive geometry. Since 2006, he has proposed some original methods based on the computation of cognitive laws that can effectively solve difficult problems in information science. He has been a Council Member of Chinese Association for Artificial Intelligence and a program committee member of many international conferences. He is also a reviewer for China National Natural Science Foundation.
Jia Wei received his BS and MS degrees in computer science from Harbin Institute of Technology in 2003 and 2006, respectively, and his PhD degree in computer science from the South China University of Technology in 2009. He is now a lecturer in the School of Computer Science and Engineering, South China University of Technology. His research fields include machine learning and image retrieval.
Zhiwen Yu is a professor in the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. Yu is a senior member of IEEE and a senior member of China Computer Federation. He received his BS and MPhil from SunYatsen University in China in 2001 and 2004, respectively, and his PhD degree in computer science from the City University of Hong Kong, China in 2008. His research interests include pattern recognition, bioinformatics, multimedia, intelligent computing, and data mining. Yu has published more than 80 referred journal papers and international conference papers.
Rights and permissions
About this article
Cite this article
Cai, X., Wen, G., Wei, J. et al. Relative manifold based semi-supervised dimensionality reduction. Front. Comput. Sci. 8, 923–932 (2014). https://doi.org/10.1007/s11704-014-3193-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11704-014-3193-8