Dimensionality reduction via preserving local information
Introduction
In many applications, many multimedia data in smart cities are characterized by hundreds or even thousands of features, such as images. However, directly using of high dimensional data may significantly degrade the performance of many applications. Dimensionality reduction, as a crucial technique, has yielded a surge of interest research. It is worth noting that finding neighbors is an essential step of graph embedding algorithms for dimensionality reduction. Since Roweis et al. [1] find nearest neighbors to reconstruct each sample to preserve the local geometric structure, which could seek a low-dimensional embedding of high-dimensional inputs, there are many proposed graph embedding algorithms for dimensionality reduction, which reconstruct the samples by its neighbors or preserve the local structure by finding nearest neighbors, such as [[2], [3], [4], [5], [6], [7]]. However, they are all unsupervised algorithms, which cannot work well in the task of classification. To tackle the unsupervised problem, many algorithms have been proposed [[8], [9], [10], [11], [12], [13]], such as [13] proposed a new multi-view discriminative manifold embedding method. They all can preserve geometric structure of samples. However, they ignore the similarity information, which is important in the task of visualization and classification.
Fig. 1 gives a vivid diagram to illustrate the procedure of graph embedding. In Fig. 1, there are two ways finding appropriate neighbors to construct adjacency graphs: Fig. 1 (a1), which is the traditional way constructing adjacency graph; and Fig. 1 (b1), which is the ideal way constructing adjacency graph. And we also have two results: Fig. 1 (a2), which is the result of constructing the adjacency graph with the traditional way; and Fig. 1 (b2), which is the result of constructing the adjacency graph with the ideal way. Our purpose is to make the representations of these two samples as close as possible in the new space whether they are close or not in the high-dimensional space. However, traditional way finds nearest neighbors, measured by Euclidean, constructing adjacency graph. Thus, if two same class samples have a far distance, they cannot establish a relationship, so that they will not be close in the subspace. In the task of classification, we should give priority to the distant samples in the same class, which will make the classification ability of the algorithm seriously deteriorate if we ignore them. The fundamental challenge is how to find the appropriate neighbors to establish the relationship to achieve the same result of ideal way.
Inspired by recent advances [[14], [15], [16], [17], [18], [19], [20], [21]], in this paper, we propose a local similarity preserving (LSP) function for finding appropriate neighbors, which can preserve the similarity information of the samples. Based on LSP function, we also propose two algorithms, LSPD algorithm, which finds the appropriate neighbors and preserves the similarity information by using LSP function; and LSPD algorithm, which also preserves the geometric structure of data based on LSPD. Extensive experiments demonstrate that the effectiveness and efficiency of our algorithms comparing with the state-of-the-art algorithms.
The main contribution of our work is that we propose a local similarity preserving function and two algorithms, LSPD and LSPD based on LSP function. Both LSPD and LSPD outperform the related state-of-the-art algorithms in digits and face data sets. Moreover, we analyze the experiment results, which demonstrate the effectiveness and efficiency of our proposed algorithms.
Section snippets
Related work
Our work is inspired by graph embedding for dimensionality reduction. Therefore, we briefly review some classical algorithms, which are closely related to our work.
Many graph embedding algorithms are proposed in recent years, such as discriminant neighborhood embedding [22], exponential local discriminant embedding [23], orthogonal neighborhood preserving projections [24], locality sensitive discriminant analysis [25], marginal fisher analysis [[26], [27]], local feature discriminant
LSP for graph embedding algorithms
In this section, a local similarity preserving (LSP) function for finding appropriate neighbors is proposed to preserve the local similarity information, and we also present the details of the proposed algorithms.
Experiments
In this section, several experiments are conducted to validate the effectiveness and efficiency of the proposed algorithms.
Conclusion
In this paper, we present a novel local similarity preserving function for preserving the local similarity information in the process of dimensionality reduction. On the basis of LSP function, we further propose two algorithms, LSPD, which preserves the local similarity information of data, and LSPD, which not only preserves the locality similarity information of samples, but also keeps the geometric structure, respectively. Extensive experiments on visualization and classification show the
Acknowledgments
This work is supported in part by the National Science Foundation of China (Grant No. 61472047).
Shangguang Wang received his Ph.D. degree at Beijing University of Posts and Telecommunications in 2011. He is an associate professor at the State Key Laboratory of Networking and Switching Technology (BUPT). He has published more than 100 papers, and played a key role at many international conferences, such as general chair and PC chair. His research interests include service computing, cloud computing, and mobile edge computing. He is a senior member of the IEEE, and the Editor-in-Chief of
References (35)
- et al.
Multi-instance multi-label learning
Artificial Intelligence
(2012) - et al.
Discriminant neighborhood embedding for classification
Pattern Recognit.
(2006) - et al.
Double adjacency graphs-based discriminant neighborhood embedding
Pattern Recognit.
(2015) - et al.
Discriminant structure embedding for image recognition
Neurocomputing
(2016) - et al.
Graph embedding in vector spaces by node attribute statistics
Pattern Recognit.
(2012) - et al.
Nonlinear dimensionality reduction by locally linear embedding
Science
(2000) - et al.
A global geometric framework for nonlinear dimensionality reduction
Science
(2000) - et al.
Laplacian eigenmaps and spectral techniques for embedding and clustering
Adv. Neural Inf. Process. Syst.
(2001) - et al.
Locality-preserving projections
Adv. Neural Inf. Process. Syst.
(2003) - X.F. He, S.C. Yan, Y.X. Hu, H.J. Zhang, Learning a locality preserving subspace for visual recognition, in: Proc. the...
Globally maximizing, locally minimizing: Unsupervised discriminant projection with applications to face and palm biometrics
IEEE Trans. Pattern Anal. Mach. Intell.
Nonlinear dimensionality reduction with local spline embedding
IEEE Trans. Knowl. Data Eng.
Maximum margin projection subspace learning for visual data analysis
IEEE Trans. Image Process.
Multi-view discriminative manifold embedding for pattern classification
J. Intell. Comput.
Cited by (14)
Supervised discriminative dimensionality reduction by learning multiple transformation operators
2021, Expert Systems with ApplicationsCitation Excerpt :Locally Preserving Projection (LPP) is a popular example of graph embedding methods, preserving the local structure of data in the reduced subspace (He & Niyogi, 2004). While the original version of LPP is unsupervised, there are several supervised extensions of LPP (Wang, Ding, Hsu, & Yang, 2020; Zhang, Xue, Lu, & Guo, 2006). Double graphs-based discriminant projection (DGDP) is a recent similar work, which constructs two adjacent graphs instead of one.
Fault diagnosis of rotor based on Local-Global Balanced Orthogonal Discriminant Projection
2021, Measurement: Journal of the International Measurement ConfederationCitation Excerpt :In the light of Fig. 3, the regulate parameters (α and β) are set to be α = 0.9 and β = 0.9. Visual comparison of low-dimensional features: In order to verify the dimensionality reduction and feature extraction effect of LGBODP, LPP, SOLFDA [33], MNMP [34], LSPD [35], ODP and LGBODP algorithms are used to perform dimensionality reduction comparison experiments. In this experiment, the parameter values of each method are obtained by combining grid search method and cross-validation method.
Towards smarter cities: Learning from Internet of Multimedia Things-generated big data
2020, Future Generation Computer SystemsDiscriminative globality and locality preserving graph embedding for dimensionality reduction
2020, Expert Systems with ApplicationsCitation Excerpt :LPP is a linear extension of Laplacian eigenmap (LE) that is manifold learning (Belkin & Niyogi, 2003). Since LPP can well reflect the local geometry of high-dimensional data, its many variants were introduced, such as in Yang, Zhang, Yang, and Niu (2007), Zhang, Xue, Lu, and Guo (2006), Xu, Zhong, Yang, and Zhang (2010), Gao, Liu, Zhang, Gao, and Li (2013), Gou and Zhang (2013), Wang et al. (2017b), Ding and Zhang (2015), Wang, Ding, Hsu, and Yang (2018), Lu, Ding, Xu, and Wang (2018), Yang, Liu, Yu, and Wang (2018) and Wang et al. (2017a). Yang et al. proposed unsupervised discriminant projections as a simplified version of LPP (Yang et al., 2007).
Shangguang Wang received his Ph.D. degree at Beijing University of Posts and Telecommunications in 2011. He is an associate professor at the State Key Laboratory of Networking and Switching Technology (BUPT). He has published more than 100 papers, and played a key role at many international conferences, such as general chair and PC chair. His research interests include service computing, cloud computing, and mobile edge computing. He is a senior member of the IEEE, and the Editor-in-Chief of the International Journal of Web Science.
Chuntao Ding received the B.S. and M.S. degrees from SIAS international University in 2012 and Soochow University in 2015, respectively, both in software engineering. He is currently a Ph.D. at the State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications (BUPT). His research interests include machine learning, mobile edge computing.
Ching-Hsien Hsu is a professor in the Department of Information Engineering and Computer Science, Feng Chia University, Taichung, Taiwan; He was distinguished chair professor at Tianjin University of Technology, China, during 2012–2016. His research includes high performance computing, cloud computing, parallel and distributed systems, big data analytics and intelligence. He has published 200 papers in these areas, including top journals such as IEEE TPDS, IEEE TSC, IEEE TCC, IEEE TETC, IEEE T-SUSC, IEEE Systems, IEEE Network, IEEE Communications, ACM TOMM. Dr. Hsu is serving as editorial board for a number of prestigious journals, including IEEE TSC, IEEE TCC, IJCS, JoCS. He has been acting as an author/co-author or an editor/co-editor of 10 books from Elsevier, Springer, IGI Global, World Scientific and McGraw-Hill. Dr. Hsu was awarded nine times distinguished award for excellence in research from Chung Hua University. He is vice chair of IEEE TCCLD, executive committee of IEEE TCSC, Taiwan Association of Cloud Computing and an IEEE senior member.
Fangchun Yang received his Ph.D.in communications and electronic systems from the Beijing University of Posts and Telecommunication in 1990. He is currently professor at the Beijing University of Posts and Telecommunication, China. He has published six books and more than 80 papers. His current research interests include network intelligence, service computing, communications software, soft-switching technology, and network security. He is a Fellow of the IET.