Abstract
Dimensionality reduction is vital in many fields, such as computer vision and pattern recognition. This paper proposes an unsupervised dimensionality reduction algorithm based on multi-local linear regression. The algorithm first divides the high-dimensional data into many localities. Under the criterion of local homeomorphism, the continuous dependency relationship of the high-dimensional data is maintained in each locality in the low-dimensional space. At the same time, due to the overlap of locality divisions, that is, each data may belong to multiple localities. Therefore, the algorithm performs a multi-local linear prediction on each target data point, to better capture the internal geometric structure of the data. Finally, to coordinate the predictions of the target data points by each locality, we require that the variance between the predictions of each locality to the same target point should be as small as possible. We perform experiments on synthetic and real datasets. Compared with the existing advanced algorithms, the experimental results show that the proposed algorithm has good feasibility.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig7_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig8_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig9_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig10_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10044-021-01022-7/MediaObjects/10044_2021_1022_Fig11_HTML.png)
Similar content being viewed by others
References
Yao X, Han J, Zhang D, Nie F (2017) Revisiting co-saliency detection: a novel approach based on two-stage multi-view spectral rotation coclustering. IEEE Trans Image Process 26(7):3196–3209
Wang W, Shen J, Shao L (2018) Video salient object detection via fully convolutional networks. IEEE Trans Image Process 27(1):38–49
Lu J, Plataniotis KN, Venetsanopoulos AN (2003) Face recognition using LDA-based algorithms. IEEE Trans Neural Netw 14(1):195–200
Zhang D, Meng D, Han J (2017) Co-saliency detection via a selfpaced multiple-instance learning framework. IEEE Trans Pattern Anal Mach Intell 39(5):865–878
Jolliffe I (2002) Principal component analysis. Wiley, Hoboken
Fukunaga K (2013) Introduction to statistical. Pattern Recognition
Comon P (1994) Independent component analysis, a new concept. Signal Process 36(3):287–314
Schlkopf B, Smola A, Müller K-R (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319
Schlkopf B, Smola A, Müller K-R (1997) Kernel principal component analysis. Artificial Neural Networks-ICANN. Berlin, pp 583–588
Müller K-R, Mika S, Rtsch G, Tsuda K, Schlkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw 12(2):181–201
You D, Hamsici OC, Martinez AM (2011) Kernel optimization in discriminant analysis. IEEE Trans Pattern Anal Mach Intell 33(3):631–638
Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization. ACM 267–273
Seung HS, Lee DD (2000) The manifold ways of perception. Science 290(5500):2268–2269
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326
Criminisi A, Shotton J, Konukoglu E (2012) Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Found Trends Comput Graph Vis 7(2–3):81–227
Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):1959–1966
Belkin M, Niyogi P (2002) Laplacian eigenmaps and spectral techniques for embedding and clustering. Proc Adv Neural Inf Process Syst 14:585–591
He X, Cai D, Yan S, Zhang H (2005) Neighborhood preserving embedding
He X, Niyogi P (2005) Locality preserving projections. Proc Adv Neural Inf Process Syst 45:186–197
Zhang Z, Zha H (2003) Nonlinear dimension reduction via local tangent space alignment. Intelligent data engineering and automated learning, international conference, Ideal, Hong Kong, China, March, Revised Papers DBLP
Cai D, He X, Han J, Zhang H-J (2006) Orthogonal laplacian faces for face recognition. IEEE Trans Image Process 15(11):3608–3614
Guo Y, Fu CR, Huang TS (2008) Image-based human age estimation by manifold learning and locally adjusted robust regression. IEEE Trans Image Process 17(7):1178–1188
Qian J, Yang J, Xu Y (2013) Local structure-based image decomposition for feature extraction with applications to face recognition. IEEE Trans Image Process 22(9):3591–3603
Zhang J, Yu J, Tao D (2018) Local deep-feature alignment for unsupervised dimension reduction. IEEE Trans Image Process 27(5):2420–2432
Qiao H, Zhang P, Wang D, Zhang B (2013) An explicit nonlinear mapping for manifold learning. IEEE Trans Cybern 43(1):51–63
Yao C, Han J, Nie F, Xiao F (2018) Local regression and global information-embedded dimension reduction. IEEE Trans Neural Netw 29(10):4882–4893
Wang S, Wang H (2017) Unsupervised feature selection via low-rank approximation and structure learning. Knowl Based Syst 124:70–79
Yin W, Ma Z (2019) LE and LLE regularized nonnegative tucker decomposition for clustering of high dimensional datasets. Neurocomputing 364:77–94
Jossa F, Farinaro E, Panico S, Krogh V, Celentano E, Galasso R, Mancini M, Trevisan M (1994) Serum uric acid and hypertension: the Olivetti heart study. J Hum Hypertension 8(9):677–681
Li Deng T (2012) The MNIST database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Process Mag 29(6):141–142
Sun Y, Gao J, Hong X et al (2016) Heterogeneous tensor decomposition for clustering via manifold optimization. IEEE Trans Pattern Anal Mach Intell 38(3):476–489
Nene SA, Nayar SK, Murase H (1996) Columbia object image library (coil-20). Columbia Univ., New York, NY, USA, Tech. Rep, p CUCS-005-96
Zhang Z, Zhao K (2013) Low-rank matrix approximation with manifold regularization. IEEE Trans Pattern Anal Mach Intel 35(7):1717–1729
Chen J, Ma Z, Liu Y (2013) Local coordinates alignment with global preservation for dimensionality reduction. IEEE Trans Neural Netw Learn Syst 24(1):106–117
Levina E, Bickel PJ (2005) Maximum likelihood estimation of intrinsic dimension
Rozza A, Lombardi G, Ceruti C, Casiraghi E, Campadelli P (2012) Novel high intrinsic dimensionality estimators. Mach Learn J 89(1):37–65
Costa J, Hero AO (2004) Geodesic entropic graphs for dimension and entropy estimation in manifold learning. IEEE Trans Signal Process 52(8):2210–2221
Yao C, Han J, Nie F, Xiao F, Li X (2018) Local regression and global information-embedded dimension reduction. IEEE Trans Neural Netw Learn Syst 29(10):4882–4893
Qiu Y, Zhou G, Wang Y, Zhang Y, Xie S (2020) A generalized graph regularized non-negative tucker decomposition framework for tensor data representation. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2020.2979344
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was supported in part by the Natural Science Foundation of China through the Project “Research on Nonlinear Alignment Algorithm of Local Coordinates in Manifold Learning” under Grant 61773022.
Rights and permissions
About this article
Cite this article
Huang, H., Ma, Z., Zhang, G. et al. Dimensionality reduction based on multi-local linear regression and global subspace projection distance minimum. Pattern Anal Applic 24, 1713–1730 (2021). https://doi.org/10.1007/s10044-021-01022-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-021-01022-7