Skip to main content
SpringerLink
Log in

Dimensionality reduction based on multi-local linear regression and global subspace projection distance minimum

  • Theoretical advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Dimensionality reduction is vital in many fields, such as computer vision and pattern recognition. This paper proposes an unsupervised dimensionality reduction algorithm based on multi-local linear regression. The algorithm first divides the high-dimensional data into many localities. Under the criterion of local homeomorphism, the continuous dependency relationship of the high-dimensional data is maintained in each locality in the low-dimensional space. At the same time, due to the overlap of locality divisions, that is, each data may belong to multiple localities. Therefore, the algorithm performs a multi-local linear prediction on each target data point, to better capture the internal geometric structure of the data. Finally, to coordinate the predictions of the target data points by each locality, we require that the variance between the predictions of each locality to the same target point should be as small as possible. We perform experiments on synthetic and real datasets. Compared with the existing advanced algorithms, the experimental results show that the proposed algorithm has good feasibility.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Yao X, Han J, Zhang D, Nie F (2017) Revisiting co-saliency detection: a novel approach based on two-stage multi-view spectral rotation coclustering. IEEE Trans Image Process 26(7):3196–3209

    Article  MathSciNet  Google Scholar 

  2. Wang W, Shen J, Shao L (2018) Video salient object detection via fully convolutional networks. IEEE Trans Image Process 27(1):38–49

    Article  MathSciNet  Google Scholar 

  3. Lu J, Plataniotis KN, Venetsanopoulos AN (2003) Face recognition using LDA-based algorithms. IEEE Trans Neural Netw 14(1):195–200

    Article  Google Scholar 

  4. Zhang D, Meng D, Han J (2017) Co-saliency detection via a selfpaced multiple-instance learning framework. IEEE Trans Pattern Anal Mach Intell 39(5):865–878

    Article  Google Scholar 

  5. Jolliffe I (2002) Principal component analysis. Wiley, Hoboken

    MATH  Google Scholar 

  6. Fukunaga K (2013) Introduction to statistical. Pattern Recognition

  7. Comon P (1994) Independent component analysis, a new concept. Signal Process 36(3):287–314

    Article  Google Scholar 

  8. Schlkopf B, Smola A, Müller K-R (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319

    Article  Google Scholar 

  9. Schlkopf B, Smola A, Müller K-R (1997) Kernel principal component analysis. Artificial Neural Networks-ICANN. Berlin, pp 583–588

  10. Müller K-R, Mika S, Rtsch G, Tsuda K, Schlkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw 12(2):181–201

    Article  Google Scholar 

  11. You D, Hamsici OC, Martinez AM (2011) Kernel optimization in discriminant analysis. IEEE Trans Pattern Anal Mach Intell 33(3):631–638

    Article  Google Scholar 

  12. Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization. ACM 267–273

  13. Seung HS, Lee DD (2000) The manifold ways of perception. Science 290(5500):2268–2269

    Article  Google Scholar 

  14. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  15. Criminisi A, Shotton J, Konukoglu E (2012) Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Found Trends Comput Graph Vis 7(2–3):81–227

    MATH  Google Scholar 

  16. Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):1959–1966

    Article  Google Scholar 

  17. Belkin M, Niyogi P (2002) Laplacian eigenmaps and spectral techniques for embedding and clustering. Proc Adv Neural Inf Process Syst 14:585–591

    Google Scholar 

  18. He X, Cai D, Yan S, Zhang H (2005) Neighborhood preserving embedding

  19. He X, Niyogi P (2005) Locality preserving projections. Proc Adv Neural Inf Process Syst 45:186–197

    Google Scholar 

  20. Zhang Z, Zha H (2003) Nonlinear dimension reduction via local tangent space alignment. Intelligent data engineering and automated learning, international conference, Ideal, Hong Kong, China, March, Revised Papers DBLP

  21. Cai D, He X, Han J, Zhang H-J (2006) Orthogonal laplacian faces for face recognition. IEEE Trans Image Process 15(11):3608–3614

    Article  Google Scholar 

  22. Guo Y, Fu CR, Huang TS (2008) Image-based human age estimation by manifold learning and locally adjusted robust regression. IEEE Trans Image Process 17(7):1178–1188

    Article  MathSciNet  Google Scholar 

  23. Qian J, Yang J, Xu Y (2013) Local structure-based image decomposition for feature extraction with applications to face recognition. IEEE Trans Image Process 22(9):3591–3603

    Article  Google Scholar 

  24. Zhang J, Yu J, Tao D (2018) Local deep-feature alignment for unsupervised dimension reduction. IEEE Trans Image Process 27(5):2420–2432

    Article  MathSciNet  Google Scholar 

  25. Qiao H, Zhang P, Wang D, Zhang B (2013) An explicit nonlinear mapping for manifold learning. IEEE Trans Cybern 43(1):51–63

    Article  Google Scholar 

  26. Yao C, Han J, Nie F, Xiao F (2018) Local regression and global information-embedded dimension reduction. IEEE Trans Neural Netw 29(10):4882–4893

    Article  MathSciNet  Google Scholar 

  27. Wang S, Wang H (2017) Unsupervised feature selection via low-rank approximation and structure learning. Knowl Based Syst 124:70–79

    Article  Google Scholar 

  28. Yin W, Ma Z (2019) LE and LLE regularized nonnegative tucker decomposition for clustering of high dimensional datasets. Neurocomputing 364:77–94

    Article  Google Scholar 

  29. Jossa F, Farinaro E, Panico S, Krogh V, Celentano E, Galasso R, Mancini M, Trevisan M (1994) Serum uric acid and hypertension: the Olivetti heart study. J Hum Hypertension 8(9):677–681

    Google Scholar 

  30. Li Deng T (2012) The MNIST database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Process Mag 29(6):141–142

    Article  Google Scholar 

  31. Sun Y, Gao J, Hong X et al (2016) Heterogeneous tensor decomposition for clustering via manifold optimization. IEEE Trans Pattern Anal Mach Intell 38(3):476–489

    Article  Google Scholar 

  32. Nene SA, Nayar SK, Murase H (1996) Columbia object image library (coil-20). Columbia Univ., New York, NY, USA, Tech. Rep, p CUCS-005-96

  33. Zhang Z, Zhao K (2013) Low-rank matrix approximation with manifold regularization. IEEE Trans Pattern Anal Mach Intel 35(7):1717–1729

    Article  Google Scholar 

  34. Chen J, Ma Z, Liu Y (2013) Local coordinates alignment with global preservation for dimensionality reduction. IEEE Trans Neural Netw Learn Syst 24(1):106–117

    Article  Google Scholar 

  35. Levina E, Bickel PJ (2005) Maximum likelihood estimation of intrinsic dimension

  36. Rozza A, Lombardi G, Ceruti C, Casiraghi E, Campadelli P (2012) Novel high intrinsic dimensionality estimators. Mach Learn J 89(1):37–65

    Article  MathSciNet  Google Scholar 

  37. Costa J, Hero AO (2004) Geodesic entropic graphs for dimension and entropy estimation in manifold learning. IEEE Trans Signal Process 52(8):2210–2221

    Article  MathSciNet  Google Scholar 

  38. Yao C, Han J, Nie F, Xiao F, Li X (2018) Local regression and global information-embedded dimension reduction. IEEE Trans Neural Netw Learn Syst 29(10):4882–4893

    Article  MathSciNet  Google Scholar 

  39. Qiu Y, Zhou G, Wang Y, Zhang Y, Xie S (2020) A generalized graph regularized non-negative tucker decomposition framework for tensor data representation. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2020.2979344

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronics and Information Technology, Sun Yat-Sen University, Guangzhou, 510006, China

    Haidong Huang, Zhengming Ma, Guokai Zhang & Huibin Wu

Authors
  1. Haidong Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengming Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guokai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huibin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengming Ma.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the Natural Science Foundation of China through the Project “Research on Nonlinear Alignment Algorithm of Local Coordinates in Manifold Learning” under Grant 61773022.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, H., Ma, Z., Zhang, G. et al. Dimensionality reduction based on multi-local linear regression and global subspace projection distance minimum. Pattern Anal Applic 24, 1713–1730 (2021). https://doi.org/10.1007/s10044-021-01022-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-021-01022-7

Keywords

Search

Navigation