Skip to main content
SpringerLink
Log in

Dimensionality reduction algorithm of tensor data based on orthogonal tucker decomposition and local discrimination difference

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Dimensionality Reduction (DR) is a significant subject which have aroused extensive attention of researchers. In this paper, a novel method is proposed to reduce the dimensionality of tensor data based on orthogonal Tucker decomposition model and local discrimination difference regularization (OTDLDD-TDR). The proposed method not only defines the relation between high-dimensional and low-dimensional data spaces from the global view point, but also well preserves both geometric distribution (internal attribute) and tag information (external attribute) belonging to data from the local perspective. For the purpose of modeling the combination of these two types of information, we decompose the tensor data into individual parts on the basis of data distribution in sample space. Then, we calculate the local discrimination difference with regard to each part via the categories assigned in specific application scenarios. We seek to minimize the output of local discrimination difference, and impose it as a regularization constraint on the orthogonal Tucker decomposition model to finally achieve DR for tensor. The proposed method is the first attempt to organically combine tensor Tucker decomposition and local discrimination difference. We conducted comparison experiments for the proposed method, state-of-the-art methods published in recent years, and classic and representative methods. The experimental results in data classification and clustering on eight real-world datasets demonstrate the effectiveness of OTDLDD-TDR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Wang W, Wang R, Huang Z, Shan S, Chen X (2018) Discriminant analysis on riemannian manifold of gaussian distributions for face recognition with image sets. IEEE Trans Image Process 27(1):151–163. https://doi.org/10.1109/TIP.2017.2746993

    Article  MathSciNet  MATH  Google Scholar 

  2. Skogen K, Schulz A, Helseth E, Ganeshan B, Dormagen JB, Server AJAR (2019) . Texture Analysis on Diffusion Tensor Imaging: Discriminating Glioblastoma from Single Brain Metastasis 60(3):356–366

    Google Scholar 

  3. Chen K-X, Wu X-JJCVM (2018) . Component SPD Matrices: A Low-dimensional Discriminative Data Descriptor for Image Set Classification 4(3):245–252

    MathSciNet  Google Scholar 

  4. Han J, Cheng G, Li Z, Zhang D (2018) A unified metric Learning-Based framework for Co-Saliency detection. IEEE Transactions on Circuits and Systems for Video Technology 28(10):2473–2483. https://doi.org/10.1109/TCSVT.2017.2706264

    Article  Google Scholar 

  5. Jolliffe I (2005) Principal component analysis. In: Encyclopedia of statistics in behavioral science. https://doi.org/10.1002/0470013192.bsa501

  6. Li C, Wang BJCNU (2014) Fisher linear discriminant analysis

  7. Balasubramanian M, Schwartz EL, Tenenbaum JB, de Silva V, Langford JCJS (2002) . The Isomap Algorithm and Topological Stability 295(5552):7–7

    Google Scholar 

  8. Zhang Z, Zha H (2003) Nonlinear dimension reduction via local tangent space alignment. In: International conference on intelligent data engineering and automated learning, Springer, pp 477–481

  9. He X, Niyogi PJAinips (2004) . Locality Preserving Projections 16(16):153–160

    Google Scholar 

  10. Belkin M, Niyogi PJNc (2003) . Laplacian Eigenmaps for Dimensionality Reduction and Data Representation 15(6):1373–1396

    Google Scholar 

  11. Roweis ST, LKJs Saul (2000) . Nonlinear Dimensionality Reduction by Locally Linear Embedding 290(5500):2323–2326

    Google Scholar 

  12. Hoffmann HJPr (2007) . Kernel PCA for Novelty Detection 40(3):863–874

    Google Scholar 

  13. Wang G, Shi N, Shu Y, Liu DJNpl (2016) . Embedded Manifold-based Kernel Fisher Discriminant Analysis for Face Recognition 43(1):1–16

    Google Scholar 

  14. Zhang S, Ma Z, Gan WJIToIP (2020) . Dimensionality Reduction for Tensor Data Based on Local Decision Margin Maximization 30:234–248

    Google Scholar 

  15. Hong D, Kolda TG, Duersch JAJSR (2020) . Generalized Canonical Polyadic Tensor Decomposition 62(1):133–163

    Google Scholar 

  16. Sidiropoulos ND, Lathauwer LD, Fu X, Huang K, Papalexakis EE, Faloutsos C (2017) Tensor decomposition for signal processing and machine learning. IEEE Trans Signal Process 65(13):3551–3582. https://doi.org/10.1109/TSP.2017.2690524

    Article  MathSciNet  MATH  Google Scholar 

  17. Oseledets IVJSJoSC (2011) . Tensor-train decomposition 33(5):2295–2317

    Google Scholar 

  18. Bergqvist G, Larsson EG (2010) The Higher-Order singular value decomposition: theory and an application [Lecture notes]. IEEE Signal Proc Mag 27(3):151–154. https://doi.org/10.1109/MSP.2010.936030

    Article  Google Scholar 

  19. Hao N, Horesh L, Kilmer M (2014) Nonnegative tensor decomposition. In: Compressed sensing & sparse filtering, Springer, pp 123–148

  20. Wang C, Li Z, Sarpong B (2021) Multimodal adaptive identity-recognition algorithm fused with gait perception. Big Data Mining and Analytics 4(4):223–232. https://doi.org/10.26599/BDMA.2021.9020006

    Article  Google Scholar 

  21. Kung H, Tu Y, Hsu C (2015) Dual subspace nonnegative graph embedding for Identity-Independent expression recognition. IEEE Trans Inform Forensics Secur 10(3):626–639. https://doi.org/10.1109/TIFS.2015.2390138

    Article  Google Scholar 

  22. Kolda TG (2006) Multilinear operators for higher-order decompositions. Citeseer

  23. Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560. https://doi.org/10.1109/TPAMI.2010.231

    Article  Google Scholar 

  24. Zhang Z, Zhao K (2013) Low-Rank Matrix approximation with manifold regularization. IEEE Trans Pattern Anal Mach Intell 35(7):1717–1729. https://doi.org/10.1109/TPAMI.2012.274

    Article  Google Scholar 

  25. Lu H, Plataniotis KN, Venetsanopoulos AN (2008) MPCA: Multilinear principal component analysis of tensor objects. IEEE Trans Neural Netw 19(1):18–39. https://doi.org/10.1109/TNN.2007.901277

    Article  Google Scholar 

  26. Huang J, Su K, El-Den J, Hu T, Li J (2014) An MPCA/LDA based dimensionality reduction algorithm for face recognition. Math Probl Eng 2014:1–12. https://doi.org/10.1155/2014/393265

    Article  MATH  Google Scholar 

  27. Zhang J, Li X, Jing P, Liu J, Su Y (2018) Low-Rank Regularized heterogeneous tensor decomposition for subspace clustering. IEEE Signal Process Lett 25(3):333–337. https://doi.org/10.1109/LSP.2017.2748604

    Article  Google Scholar 

  28. Li X, Ng MK, Cong G, Ye Y, Wu Q (2017) MR-NTD: Manifold regularization nonnegative tucker decomposition for tensor data dimension reduction and representation. IEEE Trans Neural Netw Learn Syst 28(8):1787–1800. https://doi.org/10.1109/TNNLS.2016.2545400

    Article  MathSciNet  Google Scholar 

  29. Sun Y, Gao J, Hong X, Mishra B, Yin B (2016) Heterogeneous tensor decomposition for clustering via manifold optimization. IEEE Trans Pattern Anal Mach Intell 38(3):476–489. https://doi.org/10.1109/TPAMI.2015.2465901

    Article  Google Scholar 

  30. Jiang B, Ding C, Tang J, Luo B (2019) Image representation and learning with Graph-Laplacian tucker tensor decomposition. IEEE Trans Cybern 49(4):1417–1426. https://doi.org/10.1109/TCYB.2018.2802934

    Article  Google Scholar 

  31. Yuan S, Mao X, Chen L (2017) Multilinear spatial discriminant analysis for dimensionality reduction. IEEE Trans Image Process 26(6):2669–2681. https://doi.org/10.1109/TIP.2017.2685343

    Article  MathSciNet  MATH  Google Scholar 

  32. Broxson BJ (2006) The kronecker product

  33. The Olivetti & Oracle Research Laboratory Face Database of Faces. (2016)

  34. The Yale Face Database B. (2006)

  35. Leibe B, Schiele B (2003) Analyzing appearance and contour based methods for object categorization. In: 2003 IEEE computer society conference on computer vision and pattern recognition. 2003 Proceedings., 18-20 June 2003, pp II–409. https://doi.org/10.1109/CVPR.2003.1211497

  36. The UMIST Face Database. (1998)

  37. Libor Spacek’s Facial Images Databases. (2009)

  38. Hull JJJITopa, Intelligence M (1994) . A database for handwritten text recognition research. 16(5):550–554

    Google Scholar 

  39. Sim T, Baker S, Bsat M (2003) The CMU pose, illumination, and expression database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1618. https://doi.org/10.1109/TPAMI.2003.1251154

    Article  Google Scholar 

  40. Chen X (2011) Cai D Large scale spectral clustering with landmark-based representation. In: Twenty-fifth AAAI conference on artificial intelligence

  41. Estévez PA, Tesmer M, Perez CA, JMJITonn Zurada (2009) . Normalized Mutual Information Feature Selection 20(2):189–201

    Google Scholar 

  42. Pereira DG, Afonso A, Medeiros FMJCiS-S (2015) Computation. Overview of Friedman’s Test and Post-hoc Analysis 44(10):2636–2653

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronics and Information Technology, Sun-Yat-Sen University, Guangzhou, China

    Wenxu Gao, Zhengming Ma & Xuejing Yuan

Authors
  1. Wenxu Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengming Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuejing Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengming Ma.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: an example of cross validation for hyperparameters selection

Appendix: an example of cross validation for hyperparameters selection

k neigbors in classification

k clusters in clustering

Clustering accuracy

Clustering NMI

Classification accuracy

1

1

0.1

0.002990249

0.10736

1

2

0.100333333

0.00332274

0.106546667

1

3

0.102333333

0.005323945

0.106866667

1

4

0.196006667

0.119992083

0.107

1

5

0.252166667

0.201983018

0.106866667

1

6

0.259373333

0.21614893

0.1064

1

7

0.28328

0.249578824

0.108493333

1

8

0.304233333

0.277147051

0.106333333

1

9

0.350946667

0.329120288

0.106826667

1

10

0.405093333

0.358727128

0.107213333

3

1

0.1

0.002990249

0.957733333

3

2

0.100333333

0.00332274

0.959013333

3

3

0.102333333

0.005323945

0.956826667

3

4

0.194266667

0.117861855

0.959346667

3

5

0.262866667

0.216403903

0.959453333

3

6

0.261093333

0.216956456

0.95788

3

7

0.282586667

0.248949389

0.95852

3

8

0.30426

0.277166342

0.958853333

3

9

0.347293333

0.326006921

0.95732

3

10

0.40508

0.357616139

0.95908

5

1

0.1

0.002990249

0.955186667

5

2

0.100333333

0.00332274

0.954426667

5

3

0.102333333

0.005323945

0.955

5

4

0.194253333

0.117770819

0.954546667

5

5

0.25592

0.206837358

0.954466667

5

6

0.2665

0.225643294

0.954293333

5

7

0.28316

0.249486005

0.95528

5

8

0.304206667

0.277028743

0.953746667

5

9

0.34582

0.323015533

0.95588

5

10

0.40506

0.357261795

0.95344

7

1

0.1

0.002990249

0.951573333

7

2

0.100333333

0.00332274

0.950106667

7

3

0.102333333

0.005323945

0.951026667

7

4

0.19602

0.119923619

0.951733333

7

5

0.253406667

0.203957807

0.950666667

k neigbors in classification

k clusters in clustering

Clustering accuracy

Clustering NMI

Classification accuracy

7

6

0.260186667

0.216671227

0.951466667

7

7

0.281993333

0.24838152

0.951693333

7

8

0.304226667

0.277069473

0.95128

7

9

0.350946667

0.328048329

0.951053333

7

10

0.4051

0.35790672

0.950866667

9

1

0.1

0.002990249

0.947053333

9

2

0.100333333

0.00332274

0.948013333

9

3

0.102333333

0.005323945

0.94764

9

4

0.196006667

0.119980383

0.946853333

9

5

0.258686667

0.210747986

0.948026667

9

6

0.260873333

0.217333833

0.946973333

9

7

0.283066667

0.248574859

0.947386667

9

8

0.304246667

0.277099256

0.946333333

9

9

0.34834

0.325486866

0.94812

9

10

0.405026667

0.357813192

0.946666667

11

1

0.1

0.002990249

0.944066667

11

2

0.100333333

0.00332274

0.943533333

11

3

0.102333333

0.005323945

0.945013333

11

4

0.196

0.120011443

0.9424

11

5

0.257146667

0.208703978

0.943466667

11

6

0.270413333

0.230616165

0.944106667

11

7

0.28368

0.249866845

0.94408

11

8

0.30422

0.27710678

0.943546667

11

9

0.351226667

0.331495983

0.94436

11

10

0.40502

0.35760244

0.944426667

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, W., Ma, Z. & Yuan, X. Dimensionality reduction algorithm of tensor data based on orthogonal tucker decomposition and local discrimination difference. Appl Intell 52, 14518–14540 (2022). https://doi.org/10.1007/s10489-022-03165-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03165-4

Keywords

Search

Navigation