research-article

Robust Adaptive Linear Discriminant Analysis with Bidirectional Reconstruction Constraint

Authors:

Baocai YinAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data (TKDD), Volume 14, Issue 6

Article No.: 75, Pages 1 - 20

https://doi.org/10.1145/3409478

Published: 28 September 2020 Publication History

Abstract

Linear discriminant analysis (LDA) is a well-known supervised method for dimensionality reduction in which the global structure of data can be preserved. The classical LDA is sensitive to the noises, and the projection direction of LDA cannot preserve the main energy. This article proposes a novel feature extraction model with l_2,1 norm constraint based on LDA, termed as RALDA. This model preserves within-class local structure in the latent subspace according to the label information. To reduce information loss, it learns a projection matrix and an inverse projection matrix simultaneously. By introducing an implicit variable and matrix norm transformation, the alternating direction multiple method with updating variables is designed to solve the RALDA model. Moreover, both computational complexity and weak convergence property of the proposed algorithm are investigated. The experimental results on several public databases have demonstrated the effectiveness of our proposed method.

Supplementary Material

a75-guo-suppl.pdf (guo.zip)

Supplemental movie, appendix, image and software files for, On Fundamental Principles for Thermal-Aware Design on Periodic Real-Time Multi-Core Systems

Download
99.39 KB

References

[1]

A. Antoniadis, S. Lambert-Lacroix, and F. Leblanc. 2003. Effective dimension reduction methods for tumor classification using gene expression data. Bioinformatics 19, 5 (2003), 563--570.

[2]

P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. 1997. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 7 (1997), 711--720.

Digital Library

[3]

S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning 3, 1 (2011), 1--122.

[4]

V. Brattka, G. Gherardi, and A. Marcone. 2012. The Bolzano–Weierstrass theorem is the jump of weak Kőnig’s lemma. Annals of Pure and Applied Logic 163, 6 (2012), 623--655.

[5]

D. Cai, X. He, J. Han, and T. S. Huang. 2010. Graph regularized nonnegative matrix factorization for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 8 (2010), 1548--1560.

[6]

D. Cai, X. He, Y. Hu, J. Han, and T. S. Huang. 2007. Learning a spatially smooth subspace for face recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 1--7.

[7]

H. Chen, J. Li, J. Gao, Y. Sun, Y. Hu, and B. Yin. 2019. Maximally correlated principal component analysis based on deep parameterization learning. ACM Transactions on Knowledge Discovery from Data 13, 4 (2019), 1--17.

[8]

M. Chen, Q. Wang, and X. Li. 2018a. Discriminant analysis with graph learning for hyperspectral image classification. Remote Sensing 10, 6 (2018), 836.

[9]

X. Chen, G. Yuan, W. Wang, F. Nie, X. Chang, and J. Z. Huang. 2018b. Local adaptive projection framework for feature selection of labeled and unlabeled data. IEEE Transactions on Neural Networks and Learning Systems 29, 12 (2018), 6362--6373.

[10]

D. Díaz-Vico and J. R. Dorronsoro. 2019. Deep least squares Fisher discriminant analysis. IEEE Transactions on Neural Networks and Learning Systems 31, 8 (2019), 2752–2763.

[11]

Z. Fan, Y. Xu, and D. Zhang. 2011. Local linear discriminant analysis framework using sample neighbors. IEEE Transactions on Neural Networks 22, 7 (2011), 1119--1132.

Digital Library

[12]

D. B. Graham and N. M. Allinson. 1998. Characterising virtual eigensignatures for general purpose face recognition. In Face Recognition. Springer, 446--456.

[13]

R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker. 2010. Multi-pie. Image and Vision Computing 28, 5 (2010), 807--813.

Digital Library

[14]

J. Guo, W. Yin, Y. Sun, and Y. Hu. 2019. Multi-view subspace clustering with block diagonal representation. IEEE Access 7 (2019), 84829--84838.

[15]

M. Guo, F. Nie, and X. Li. 2018. Self-weighted adaptive locality discriminant analysis. In Proceedings of the 2018 25th IEEE International Conference on Image Processing. 3378--3382.

[16]

X. He, D. Cai, S. Yan, and H. J. Zhang. 2005a. Neighborhood preserving embedding. In Proceedings of the IEEE International Conference on Computer Vision, Vol. 2. 1208--1213.

[17]

X. He and P. Niyogi. 2004. Locality preserving projections. In Proceedings of the 16th International Conference on Neural Information Processing Systems. 153--160.

[18]

X. He, S. Yan, Y. Hu, P. Niyogi, and H. J. Zhang. 2005b. Face recognition using laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 3 (2005), 328--340.

Digital Library

[19]

C. Hou, F. Nie, X. Li, D. Yi, and Y. Wu. 2013. Joint embedding learning and sparse regression: A framework for unsupervised feature selection. IEEE Transactions on Cybernetics 44, 6 (2013), 793--804.

[20]

Q. Hou, Y. Wang, L. Jing, and H. Chen. 2019. Linear discriminant analysis based on kernel-based possibilistic c-means for hyperspectral images. IEEE Geoscience and Remote Sensing Letters 16, 8 (2019), 1259--1263.

[21]

P. Hu, D. Peng, Y. Sang, and Y. Xiang. 2019. Multi-view linear discriminant analysis network. IEEE Transactions on Image Processing 28, 11 (2019), 5352--5365.

Digital Library

[22]

X. Hu, Y. Sun, J. Gao, Y. Hu, and B. Yin. 2018. Locality preserving projection based on F-norm. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 1330--1337.

[23]

Q. Ji, Y. Sun, J. Gao, Y. Hu, and B. Yin. 2019. Nonlinear subspace clustering via adaptive graph regularized autoencoder. IEEE Access 7 (2019), 74122--74133.

[24]

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 11 (1998), 2278--2324.

[25]

K. C. Lee, J. Ho, and D. J. Kriegman. 2005. Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 5 (2005), 684--698.

Digital Library

[26]

C. N Li, Y. H Shao, W. Yin, and M. Z. Liu. 2020b. Robust and sparse linear discriminant analysis via an alternating direction method of multipliers. IEEE Transactions on Neural Networks and Learning Systems 31, 3 (2020), 915--926.

[27]

L. Li, M. Doroslovački, and M. H. Loew. 2019. Discriminant analysis deep neural networks. In Proceedings of the 53rd Annual Conference on Information Sciences and Systems. IEEE, 1--6.

[28]

X. Li, M. Chen, F. Nie, and Q. Wang. 2017. Locality adaptive discriminant analysis. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2201--2207.

[29]

X. Li, M. Chen, and Q. Wang. 2018. Self-tuned discrimination-aware method for unsupervised feature selection. IEEE Transactions on Neural Networks and Learning Systems 30, 8 (2018), 2275--2284.

[30]

X. Li, M. Chen, and Q. Wang. 2020a. Discrimination-aware projected matrix factorization. IEEE Transactions on Knowledge and Data Engineering 32, 4 (2020), 809--814.

[31]

X. Li, W. Hu, H. Wang, and Z. Zhang. 2010. Linear discriminant analysis using rotational invariant L1 norm. Neurocomputing 73, 13-15 (2010), 2571--2579.

Digital Library

[32]

Z. Lin, M. Chen, and Y. Ma. 2010. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055 (2010).

[33]

T. Luo, C. Hou, F. Nie, and D. Yi. 2018. Dimension reduction for non-Gaussian data by adaptive discriminative analysis. IEEE Transactions on Cybernetics 49, 3 (2018), 933--946.

[34]

P. P. Markopoulos, S. Zlotnikov, and F. Ahmad. 2019. Adaptive radar-based human activity recognition with L1-norm linear discriminant analysis. IEEE Journal of Electromagnetics, RF and Microwaves in Medicine and Biology 3, 2 (2019), 120--126.

[35]

S. A. Nene, S. K. Nayar, and H. Murase. 1996. object image library (coil-100). Technical Report, Columbia University (1996).

[36]

F. Nie, Z. Wang andR. Wang, Z. Wang, and X. Li. 2020. Adaptive local linear discriminant analysis. ACM Transactions on Knowledge Discovery from Data 14, 1 (2020), 1--19.

[37]

Feiping Nie, Shiming Xiang, and Changshui Zhang. 2007. Neighborhood MinMax projections. In Proceedings of the 20th International Joint Conference on Artifical Intelligence. 993--998.

Digital Library

[38]

V. M. Patel, N. H. Van, and R. Vidal. 2013. Latent space sparse subspace clustering. In Proceedings of theIEEE International Conference on Computer Vision. 225--232.

[39]

X. Qiu and L. Wu. 2005. Stepwise nearest neighbor discriminant analysis. In Proceedings of the International Joint Conference on Artifical Intelligence. 829--834.

[40]

S. T. Roweis and L. K. Saul. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 5500 (2000), 2323--2326.

[41]

F. S. Samaria and A. C. Harter. 1994. Parameterisation of a stochastic model for human face identification. In Proceedings of 1994 IEEE Workshop on Applications of Computer Vision. 138--142.

[42]

E. E. Schadt, M. D. Linderman, J. Sorenson, L. Lee, and G. P. Nolan. 2010. Computational solutions to large-scale data management and analysis. Nature Reviews Genetics 11, 9 (2010), 647.

[43]

T. Sim, S. Baker, and M. Bsat. 2001. The CMU pose, illumination and expression database of human faces. Carnegie Mellon University, Technical Report, CMU-RI-TR-OI-02 (2001).

[44]

M. Sugiyama. 2006. Local fisher discriminant analysis for supervised dimensionality reduction. In Proceedings of the 23rd International Conference on Machine Learning. 905--912.

Digital Library

[45]

B. Wang, Y. Hu, J. Gao, Y. Sun, H. Chen, and B. Yin. 2017. Locality preserving projections for grassmann manifold. In Proceedings of the International Joint Conference on Artifical Intelligence. 2893--2900.

[46]

S. Wold, K. Esbensen, and P. Geladi. 1987. Principal component analysis. Chemometrics and Intelligent Laboratory Systems 2, 1–3 (1987), 37--52.

[47]

J. Wright, A. Ganesh, S. Rao, Y. Peng, and Y. Ma. 2009. Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In Proceedings of theAdvances in Neural Information Processing Systems. 2080--2088.

[48]

S. Xiang, F. Nie, G. Meng, C. Pan, and C. Zhang. 2012. Discriminative least squares regression for multiclass classification and feature selection. IEEE Transactions on Neural Networks and Learning Systems 23, 11 (2012), 1738--1754.

[49]

H. Xu, C. Caramanis, and S. Mannor. 2012. Outlier-robust PCA: The high-dimensional case. IEEE Transactions on Information Theory 59, 1 (2012), 546--572.

Digital Library

[50]

L. Yan, H. Lu, C. Wang, Z. Ye, H. Chen, and H. Ling. 2019. Deep linear discriminant analysis hashing for image retrieval. Multimedia Tools and Applications 78, 11 (2019), 15101--15119.

Digital Library

[51]

J. Yang, D. Zhang, X. Yong, and J. Yang. 2005. Two-dimensional discriminant transform for face recognition. Pattern Recognition 38, 7 (2005), 1125--1129.

Digital Library

[52]

J. Ye, R. Janardan, Q. Li, and H. Park. 2006. Feature reduction via generalized uncorrelated linear discriminant analysis. IEEE Transactions on Knowledge and Data Engineering 18, 10 (2006), 1312--1322.

Digital Library

[53]

J. Ye and T. Xiong. 2006. Null space versus orthogonal linear discriminant analysis. In Proceedings of the 23rd International Conference on Machine Learning. 1073--1080.

[54]

K. Yu, X. Wu, W. Ding, and J. Pei. 2016. Scalable and accurate online feature selection for big data. ACM Transactions on Knowledge Discovery from Data 11, 2 (2016), 1--39.

[55]

C. Zhang, Q. Hu, H. Fu, P. Zhu, and X. Cao. 2017. Latent multi-view subspace clustering. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 4279--4287.

[56]

H. Zhang, J. Yang, F. Shang, C. Gong, and Z. Zhang. 2018. LRR for subspace segmentation via tractable schatten- norm minimization and factorization. IEEE Transactions on Cybernetics 49, 5 (2018), 1722--1734.

[57]

T. Zhang, D. Tao, and J. Yang. 2008. Discriminative locality alignment. In Proceedings of the 10th European Conference on Computer Vision: Part I. 725--738.

[58]

X. Zhao, J. Guo, F. Nie, L. Chen, Z. Li, and H. Zhang. 2020. Joint principal component and discriminant analysis for dimensionality reduction. IEEE Transactions on Neural Networks and Learning Systems 31, 2 (2020), 433--444.

[59]

L. Zhou, B. Xiao, X. Liu, J. Zhou, and E. R. Hancock. 2019. Latent distribution preserving deep subspace clustering. In Proceedings of the International Joint Conference on Artifical Intelligence. 4440--4446.

[60]

Y. Zhou and S. Sun. 2016. Manifold partition discriminant analysis. IEEE Transactions on Cybernetics 47, 4 (2016), 830--840.

[61]

Y. Zhu, C. Zhu, and X. Li. 2018. Improved principal component analysis and linear regression classification for face recognition. Signal Processing 145 (2018), 175--182.

Cited By

Wang ZNie FWang HHuang HWang F(2024)Toward Robust Discriminative Projections Learning Against Adversarial Patch AttacksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.332160635:12(18784-18798)Online publication date: Dec-2024
https://doi.org/10.1109/TNNLS.2023.3321606
Liu KXiao XYou JZhou Y(2024)Robust Discriminative t-Linear Subspace Learning for Image Feature ExtractionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337599734:8(7315-7327)Online publication date: Aug-2024
https://doi.org/10.1109/TCSVT.2024.3375997
Song TDai HWang SWang GZhang XZhang YJiao L(2022)TransCluster: A Cell-Type Identification Method for single-cell RNA-Seq data using deep learning based on transformerFrontiers in Genetics10.3389/fgene.2022.103891913Online publication date: 11-Oct-2022
https://doi.org/10.3389/fgene.2022.1038919
Show More Cited By

Index Terms

Robust Adaptive Linear Discriminant Analysis with Bidirectional Reconstruction Constraint
1. Computing methodologies
  1. Machine learning
    1. Machine learning algorithms
      1. Feature selection

Recommendations

Graph regularized linear discriminant analysis and its generalization

Linear discriminant analysis (LDA) is a powerful dimensionality reduction technique, which has been widely used in many applications. Although, LDA is well-known for its discriminant capability, it clearly does not capture the geometric structure of the ...
Multi-class Fukunaga Koontz discriminant analysis for enhanced face recognition

Linear subspace learning methods such as Fisher's Linear Discriminant Analysis (LDA), Unsupervised Discriminant Projection (UDP), and Locality Preserving Projections (LPP) have been widely used in face recognition applications as a tool to capture low ...
Separable linear discriminant analysis

Linear discriminant analysis (LDA) is a popular technique for supervised dimension reduction. Due to the curse of dimensionality usually suffered by LDA when applied to 2D data, several two-dimensional LDA (2DLDA) methods have been proposed in recent ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 14, Issue 6

December 2020

376 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3427188

Editors:
Charu Aggarwal
IBM T. J. Watson Research, USA
,
Xindong Wu
Minginglamp Academy of Sciences, China

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 September 2020

Accepted: 01 June 2020

Revised: 01 April 2020

Received: 01 November 2019

Published in TKDD Volume 14, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Natural Science Foundation of China
Beijing Outstanding Young Scientists Projects
Beijing Talents Project

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
267
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang ZNie FWang HHuang HWang F(2024)Toward Robust Discriminative Projections Learning Against Adversarial Patch AttacksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.332160635:12(18784-18798)Online publication date: Dec-2024
https://doi.org/10.1109/TNNLS.2023.3321606
Liu KXiao XYou JZhou Y(2024)Robust Discriminative t-Linear Subspace Learning for Image Feature ExtractionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337599734:8(7315-7327)Online publication date: Aug-2024
https://doi.org/10.1109/TCSVT.2024.3375997
Song TDai HWang SWang GZhang XZhang YJiao L(2022)TransCluster: A Cell-Type Identification Method for single-cell RNA-Seq data using deep learning based on transformerFrontiers in Genetics10.3389/fgene.2022.103891913Online publication date: 11-Oct-2022
https://doi.org/10.3389/fgene.2022.1038919
Wang JLiu ZZhang KWu QZhang M(2022)Robust sparse manifold discriminant analysisMultimedia Tools and Applications10.1007/s11042-022-12708-381:15(20781-20796)Online publication date: 12-Mar-2022
https://doi.org/10.1007/s11042-022-12708-3
Li CQi YZhao DGuo TBai L(2022)F $F$‐norm two‐dimensional linear discriminant analysis and its application on face recognitionInternational Journal of Intelligent Systems10.1002/int.2294137:11(8327-8347)Online publication date: 9-Jun-2022
https://doi.org/10.1002/int.22941
Yin SSun YGao JHu YWang BYin B(2021)Robust Image Representation via Low Rank Locality Preserving ProjectionACM Transactions on Knowledge Discovery from Data10.1145/343476815:4(1-22)Online publication date: Jun-2021
https://doi.org/10.1145/3434768

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents