Skip to main content
SpringerLink
Log in

Locality preserving difference component analysis based on the Lq norm

  • Theoretical advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

This paper develops locality preserving difference component analysis in which the intrinsic and global structure of data is exploited, and the model we propose also provides the flexibility to adapt some characteristics of data by applying the Lq norm. In order to solve the proposed model that is non-convex or non-smooth, we resort to the proximal alternating linearized optimization approach where each subproblem has good optimization properties. It is observed that the objective function in the proposed model is a semi-algebraic function. This allows us to give the convergence analysis of algorithms in terms of the Kurdyka–Lojasiewicz property. To be specific, the sequence of iterations generated by the proposed approach converges to a critical point of the objective function. The experiments on several data sets have been conducted to demonstrate the effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19:711–720

    Article  Google Scholar 

  2. Baccini A, Besse P, Falguerolles AD (1996) A L1-norm PCA and A heuristic approach. Ordinal Symbol Data Anal 1:359-368

  3. Ding C, Zhou D, He X, Zha H (2006) R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In: Proceedings of 23rd international conference on machine learning

  4. Yang J, Zhang D, Frangi A, Yang Y (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans PAMI 26(1):131–137

    Article  Google Scholar 

  5. Ye J (2005) Generalized Low rank approximations of matrices. Mach Learn 61(1–3):167–191

    Article  Google Scholar 

  6. De la Torre Fernando, Michael B (2001) Robust principal component analysis for computer vision. In: Proceedings of ICCV, pp. 362-369

  7. Liang Z, Li Y (2010) A regularization framework for robust dimensionality reduction with applications to image reconstruction and feature extraction. Pattern Recognit 43(4):1269–1281

    Article  Google Scholar 

  8. Meng D, Zhao Q, Xu Z (2012) Improve robustness of sparse PCA by L1 norm maximization. Pattern Recognit 45(1):487–497

    Article  Google Scholar 

  9. Ke Q, Kanade T (2005) Robust L1 norm factorization in the presence of outliers and missing data by alternative convex programming. In: Proceedings of IEEE conference on computer vision and pattern recognition

  10. Kwak N (2009) Principal component analysis based on L1-norm maximization. IEEE Trans Pattern Anal Mach Intell 30(9):1672–1680

    Article  Google Scholar 

  11. Nie F, Huang H, Ding C, Luo D, Wang H (2011) Robust principal component analysis with non-greedy L1-Norm maximization. In: Proceedings of 22\({nd}\) international conference on artificial intelligence, pp. 1433–1438

  12. Wang R, Nie F, Yang X, Gao F, Yao M (2015) Robust 2DPCA with non-greedy L1-nom maximization for image analysis. IEEE Trans Cybern 45(5):1108–1122

    Article  Google Scholar 

  13. Li X, Pang Y, Yuan Y (2009) L1-norm based 2DPCA. IEEE Trans Syst Man Cybern Part B 40(9):1170–1175

    Google Scholar 

  14. Pang Y, Li X, Yuan Y (2010) Robust tensor analysis with L1 norm. IEEE Trans Circuits Syst Video Technol 20(2):172–178

    Article  Google Scholar 

  15. Zhong F, Zhang J (2013) Linear discriminant analysis based on L1 norm maximization. IEEE Trans Image Process 22(8):3018–3027

    Article  MathSciNet  Google Scholar 

  16. Wang H, Lu X, Hu Z, Zheng W (2014) Fisher discriminant analysis with L1 norm. IEEE Trans Cybern 44(6):828–842

    Article  Google Scholar 

  17. Zhong F, Zhang J, Li D (2014) Discriminant locality preserving projections based on L1 norm maximization. IEEE Trans Neural Netw Learn Syst 25(11):2065–2074

    Article  Google Scholar 

  18. Li Z, Tang J, Mei T (2019) Deep collaborative embedding for social image understanding. IEEE Trans PAMI (in press)

  19. Kwak N (2015) Principal component analysis based on Lp-norm maximization. IEEE Trans Cybern 44(5):594–609

    Article  Google Scholar 

  20. Liang Z, Xia S, Zhou Y, Zhang L, Li Y (2013) Feature extraction based on Lp norm generalized principal component analysis. Pattern Recognit Lett 34(9):1037–1045

    Article  Google Scholar 

  21. Oh J, Kwak N (2013) Generalization of linear discriminant analysis using Lp norm. Pattern Recognit Lett 34(6):679–685

    Article  Google Scholar 

  22. Li Z, Tang J (2015) Unsupervised feature selection via nonnegative spectral analysis and redundancy control. IEEE Trans Image Process 24(12):5345–5355

    MathSciNet  MATH  Google Scholar 

  23. Li Z, Tang J, He X (2018) Robust structured nonnegative matrix factorization for image representation. IEEE Trans Neural Netw Learn Syst 29(5):1947–1960

    Article  MathSciNet  Google Scholar 

  24. Li Z, Liu J, Tang J, Hu H (2015) Robust structured subspace learning for data representation. IEEE Trans PAMI 37(10):2085–2098

    Article  Google Scholar 

  25. Zhao H, Wang Z, Nei F (2019) A new formulation of linear discriminant analysis for robust dimensionality reduction. In: IEEE Transactions on Knowledge and data engineering

  26. Liang Z, Xia S, Zhou Y (2013) Normalized discriminant analysis for dimensionality reduction. Neurocomputing 111:153–159

    Article  Google Scholar 

  27. He X, Yan S, Hu Y, Niyogi P, Zhang HJ (2005) Face recognition using Laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):38–340

    Google Scholar 

  28. Attouch H, Bolte J, Redont P, Soubeyran A (2010) Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Lojasiewicz inequality. Math Oper Res 35(2):438–457

    Article  MathSciNet  Google Scholar 

  29. Bolte J, Sabach S, Teboulle M (2013) Proximal alternating linearized minimization and projection methods for nonconvex and nonsmooth problems. Math Program 146:1–36

    MATH  Google Scholar 

  30. Boyd S, Vandenberghe L (2004) Convex optimization, Cambridge. Cambridge Univ. Press, Cambridge

    Book  Google Scholar 

  31. Frank M, Wolfe P (1956) An algorithm for quadratic programming. Naval Res Logist Q 3:95–110

    Article  MathSciNet  Google Scholar 

  32. Garber D, Hazan E (2015) Faster rates for the Frank–Wolfe method over strongly-convex sets. In: ICML, pp. 541–549

  33. Jaggi M (2013) Revisiting Frank–Wolfe: projection-free sparse convex optimization. ICML 28(1):427–435

    Google Scholar 

  34. Nanculef R, Frandi E, Sartori C, Allende H (2014) A novel Frank–Wolfe algorithm. analysis and applications to large-scale SVM training. Inf Sci 285:66–99

    Article  MathSciNet  Google Scholar 

  35. Bubeck S (2015) Convex optimization: algorithms and complexity. Found Trends Mach Learn 8(3–4):231–358

    Article  Google Scholar 

  36. Yang WH (1991) On generalized holder Inequality. Nonlinear Anal Theory Appl 16(5):489–498

    Article  Google Scholar 

  37. Liu J, Ye J (2010) Efficient L1/Lq Norm Regularization, arXiv:1009.4766v1

  38. Sra S (2011) Fast projections onto L1,q-norm balls for grouped feature selection. In: European conference MLKD in databases

  39. Manton JH (2002) Optimization algorithms exploiting unitary constraints. IEEE Trans Signal Process 50(3):635–650

    Article  MathSciNet  Google Scholar 

  40. Golub G, Loan CV (1996) Matrix computations. Johns Hopkins, Baltimore

    MATH  Google Scholar 

  41. Mordukhovich Boris S (2006) Variational analysis and generalized differentiation I basic theory. Springer, Berlin

    Book  Google Scholar 

  42. http://www.can-or.co.uk/Facedatbase.html

  43. Martines AM, Benavente R (1998) The AR face database. Technical report, CVC

Download references

Acknowledgements

This work is partially supported by the Fundamental Research Funds for the Central Universities (2015XKMS084).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, No1, Daxue Road, Xuzhou, 221116, Jiangsu, People’s Republic of China

    Zhizheng Liang, Lei Zhang, Jin Liu & Yong Zhou

  2. The Department of Computer Science, Wayne State University, Detroit, MI, USA

    Xuewen Chen

Authors
  1. Zhizheng Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuewen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhizheng Liang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, Z., Chen, X., Zhang, L. et al. Locality preserving difference component analysis based on the Lq norm. Pattern Anal Applic 23, 735–749 (2020). https://doi.org/10.1007/s10044-019-00834-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-019-00834-y

Keywords

Search

Navigation