Skip to main content
Log in

PLS and dimension reduction for classification

  • Original Paper
  • Published:
Computational Statistics Aims and scope Submit manuscript

Abstract

Barker and Rayens (J Chemometrics 17:166–173, 2003) offered convincing arguments that partial least squares (PLS) is to be preferred over principal components analysis (PCA) when discrimination is the goal and dimension reduction is required, since at least with PLS as the dimension reduction tool, information involving group separation is directly involved in the structure extraction. In this paper the basic results in Barker and Rayens (J Chemometrics 17:166–173, 2003) are reviewed and some of their ideas and comparisons are illustrated on a real data set, something which Barker and Rayens did not do. More importantly, new results are introduced, including a formal proof for the superiority of PLS over PCA in the two-group case, as well as new connections between PLS for discrimination and an extended class of PLS-like techniques known as “oriented PLS” (OrPLS). In the latter case, a particularly simple subclass of OrPLS procedures, when used to achieve the dimension reduction, is shown to always produce a lower misclassification rate than when “ordinary” PLS is used for the same purpose.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Barker M, Rayens WS (2003) A partial least squares paradigm for discrimination. J Chemometrics 17:166–173

    Article  Google Scholar 

  • Bartlett MS (1938) Further aspects of the theory of multiple regression. Proc Camb Philol Soc 34:33–40

    Article  MATH  Google Scholar 

  • de Jong S (1993) PLS fits closer than PCR. J Chemometrics 7: 551–557

    Article  Google Scholar 

  • Diamantaras KI, Kung SY (1996) Principal component neural networks. Wiley, New York

    MATH  Google Scholar 

  • Frank I, Friedman J (1993) Statistical view of chemometric regression tools. Technometrics 35:109–135

    Article  MATH  Google Scholar 

  • Friedman J (1989) Regularized discriminant analysis. J Am Stat Assoc 84(405):165–175

    Article  Google Scholar 

  • Greene T, Rayens WS (1989) Partially pooled covariance matrix estimation in discriminant analysis. Commun Stat 18(10):3679–3702

    Article  MATH  MathSciNet  Google Scholar 

  • Hastie TJ, Buja A, Tibshirani R (1995) Penalized discriminant analysis. Ann Stat 23(1):73–102

    MATH  MathSciNet  Google Scholar 

  • Lavine BK, Davidson CE, Rayens WS (2004) Machine learning based pattern recognition applied to microarray data. Comb Chem High Throughput Screen 7:115–131

    Google Scholar 

  • Rayens WS (1990) A role for covariance stabilization in the construction of the classical mixture surface. J Chemometrics 4(2):159–170

    Article  Google Scholar 

  • Rayens WS, Andersen A (2003) Oriented partial least squares. Riv Stat Appl-Ital J Appl Stat, RCE Edizioni. Napoli 15(3):367–388

    Google Scholar 

  • Rayens WS, Greene T (1991) Covariance pooling and stabilization for classification. Comput Stat Data Anal 11: 17–42

    Article  MathSciNet  Google Scholar 

  • Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Wold H (1966) Estimation of principal components and related models by iterative least squares. In: Multivariate analysis. Academic, New York

  • Wold H (1981) Soft modeling: The basic design and some extensions. In: Systems under indirect observation, causality-structure-prediction. North Holland, Amsterdam

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to William Rayens.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, Y., Rayens, W. PLS and dimension reduction for classification. Computational Statistics 22, 189–208 (2007). https://doi.org/10.1007/s00180-007-0039-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00180-007-0039-y

Keywords

Navigation