Abstract
Barker and Rayens (J Chemometrics 17:166–173, 2003) offered convincing arguments that partial least squares (PLS) is to be preferred over principal components analysis (PCA) when discrimination is the goal and dimension reduction is required, since at least with PLS as the dimension reduction tool, information involving group separation is directly involved in the structure extraction. In this paper the basic results in Barker and Rayens (J Chemometrics 17:166–173, 2003) are reviewed and some of their ideas and comparisons are illustrated on a real data set, something which Barker and Rayens did not do. More importantly, new results are introduced, including a formal proof for the superiority of PLS over PCA in the two-group case, as well as new connections between PLS for discrimination and an extended class of PLS-like techniques known as “oriented PLS” (OrPLS). In the latter case, a particularly simple subclass of OrPLS procedures, when used to achieve the dimension reduction, is shown to always produce a lower misclassification rate than when “ordinary” PLS is used for the same purpose.
Similar content being viewed by others
References
Barker M, Rayens WS (2003) A partial least squares paradigm for discrimination. J Chemometrics 17:166–173
Bartlett MS (1938) Further aspects of the theory of multiple regression. Proc Camb Philol Soc 34:33–40
de Jong S (1993) PLS fits closer than PCR. J Chemometrics 7: 551–557
Diamantaras KI, Kung SY (1996) Principal component neural networks. Wiley, New York
Frank I, Friedman J (1993) Statistical view of chemometric regression tools. Technometrics 35:109–135
Friedman J (1989) Regularized discriminant analysis. J Am Stat Assoc 84(405):165–175
Greene T, Rayens WS (1989) Partially pooled covariance matrix estimation in discriminant analysis. Commun Stat 18(10):3679–3702
Hastie TJ, Buja A, Tibshirani R (1995) Penalized discriminant analysis. Ann Stat 23(1):73–102
Lavine BK, Davidson CE, Rayens WS (2004) Machine learning based pattern recognition applied to microarray data. Comb Chem High Throughput Screen 7:115–131
Rayens WS (1990) A role for covariance stabilization in the construction of the classical mixture surface. J Chemometrics 4(2):159–170
Rayens WS, Andersen A (2003) Oriented partial least squares. Riv Stat Appl-Ital J Appl Stat, RCE Edizioni. Napoli 15(3):367–388
Rayens WS, Greene T (1991) Covariance pooling and stabilization for classification. Comput Stat Data Anal 11: 17–42
Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, Cambridge
Wold H (1966) Estimation of principal components and related models by iterative least squares. In: Multivariate analysis. Academic, New York
Wold H (1981) Soft modeling: The basic design and some extensions. In: Systems under indirect observation, causality-structure-prediction. North Holland, Amsterdam
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, Y., Rayens, W. PLS and dimension reduction for classification. Computational Statistics 22, 189–208 (2007). https://doi.org/10.1007/s00180-007-0039-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00180-007-0039-y