Skip to main content
SpringerLink
Log in

Optimal linear projections for enhancing desired data statistics

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

Problems involving high-dimensional data, such as pattern recognition, image analysis, and gene clustering, often require a preliminary step of dimension reduction before or during statistical analysis. If one restricts to a linear technique for dimension reduction, the remaining issue is the choice of the projection. This choice can be dictated by desire to maximize certain statistical criteria, including variance, kurtosis, sparseness, and entropy, of the projected data. Motivations for such criteria comes from past empirical studies of statistics of natural and urban images. We present a geometric framework for finding projections that are optimal for obtaining certain desired statistical properties. Our approach is to define an objective function on spaces of orthogonal linear projections—Stiefel and Grassmann manifolds, and to use gradient techniques to optimize that function. This construction uses the geometries of these manifolds to perform the optimization. Experimental results are presented to demonstrate these ideas for natural and facial images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bell, A.J., Sejnowski, T.J.: An information maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129–1159 (1995)

    Article  Google Scholar 

  • Comon, P.: Independent component analysis, a new concept? Signal Process. Special issue on higher-order statistics 36(3), 287–314 (1994)

    Article  MATH  Google Scholar 

  • Cook, D.: Testing predictor contributions in sufficient dimension reduction. Ann. Stat. 32(3), 1062–1092 (2004)

    Article  MATH  Google Scholar 

  • Cook, D., Li, B.: Dimension reduction for conditional mean in regression. Ann. Stat. 30(2), 455–474 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  • Donoho, D.L., Flesia, A.G.: Can recent innovations in harmonic analysis “explain” key findings in natural image statistics? Netw. Comput. Neural Syst. 12(3), 371–393 (2001)

    MATH  Google Scholar 

  • Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  • Field, D.J.: What is the goal of sensory coding? Neural Comput. 6(4), 559–601 (1994)

    Article  MathSciNet  Google Scholar 

  • Fiori, S.: A minor subspace algorithm based on neural Stiefel dynamics. Int. J. Neural Syst. 19(5), 339–350 (2002a)

    Article  Google Scholar 

  • Fiori, S.: A theory for learning based on rigid bodies dynamics. IEEE Trans. Neural Netw. 13(3), 521–531 (2002b)

    Article  Google Scholar 

  • Geman, S., Hwang, C.-R.: Diffusions for global optimization. SIAM J. Control Optim. 24(5), 1031–1043 (1987)

    Article  MathSciNet  Google Scholar 

  • Golub, G.H., Van Loan, C.: Matrix Computations. The John Hopkins University Press, Baltimore (1989)

    MATH  Google Scholar 

  • Hyvärinen, A.: Fast and robust fixed-point algorithm for independent component analysis. IEEE Trans. Neural Netw. 10(3), 626–634 (1999)

    Article  Google Scholar 

  • Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. Wiley, New York (2001)

    Book  Google Scholar 

  • Johnson, R.A., Wichern, D.W.: Applied Multivariate Statistical Analysis. Prentice Hall, New York (2001)

    Google Scholar 

  • Liu, X., Srivastava, A., Gallivan, K.A.: Optimal linear representations of images for object recognition. In: Proceedings of 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 229–234 (2003)

  • Mallat, S.G.: Theory for multiresolution signal decomposition: The wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989)

    Article  MATH  Google Scholar 

  • Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996a)

    Article  Google Scholar 

  • Olshausen, B.A., Field, D.J.: Natural image statistics and efficient coding. Netw. Comput. Neural Syst. 7, 333–339 (1996b)

    Article  Google Scholar 

  • Robert, C.P., Casella, G.: Monte Carlo Statistical Methods. Springer Texts in Statistics. Springer, New York (1999)

    MATH  Google Scholar 

  • Srivastava, A.: A Bayesian approach to geometric subspace estimation. IEEE Trans. Signal Process. 48(5), 1390–1400 (2000)

    Article  MathSciNet  Google Scholar 

  • Srivastava, A., Lee, A.B., Simoncelli, E.P., Zhu, S.-C.: On advances in statistical modeling of natural images. J. Math. Imaging Vis. 18, 17–33 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  • Srivastava, A., Liu, X.: Tools for application-driven linear dimension reduction. J. Neurocomput. 67, 136–160 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Vladivostok State University of Economics and Service, Vladivostok, 690600, Russia

    Evgenia Rubinshtein

  2. Department of Statistics, Florida State University, Tallahassee, FL, 32306, USA

    Anuj Srivastava

Authors
  1. Evgenia Rubinshtein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anuj Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Evgenia Rubinshtein.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rubinshtein, E., Srivastava, A. Optimal linear projections for enhancing desired data statistics. Stat Comput 20, 267–282 (2010). https://doi.org/10.1007/s11222-009-9120-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-009-9120-4

Keywords

Search

Navigation