Skip to main content
SpringerLink
Log in

A Simple Bayesian Algorithm for Feature Ranking in High Dimensional Regression Problems

  • Conference paper
AI 2011: Advances in Artificial Intelligence (AI 2011)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7106))

Included in the following conference series:

Abstract

Variable selection or feature ranking is a problem of fundamental importance in modern scientific research where data sets comprising hundreds of thousands of potential predictor features and only a few hundred samples are not uncommon. This paper introduces a novel Bayesian algorithm for feature ranking (BFR) which does not require any user specified parameters. The BFR algorithm is very general and can be applied to both parametric regression and classification problems. An empirical comparison of BFR against random forests and marginal covariate screening demonstrates promising performance in both real and artificial experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Breiman, L.: Better subset regression using the nonnegative garrote. Technometrics 37, 373–384 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  2. Tibshirani, R.: Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society (Series B) 58(1), 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  3. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society (Series B) 67(2), 301–320 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  4. Zou, H.: The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101(476), 1418–1429 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  5. James, G.M., Radchenko, P.: A generalized Dantzig selector with shrinkage tuning. Biometrika 96(2), 323–337 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  6. Fan, J., Samworth, R., Wu, Y.: Ultrahigh dimensional feature selection: Beyond the linear model. Journal of Machine Learning Research 10, 2013–2038 (2009)

    MathSciNet  MATH  Google Scholar 

  7. Hall, P., Miller, H.: Using generalized correlation to effect variable selection in very high dimensional problems. Journal of Computational and Graphical Statistics 18(3), 533–550 (2009)

    Article  MathSciNet  Google Scholar 

  8. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. The Annals of Statistics 32(2), 407–451 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  9. Friedman, J., Hastie, T., Höfling, H., Tibshirani, R.: Pathwise coordinate optimization. The Annals of Applied Statistics 1(2), 302–332 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  10. Zou, H., Hastie, T., Tibshirani, R.: On the “degrees of freedom” of the lasso. The Annals of Statistics 35(5), 2173–2192 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  11. Leng, C., Lin, Y., Wahba, G.: A note on the lasso and related procedures in model selection. Statistica Sinica 16(4), 1273–1284 (2006)

    MathSciNet  MATH  Google Scholar 

  12. Park, T., Casella, G.: The Bayesian lasso. Journal of the American Statistical Association 103(482), 681–686 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  13. Kyung, M., Gill, J., Ghosh, M., Casella, G.: Penalized regression, standard errors, and Bayesian lassos. Bayesian Analysis 5(2), 369–412 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for MEGA Epidemiology, The University of Melbourne, Carlton, VIC, 3053, Australia

    Enes Makalic & Daniel F. Schmidt

Authors
  1. Enes Makalic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel F. Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Engineering and Mathematical Sciences, La Trobe University, 3086, Melbourne, VIC, Australia

    Dianhui Wang

  2. School of Computer Science and Software Engineering, The University of Western Australia, 6009, Perth, WA, Australia

    Mark Reynolds

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Makalic, E., Schmidt, D.F. (2011). A Simple Bayesian Algorithm for Feature Ranking in High Dimensional Regression Problems. In: Wang, D., Reynolds, M. (eds) AI 2011: Advances in Artificial Intelligence. AI 2011. Lecture Notes in Computer Science(), vol 7106. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25832-9_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-25832-9_23

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-25831-2

  • Online ISBN: 978-3-642-25832-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation