Skip to main content
SpringerLink
Log in

More Powerful Control of the False Discovery Rate Under Dependence

  • Original Article
  • Published:
Statistical Methods and Applications Aims and scope Submit manuscript

Abstract

In a breakthrough paper, Benjamini and Hochberg (J Roy Stat Soc Ser B 57:289–300, 1995) proposed a new error measure for multiple testing, the FDR; and developed a distribution-free procedure to control it under independence among the test statistics. In this paper we argue by extensive simulation and theoretical considerations that the assumption of independence is not needed. Along the lines of (Ann Stat 32:1035–1061, 2004b), we moreover provide a more powerful method, that exploits an estimator of the number of false nulls among the tests. We propose a whole family of iterative estimators that prove robust under dependence and independence between the test statistics. These estimators can be used to improve also classical multiple testing procedures, and in general to estimate the weight of a known component in a mixture distribution. Innovations are illustrated by simulations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Abrahmsen P (1997) A review of gaussian random fields and correlation functions. Technical Report 917, Norwegian Computing Center, Norway

    Google Scholar 

  • Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: A practical and powerful approach to multiple testing. J Roy Stat Soc (Ser B) 57:289–300

    MATH  MathSciNet  Google Scholar 

  • Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29:1165–1188

    Article  MATH  MathSciNet  Google Scholar 

  • Benjamini Y, Krieger AM, Yekutieli D (2004) Two staged linear step up FDR controlling procedures. Technical report Department of statistics, Tel Aviv University, Tel Aviv

    Google Scholar 

  • Bickel DR (2004) On “strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates”: does a large number of tests obviate confidence intervals of the FDR?. Technical report, Office of Biostatistics and Bioinformatics Medical College of Georgia, Georgia

    Google Scholar 

  • Bovenhuis H, Spelman RJ (2000) Selective genotyping to detect quantitative trait loci for multiple traits in outbred populations. J Dairy Sci 83(1):173–180

    Article  Google Scholar 

  • Efron B, Tibshirani R, Storey JD, Tusher V (2001) Empirical Bayes analysis of a microarray experiment. J Am Stat Assoc 96:1151–1160

    Article  MATH  MathSciNet  Google Scholar 

  • Ellis SP, Underwood MD, Arango V (2000) Mixed models and multiple comparisons in analysis of human neurochemical maps. Psychiat Res-Neuroim 9:111–119

    Article  Google Scholar 

  • Esary JD, Proschan F, Walkup DW (1967) Association of random variables, with applications. Ann Math Stat 38:1466–1474

    Article  MathSciNet  MATH  Google Scholar 

  • Farcomeni A (2004) Multiple testing procedures under dependence, with applications. Ph.D. Thesis, Università degli studi di Roma “La Sapienza”; Dipartimento di Statistica Probabilità e Statistiche Applicate.

  • Genovese CR, Wasserman L (2002) Operating characteristics and extensions of the FDR procedure. J Roy Stat Soc (Ser B) 64:499–518

    Article  MATH  MathSciNet  Google Scholar 

  • Genovese CR, Wasserman L (2004a) Exceedance control of the false discovery proportion. Technical report, Department of statistics, Carnegie Mellon University, Pittsburgh

    Google Scholar 

  • Genovese CR, Wasserman L (2004b) A stochastic process approach to false discovery control. Ann Stat 32:1035–1061

    Article  MATH  MathSciNet  Google Scholar 

  • Hochberg Y, Tamhane AC (1987) Multiple comparisons procedures. Wiley, New York

    Google Scholar 

  • Ip EH (2001) Testing for local dependency in dichotomous and polytomous item response models. Psychometrika 66(1):109–132

    Article  MathSciNet  Google Scholar 

  • Johnson ME (1987) Multivariate statistical simulation. Wiley, New York

    MATH  Google Scholar 

  • van der Laan MJ, Dudoit S, Pollard KS (2003) Multiple testing Part III. Procedures for control of the generalized family-wise error rate and proportion of false positives. Technical Report 141, Division of Biostatistics, UC Berkeley

  • Merriam EP, Genovese CR, Colby CL (2003) Spatial updating in human parietal cortex. Neuron 39:361–373

    Article  Google Scholar 

  • Mosig MO, Lipkin E, Khutoreskaya G, Tchourzyna E, Soller M, Fridmann AA (2001) Whole genome scan for quantitative trait loci affecting milk protein percentage in Israeli-Holstein cattle, by means of selective milk DNA pooling in a daughter design, using an adjusted false discovery rate criterion. Genetics 157:1683–1698

    Google Scholar 

  • Pollard KS, van der Laan MJ (2003) Resampling-based multiple testing: Asymptotic control of type I error and applications to gene expression data. Technical Report 121, Division of Biostatistics, UC Berkeley

    Google Scholar 

  • Reiner A, Yekutieli D, Benjamini Y (2003) Identifying differentially expressed genes using false discovery rate controlling procedures. Bioinformatics 19:368–375

    Article  Google Scholar 

  • Sarkar SK (2002) Some results on false discovery rate in stepwise multiple testing procedures. Ann Stat 30:239–257

    Article  MATH  Google Scholar 

  • Simes RJ (1986) An improved Bonferroni procedure for multiple tests of significance. Biometrika 73:751–754

    Article  MATH  MathSciNet  Google Scholar 

  • Smith R (2001) Environmental statistics. (2001) Environmental statistics. http://www.math.uio.no/˜glad/envnotes.ps

  • Storey JD (2002) A direct approach to false discovery rates. J Roy Stat Soc(Ser B) 64:479–498

    Article  MATH  MathSciNet  Google Scholar 

  • Storey JD (2003) The positive false discovery rate: A Bayesian interpretation and the q-value. Ann Stat 31:2013–2035

    Article  MATH  MathSciNet  Google Scholar 

  • Storey JD, Tibshirani R (2001) Estimating false discovery rates under dependence, with applications to DNA microarrays. Technical Report 2001-28, Department of Statistics, Stanford University, Stanford

    Google Scholar 

  • Storey JD, Taylor JE, Siegmund D (2004) Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach. J Roy Stat Soc(Ser B) 66:187–205

    Article  MATH  MathSciNet  Google Scholar 

  • Swanepoel JWH (1999) The limiting behavior of a modified maximal symmetric 2s−spacing with applications. Ann Stat 27:24–35

    Article  MATH  MathSciNet  Google Scholar 

  • Woodroofe M, Sun J (1999) Testing uniformity versus a monotone density. Ann Stat 27:338–360

    Article  MATH  MathSciNet  Google Scholar 

  • Worsley KJ, Marrett S, Neelin P, Vandal AC, Friston KJ, Evans AC (1996) A unified statistical approach for determining significant signals in images of cerebral activation. Hum Brain Mapp 4:58–73

    Article  Google Scholar 

  • Yekutieli D, Benjamini Y (1999) Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. J Stat Plann Inference 82:171–196

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Statistica, Probabilitá e Statistiche Applicate, Piazzale Aldo Moro 5, 00185, Rome, Italy

    Alessio Farcomeni

Authors
  1. Alessio Farcomeni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessio Farcomeni.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Farcomeni, A. More Powerful Control of the False Discovery Rate Under Dependence. JISS 15, 43–73 (2006). https://doi.org/10.1007/s10260-006-0002-z

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10260-006-0002-z

Keywords

Search

Navigation