Summary
A new approach for multiplicity control (Optimal Subset) is presented. This is based on the selection of the best subset of partial (univariate) hypotheses producing the minimal p-value. In this work, we show how to perform this new procedure in the permutation framework, choosing suitable combining functions and permutation strategies. The optimal subset approach can be very useful in exploratory studies because it performs a weak control for multiplicity which can be a valid alternative to the False Discovery Rate (FDR). A comparative simulation study and an application to neuroimaging real data shows that it is particularly useful in presence of a high number of hypotheses. We also show how stepwise regression may be a special case of Optimal Subset procedures and how to adjust the p-value of the selected model taking into account for the multiplicity arising from the possible different models selected by a stepwise regression.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Benjamini, Y., Hochberg, Y. (1995). Controlling the false discovery rate: a new and powerful approach to multiple testing.Journal of the Royal Statistical Society, Series B, 57, 1289–1300.
Brook, R. J., Arnold, G. C. (1985).Applied regression analysis and experimental design. Marcel Dekker, New York.
Copas, J. B. (1983). Regression, prediction and shrinkage (with discussion).Journal of the Royal Statistical Society, Series B, 45: 311–354.
Finos, L., Pesarin, F., Salmaso, L. (2003). Combined tests for controlling multiplicity by closed testing procedures.Italian Journal of Applied Statistics, 15, 301–329.
Fisher, R.A. (1955).Statistical methods and scientific inference. Edinburgh: Oliver &: Boyd.
Freedman, L.S., Pee, D. and Midthune, D.N. (1992). The problem of underestimating the residual error variance in forward stepwise regression.The Statistician, 41, 405–412.
Lubar, J. F., Congedo, M., Askew, J. H., 2003. Low-resolution electromagnetic tomography (LORETA) of cerebral activity in chronic depressive disorder.International Journal of Psychophysiology, 49(3), 175–185.
Holm S. (1979). A simple sequentially rejective multiple test procedureScandinavian Journal of Statistics, 6, 65–70.
Marcus, R., Peritz, E. and Gabriel, K.R. (1976). On closed testing procedures with special reference to ordered analysis of variance.Biometrika, 63, 655–660.
Pascual-Marqui, R. D., 1999. Review of Methods for Solving the EEG Inverse Problem.International Journal of Bioelectromagnetism, 1 (1), 75–86.
Pesarin, F. (2001),Multivariate permutation tests with application to biostatistics. Wiley, Chichester.
Sherlin, L., Congedo, M. (2003). Obsessive Compulsive Disorder localized using Low-Resolution Electromagnetic Tomography, 34th annual meeting of the Association for Applied Psychophysiology & Biofeedback, March 27–30, Jacksonville, FL, USA.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Finos, L., Salmaso, L. A new nonparametric approach for multiplicity control:Optimal Subset procedures. Computational Statistics 20, 643–654 (2005). https://doi.org/10.1007/BF02741320
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF02741320