Rough Sets in the Interpretation of Statistical Tests Outcomes for Genes Under Hypothetical Balancing Selection

Abstract

Detection of natural selection at the molecular level is one of the crucial problems in contemporary population genetics. There exists a number of statistical tests designed for it, however, the interpretation of the outcomes is often obscure, because of the existence of factors like population growth, migration and recombination. In his earlier work, the author has proposed the multi-null methodology, and he applied it for four genes implicated in human familial cancer: ATM, RECQL, WRN and BLM. Because of high computational effort required for estimating critical values under nonclassical nulls, mentioned methodology is not appropriate for selection screening. In the current paper, the author presents novel, rough set based methodology, helpful in the interpretation of tests outcomes applied versus only classical nulls. This method does not require long-lasting simulations and, as it is shown in the paper, it gives reliable results.