An Alternative to p-Values in Hypothesis Testing with Applications in Model Selection of Stock Price Data

Abstract

In support of the American Statistical Association’s statement on p-value in 2016, see [8], we investigate, in this paper, a classical question in model selection, namely finding a “best-fit” probability distribution to a set of data. Throughout history, there have been a number of tests designed to determine whether a particular distribution fit a set of data, for instance, see [6]. The popular approach is to compute certain test statistics and base the decisions on the p values of these test statistics. As pointed out numerous times in the literature, see [5] for example, p values suffer serious drawbacks which make it untrustworthy in decision making. One typical situation is when the p value is larger than the significance level $\alpha$ which results in an inconclusive case. In many studies, a common mistake is to claim that the null hypothesis is true or most likely whereas a big p value merely implies that the null hypothesis is statistically consistent with the observed data; there is no indication that the null hypothesis is “better” than any other hypothesis in the confidence interval. We notice this situation happens a great deal in testing goodness of fit. Therefore, hereby, we propose an approach using the Akaike information criterion (AIC) or the Bayesian information criterion (BIC) to make a selection of the best fit distribution among a group of candidates. As for applications, a variety of stock price data are processed to find a fit distribution. Both the p value and the new approach are computed and compared carefully. The virtue of our approach is that there is always a justified decision made in the end; and, there will be no inconclusiveness whatsoever.