Bias-Variance Trade-Offs: Novel Applications

Definition

Consider a given random variable $\underline{F}$ and a random variable that we can modify, $\hat{\underline{F}}$. We wish to use a sample of $\hat{\underline{F}}$ as an estimate of a sample of F. The mean squared error (MSE) between such a pair of samples is a sum of four terms. The first term reflects the statistical coupling between F and $\hat{\underline{F}}$ and is conventionally ignored in bias-variance analysis. The second term reflects the inherent noise in F and is independent of the estimator $\hat{\underline{F}}$. Accordingly, we cannot affect this term. In contrast, the third and fourth terms depend on $\hat{\underline{F}}$. The third term, called the bias, is independent of the precise samples of both F and $\hat{\underline{F}}$ and reflects the difference between the means of F and $\hat{\underline{F}}$. The fourth term, called the variance, is independent of the precise sample of Fand reflects the inherent noise in the estimator as one samples...