Estimation: An Overview

Introduction

Let P be a probability distribution on some space X. A random sample from P is a collection of independent random variables X ₁, …, X _n , all with the same distribution P. We then also call X ₁, …, X _n independent copies of a population random variable X, where X has distribution P. In statistics, the probability measure P is not known, and the aim is to estimate aspects of P using the observed sample X ₁, …, X _n . Formally, an estimator, say T, is any given known function of the data, i.e., T = T (X ₁, …, X _n ).

Example 1.

Suppose the observations are real-valued, i.e., the sample space X is the real line ℝ. An estimator of the population mean μ := EX is the sample mean $\hat \mu$ := ∑ ⁿ_i=1 X _i ∕n.

A statistical model is a collection of probability measures P. If the true distribution is in the model class P, we call the model well-specified.

In many situations, it is useful to parametrize the distributions in P, i.e., to write

$$P :=\{ {P}_{\theta } : \theta \in...