McDiarmid’s Inequality

Synonyms

Bounded differences inequality

Definition

McDiarmid’s inequality shows how the values of a bounded function of independent random variables concentrate about its mean. Specifically, suppose $f : \mathcal{X}^{n} \rightarrow R$ satisfies the bounded differences property. That is, for all $i = 1,\ldots,n$ there is a $c_{i} \geq 0$ such that for all $x_{1},\ldots,x_{n},x' \in \mathcal{X}$

$$\displaystyle\begin{array}{rcl} & &\vert f(x_{1},\ldots,x_{n}) {}\\ & & -f(x_{1},\ldots,x_{i-1},x',x_{i+1},\ldots,x_{n})\vert \leq c_{i}. {}\\ \end{array}$$

If $\mathbf{X} = (X_{1},\ldots,X_{n}) \in \mathcal{X}^{n}$ is a random variable drawn according to Pⁿ and $\mu = E_{P^{n}}[f[\mathbf{X}]$ then for all $\epsilon > 0$

$$\displaystyle{P^{n}(f(\mathbf{X})-\mu \geq \epsilon ) \leq \exp \left ( \frac{2\epsilon ^{2}} {\sum _{i=1}^{n}c_{i}^{2}}\right ).}$$

McDiarmid’s is a generalization of Hoeffding’s inequality, which can be obtained by assuming $\mathcal{X} = [a,b]$and...