Uniform Random Number Generation and Secret Key Agreement for General Sources by Using Sparse Matrices

Abstract

In this paper, we investigate the problems of uniform random number generation, independent uniform random number generation, and secret key agreement, which provide the information theoretic security. We consider the strong uniformity and strong independence, where it has been unclear whether or not sparse matrices can be applied to these problems for general (correlated) sources with respect to these criteria. To prove the theorems, we first introduce the notion of the balanced-coloring property and the collision-resistance property. We next apply these properties to the problems. Since an ensemble of sparse matrices (with logarithmic column weight) over a finite field satisfies these properties, we can construct a code achieving the fundamental limits by using sparse matrices.

This paper is based on Proc. 2012 IEEE Inform. Theory Workshop, Lausanne, Switzerland, Sep. 3–7, 2012, pp. 612–616.

Appendices

Appendix 1: Proof of Lemmas 1.1 and 1.2

In the following proofs, we use the following lemma.

Lemma 6.5

([31]) Let p and q be probability distributions on the same set $\mathscr {U}$. Then

$$\begin{aligned} |H(p)-H(q)| \le d(p,q)\log (|\mathscr {U}|-1)+h(d(p,q)), \end{aligned}$$

where $H(\mu )\equiv \sum _u \mu (u)\log (1/\mu (u))$ and $h(\theta )\equiv -\theta \log \theta -[1-\theta ]\log (1-\theta )$.

First, we prove Lemma 1.1. From the fact that $H(p_{\overline{M}_n})=\log |\mathscr {M}_n|$, we have

$$\begin{aligned} D(\mu _{M_n}\Vert p_{\overline{M}_n})&=\sum _{\mathbf {m}\in \mathscr {M}_n}\mu _{M_n}(\mathbf {m})\log \frac{\mu _{M_n}(\mathbf {m})}{p_{\overline{M}_n}(\mathbf {m})} \nonumber \\&= \log |\mathscr {M}_n|-H(\mu _{M_n}) \nonumber \\&= \left| {H(p_{\overline{M}_n})-H(\mu _{M_n})}\right| \nonumber \\&\le d(\mu _{M_n},p_{\overline{M}_n})\log |\mathscr {M}_n|+ h(d(\mu _{M_n},p_{\overline{M}_n})) \nonumber \\&= o(1), \end{aligned}$$

(39)

where the inequality comes from Lemma 6.5.$\qquad \square$

Next, we prove Lemma 1.2. The inequality (5) is shown immediately from (1) and Lemma 1.1. From Lemma 6.5, we have

$$\begin{aligned}&\left| {H(p_{\overline{M}_n})-H(\mu _{M_n})}\right| \nonumber \\&\le d(\mu _{M_n},p_{\overline{M}_n})\log |\mathscr {M}_n| +h(d(\mu _{M_n},p_{\overline{M}_n})) \nonumber \\&\le d(\mu _{M_nZ^n},p_{\overline{M}_n}\times \mu _{Z^n})\log |\mathscr {M}_n| + h(d(\mu _{M_nZ^n},p_{\overline{M}_n}\times \mu _{Z^n})), \end{aligned}$$

(40)

where the second inequality comes from (1). Furthermore, from Lemma 6.5, we have

$$\begin{aligned}&\sum _{\mathbf {z}}\mu _{Z^n}(\mathbf {z})\left| {H(p_{\overline{M}_n})-H(\mu _{M_n|Z^n}(\cdot |\mathbf {z}))}\right| \nonumber \\&\le \sum _{\mathbf {z}} \mu _{Z^n}(\mathbf {z})d(p_{\overline{M}_n},\mu _{M_n|Z^n}(\cdot |\mathbf {z}))\log |\mathscr {M}_n| + \sum _{\mathbf {z}}\mu _{Z^n}(\mathbf {z}) h(d(p_{\overline{M}_n},\mu _{M_n|Z^n}(\cdot ,\mathbf {z}))) \nonumber \\&\le d(\mu _{M_nZ^n},p_{\overline{M}_n}\times \mu _{Z^n})\log |\mathscr {M}_n| + h(d(\mu _{M_nZ^n},p_{\overline{M}_n}\times \mu _{Z^n})), \end{aligned}$$

(41)

where the second inequality comes from the convexity of h. Then we have (6) as

$$\begin{aligned} I(M_n;Z^n)&=\sum _{\mathbf {z}}\mu _{Z^n}(\mathbf {z})\left[ {H(\mu _{M_n})-H(\mu _{M_n|Z^n}(\cdot |\mathbf {z}))}\right] \nonumber \\&\le \sum _{\mathbf {z}}\mu _{Z^n}(\mathbf {z})\left| {H(\mu _{M_n})-H(p_{\overline{M}_n})}\right| + \sum _{\mathbf {z}}\mu _{Z^n}(\mathbf {z})\left| {H(p_{\overline{M}_n})-H(\mu _{M_n|Z^n}(\cdot |\mathbf {z}))}\right| \nonumber \\&\le 2d(\mu _{M_nZ^n},p_{\overline{M}_n}\times \mu _{Z^n})\log |\mathscr {M}_n| + 2h(d(\mu _{M_nZ^n},p_{\overline{M}_n}\times \mu _{Z^n})) \nonumber \\&\le o(1), \end{aligned}$$

(42)

where the second inequality comes from (40) and (41).$\qquad \square$

Appendix 2: Proof of Lemma 6.1

When $\mu _{X^n|Z^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z})|\mathbf {z})\ne 0$, we have

$$\begin{aligned}&E_{\mathsf {A}}\left[ { \sum _{\mathbf {m}} \left| { \frac{\mu _{X^n|Z^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z})\cap \mathscr {C}_{\mathsf {A}}(\mathbf {m})|\mathbf {z})}{\mu _{X^n|Z^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z})|\mathbf {z})} -\frac{1}{|\mathrm {Im}\mathscr {A}_n|} }\right| }\right] \nonumber \\&\le \sqrt{ \alpha _{\mathsf {A}}-1 +\frac{[\beta _{\mathsf {A}}+1]|\mathrm {Im}\mathscr {A}_n|\max _{\mathbf {x}\in \underline{\mathscr {T}}_{X|Z}(\mathbf {z})}\mu _{X^n|Z^n}(\mathbf {x}|\mathbf {z})}{\mu _{X^n|Z^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z})|\mathbf {z})} } \nonumber \\&\le \frac{\sqrt{ \alpha _{\mathsf {A}}-1 +[\beta _{\mathsf {A}}+1]|\mathrm {Im}\mathscr {A}_n|2^{-n[\underline{H}(\mathbf {X}|\mathbf {Z})-\zeta ]} }}{\mu _{X^n|Z^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z})|\mathbf {z})}, \end{aligned}$$

(43)

where the first inequality comes from Lemma 2.2 and the second inequality comes from the definition of $\underline{\mathscr {T}}_{X|Z}(\mathbf {z})$ and the fact that $\mu _{X^n|Z^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z})|\mathbf {z})\le 1$. Then we have

$$\begin{aligned}&E_{\mathsf {A}}\left[ { \sum _{\mathbf {m},\mathbf {z}} \left| { \mu _{X^nZ^n}(\mathscr {C}_A(\mathbf {m}),\mathbf {z}) - \frac{\mu _{Z^n}(\mathbf {z})}{|\mathrm {Im}\mathscr {A}_n|} }\right| }\right] \nonumber \\&\le E_{\mathsf {A}}\left[ { \sum _{\mathbf {m},\mathbf {z}} \left| { \mu _{X^nZ^n}(\mathscr {C}_{\mathsf {A}}(\mathbf {m})\cap \underline{\mathscr {T}}_{X|Z}(\mathbf {z}),\mathbf {z}) - \frac{\mu _{X^nZ^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z}),\mathbf {z})}{|\mathrm {Im}\mathscr {A}_n|} }\right| }\right] \nonumber \\&\quad + E_{\mathsf {A}}\left[ { \sum _{\mathbf {m},\mathbf {z}} \mu _{X^nZ^n}(\mathscr {C}_{\mathsf {A}}(\mathbf {m})\cap [\underline{\mathscr {T}}_{X|Z}(\mathbf {z})]^c,\mathbf {z}) }\right] + E_{\mathsf {A}}\left[ { \sum _{\mathbf {m},\mathbf {z}} \frac{\mu _{X^nZ^n}([\underline{\mathscr {T}}_{X|Z}(\mathbf {z})]^c,\mathbf {z})}{|\mathrm {Im}\mathscr {A}_n|} }\right] \nonumber \\&= \!\!\!\!\!\!\!\!\! \sum _{\begin{array}{c} \mathbf {z}:\\ \mu _Z(\mathbf {z})\ne 0 \\ \mu _{X^n|Z^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z})|\mathbf {z})\ne 0 \end{array}} \!\!\!\!\!\!\!\!\! \mu _{X^nZ^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z}),\mathbf {z}) E_{\mathsf {A}}\left[ { \sum _{\mathbf {m}} \left| { \frac{\mu _{X^n|Z^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z})\cap \mathscr {C}_{\mathsf {A}}(\mathbf {m})|\mathbf {z})}{\mu _{X^n|Z^n}(\underline{\mathscr {T}}_{X|Z}(\mathbf {z})|\mathbf {z})} -\frac{1}{|\mathrm {Im}\mathscr {A}_n|} }\right| }\right] \nonumber \\&\quad + 2 \sum _{\mathbf {z}} \mu _{X^nZ^n}([\underline{\mathscr {T}}_{X|Z}(\mathbf {z})]^c,\mathbf {z}) \nonumber \\&\le \sqrt{ \alpha _{\mathsf {A}}-1 +[\beta _{\mathsf {A}}+1]|\mathrm {Im}\mathscr {A}_n|2^{-n[\underline{H}(\mathbf {X}|\mathbf {Z})-\zeta ]} } +2\mu _{X^nZ^n}([\underline{\mathscr {T}}_{X|Z}]^c), \end{aligned}$$

(44)

which concludes the lemma.$\qquad \square$

Appendix 3: Proof of Lemma 6.2

If $g_B(B\mathbf {x}|\mathbf {y})\ne \mathbf {x}$, there is $\mathbf {x}'\in \mathscr {C}_B(B\mathbf {x})$ such that $\mathbf {x}'\ne \mathbf {x}$ and

$$\begin{aligned} \mu _{X^n|Y^n}(\mathbf {x}'|\mathbf {y})&\ge \mu _{X^n|Y^n}(\mathbf {x}|\mathbf {y}) \nonumber \\&\ge 2^{-n[\overline{H}(\mathbf {X}|\mathbf {Y})+\zeta ]}, \end{aligned}$$

(45)

where the second inequality comes from the definition of $\overline{\mathscr {T}}_{X|Y}$. This implies that $[\overline{\mathscr {T}}_{X|Y}(\mathbf {y})\setminus \{\mathbf {x}\}]\cap \mathscr {C}_B(B\mathbf {x})\ne \emptyset$. Then we have

$$\begin{aligned} p_{\mathsf {B},n}\left( {\left\{ { B: g_B(B\mathbf {x}|\mathbf {y})\ne \mathbf {x} }\right\} }\right)&\le p_{\mathsf {B},n}\left( {\left\{ { B: \left[ {\overline{\mathscr {T}}_{X|Y}(\mathbf {y})\setminus \{\mathbf {x}\}}\right] \cap \mathscr {C}_B(B\mathbf {x})\ne \emptyset }\right\} }\right) \nonumber \\&\le \frac{2^{n[\overline{H}(\mathbf {X}|\mathbf {Y})+\zeta ]}\alpha _{\mathsf {B}}}{|\mathrm {Im}\mathscr {B}_n|} +\beta _{\mathsf {B}}, \end{aligned}$$

(46)

where the second inequality comes from Lemma 2.3 and the fact that $|\overline{\mathscr {T}}_{X|Y}(\mathbf {y})|\le 2^{n[\overline{H}(\mathbf {X}|\mathbf {Y})+\zeta ]}$. We have

$$\begin{aligned}&E_{\mathsf {B}}\left[ { \sum _{\mathbf {x},\mathbf {y}}\mu _{X^nY^n}(\mathbf {x},\mathbf {y}) \chi (g_{\mathsf {B}}(\mathsf {B}\mathbf {x}|\mathbf {y})\ne \mathbf {x}) }\right] \nonumber \\*&\le \sum _{(\mathbf {x},\mathbf {y})\in \overline{\mathscr {T}}_{X|Y}}\mu _{X^nY^n}(\mathbf {x},\mathbf {y}) p_{\mathsf {B}}\left( {\left\{ { A: g_B(B\mathbf {x}|\mathbf {y})\ne \mathbf {x} }\right\} }\right) +\sum _{(\mathbf {x},\mathbf {y})\in [\overline{\mathscr {T}}_{X|Y}]^c}\mu _{X^nY^n}(\mathbf {x},\mathbf {y}) \nonumber \\&\le \frac{2^{n[\overline{H}(\mathbf {X}|\mathbf {Y})+\zeta ]}\alpha _{\mathsf {B}}}{|\mathrm {Im}\mathscr {B}_n|} +\beta _{\mathsf {B}}+\mu _{X^nY^n}([\overline{\mathscr {T}}_{X|Y}]^c), \end{aligned}$$

(47)

which concludes the lemma.$\qquad \square$

Appendix 4: Proof of Lemma 6.4

Let $\underline{\mathscr {T}}'$ and $\underline{\mathscr {T}}'(\mathbf {z})$ be defined as:

$$\begin{aligned} \underline{\mathscr {T}}'&\equiv \left\{ { (\mathbf {x},\mathbf {z}): \frac{1}{n}\log \frac{1}{\mu _{X^n|Z^n}(\mathbf {x}|\mathbf {z})}\ge R-\zeta }\right\} \\ \underline{\mathscr {T}}'(\mathbf {z})&\equiv \left\{ { \mathbf {x}: (\mathbf {x},\mathbf {z})\in \underline{\mathscr {T}}' }\right\} . \end{aligned}$$

Since $\varphi _n(\mathbf {x})\in \varphi _n([\underline{\mathscr {T}}'(\mathbf {z})]^c)$ and $\mathbf {x}\in \varphi _n^{-1}(\varphi _n(\mathbf {x}))$ for all $\mathbf {x}\in [\underline{\mathscr {T}}'(\mathbf {z})]^c$, we have

$$\begin{aligned} \sum _{\mathbf {z}} \sum _{\mathbf {m}\in \varphi _n([\underline{\mathscr {T}}'(\mathbf {z})]^c)} \sum _{\mathbf {x}\in \varphi _n^{-1}(\mathbf {m})} \mu _{X^nZ^n}(\mathbf {x},\mathbf {z})&\ge \sum _{\mathbf {z}} \sum _{\mathbf {x}\in [\underline{\mathscr {T}}'(\mathbf {z})]^c} \mu _{X^nZ^n}(\mathbf {x},\mathbf {z}) \nonumber \\&= \mu _{X^nZ^n}(\underline{\mathscr {T}}'^c). \end{aligned}$$

(48)

On the other hand, we have

$$\begin{aligned} \sum _{\mathbf {z}} \sum _{\mathbf {m}\in \varphi _n([\underline{\mathscr {T}}'(\mathbf {z})]^c)} \frac{\mu _{Z^n}(\mathbf {z})}{|\mathscr {M}_n|}&= \sum _{\mathbf {z}} \frac{\left| {\varphi _n([\underline{\mathscr {T}}'(\mathbf {z})]^c)}\right| \mu _{Z^n}(\mathbf {z})}{|\mathscr {M}_n|} \nonumber \\&\le \sum _{\mathbf {z}} \frac{|[\underline{\mathscr {T}}'(\mathbf {z})]^c|\mu _{Z^n}(\mathbf {z})}{|\mathscr {M}_n|} \nonumber \\&\le 2^{-n\zeta }, \end{aligned}$$

(49)

where the last inequality comes from the fact that $|\mathscr {M}_n|=2^{nR}$ and $|[\underline{\mathscr {T}}'(\mathbf {z})]^c|\le 2^{n[R-\zeta ]}$. From (48) and (49), we have

$$\begin{aligned} d(\mu _{M_nZ^n},p_{\overline{M}_n}\times \mu _{Z^n})&= \frac{1}{2} \sum _{\mathbf {m},\mathbf {z}} \left| { \sum _{\mathbf {x}\in \varphi ^{-1}(\mathbf {m})} \mu _{X^nZ^n}(\mathbf {x},\mathbf {z}) -\frac{\mu _{Z^n}(\mathbf {z})}{|\mathscr {M}_n|} }\right| \nonumber \\&\ge \frac{1}{2} \sum _{\mathbf {z}} \sum _{\mathbf {m}\in \varphi _n([\underline{\mathscr {T}}'(\mathbf {z})]^c)} \left[ { \sum _{\mathbf {x}\in \varphi _n^{-1}(\mathbf {m})} \mu _{X^nZ^n}(\mathbf {x},\mathbf {z}) -\frac{\mu _{Z^n}(\mathbf {z})}{|\mathscr {M}_n|} }\right] \nonumber \\&\ge \frac{1}{2}\left[ {\mu _{X^nZ^n}(\underline{\mathscr {T}}'^c)-2^{-n\zeta }}\right] . \end{aligned}$$

(50)

Then we have the fact that

$$\begin{aligned} P\left( {\frac{1}{n}\log \frac{1}{\mu _{X^n|Z^n}(X^n|Z^n)}<R-\zeta }\right)&= \mu _{X^nZ^n}(\underline{\mathscr {T}}'^c) \nonumber \\&\le 2d(\mu _{M_nZ^n},p_{\overline{M}_n}\times \mu _{Z^n})+2^{-n\zeta } \end{aligned}$$

(51)

and

$$ \lim _{n\rightarrow \infty } P\left( {\frac{1}{n}\log \frac{1}{\mu _{X^n|Z^n}(X^n|Z^n)}<R-\zeta }\right) =0 $$

by letting $n\rightarrow \infty$. From the definition of $\underline{H}(\mathbf {X}|\mathbf {Z})$, we have

$$ R-\zeta \le \underline{H}(\mathbf {X}|\mathbf {Z}), $$

which concludes the lemma by letting $\zeta \rightarrow 0$.$\qquad \square$