A randomized competitive group testing procedure

Abstract

In many fault detection problems, we want to identify all defective items from a set of n items using the minimum number of tests. Group testing is for the scenario where each test is on a subset of items, and tells whether the subset contains at least one defective item or not. In practice, the number d of defective items is often unknown in advance. In this paper, we propose a randomized group testing procedure RGT for the scenario where the number d of defectives is unknown in advance, and prove that RGT is competitive. By incorporating numerical results, we obtain improved upper bounds on the expected number of tests performed by RGT, for $1\le d\le 10^6$. In particular, for $1\le d\le 10^6$ and the special case where n is a power of 2, we obtain an upper bound of $d\log \frac{n}{d}+Cd+O(\log d)$ with $C\approx 2.67$ on the expected number of tests performed by RGT, which is better than the currently best upper bound in Cheng et al. (INFORMS J Comput 26(4):677–689, 2014). We conjecture that the above improved upper bounds based on numerical results from $1\le d\le 10^6$ actually hold for all $d\ge 1$.

Appendices

Appendix A: Proof of Lemma 4.3

Since $\lambda$ is a nonnegative integer random variable, and by Eq. (2),

$$\begin{aligned} E(\lambda )=\sum _{k=1}^{\infty }\Pr (\lambda \ge k)=\sum _{k=1}^{\infty } \Big [1-(1-2^{-k})^d\Big ]^2. \end{aligned}$$

(3)

Define $g(x)=\big [1-(1-2^{-x})^d\big ]^2$. Then, from the above $E(\lambda )=\sum _{k=1}^{\infty } g(k)$. We will approximate $E(\lambda )$ by integration of g(x). Since g(x) is monotonically decreasing for $x\ge 0$ and $g(0)=1$, we have $-1+\int _{0}^{\infty } g(x) \mathbf d x ~\le ~ E(\lambda ) ~\le ~ \int _{0}^{\infty } g(x) \mathbf d x$, that is

$$\begin{aligned} -1+\int _{0}^{\infty } \left[ 1-(1-2^{-x})^d\right] ^2 \mathbf d x ~\le ~ E(\lambda ) ~\le ~ \int _{0}^{\infty } \left[ 1-(1-2^{-x})^d\right] ^2 \mathbf d x. \end{aligned}$$

(4)

Let $H_d=\sum _{i=1}^{d}\frac{1}{i}$ be the d-th harmonic number, for $d\ge 1$. We will use the following result on the integration of g(x).

Claim 6.1

$$\begin{aligned} \int _{0}^{\infty } \left[ 1-(1-2^{-x})^d\right] ^2 \mathbf d\mathbf x=\frac{1}{\ln 2}(2H_{d}-H_{2d}). \end{aligned}$$

Before proving the above result, first notice that from the following bounds on $H_d$ (Young 1991; Havil 2003, pp. 73–75),

$$\begin{aligned} \ln d+\gamma +\frac{1}{2(d+1)}<H_d<\ln d+\gamma +\frac{1}{2d}, \end{aligned}$$

(5)

where $\gamma =0.577\cdots$ is the Euler–Mascheroni constant, and by Inequality (4), Claim 6.1 implies that for $d\ge 1$

$$\begin{aligned} E(\lambda )\ge & {} -1+\frac{1}{\ln 2}(2H_{d}-H_{2d})\\> & {} -1+\frac{1}{\ln 2}\left\{ 2\left( \ln d+\gamma +\frac{1}{2(d+1)}\right) -\left( \ln (2d)+\gamma +\frac{1}{2(2d)}\right) \right\} \\> & {} -1+\frac{1}{\ln 2}(\ln d+\gamma -\ln 2)\\= & {} \log d+\frac{\gamma }{\ln 2}-2, \end{aligned}$$

and

$$\begin{aligned} E(\lambda )\le & {} \frac{1}{\ln 2}(2H_{d}-H_{2d})\\< & {} \frac{1}{\ln 2}\left\{ 2\left( \ln d+\gamma +\frac{1}{2d}\right) -\left( \ln (2d)+\gamma +\frac{1}{2(2d+1)}\right) \right\} \\< & {} \frac{1}{\ln 2}\left( \ln d+\gamma -\ln 2+\frac{1}{d} \right) \\= & {} \log d+\frac{\gamma }{\ln 2}-1+\frac{1}{d\ln 2}, \end{aligned}$$

which establishes Lemma 4.3.

The rest of the proof is devoted to prove Claim 6.1.

$$\begin{aligned}&\int _{0}^{\infty } \Big [1-(1-2^{-x})^d\Big ]^2 \mathbf d x\\&\quad = \frac{1}{\ln 2} \int _{0}^{\infty } \Big [1-(1-e^{-x{\ln 2}})^d\Big ]^2 \mathbf d (x\ln 2)\\&\quad = \frac{1}{\ln 2} \int _{0}^{\infty } \Big [1-(1-e^{-y})^d\Big ]^2 \mathbf d y\\&\quad = \frac{1}{\ln 2} \int _{0}^{\infty } \Big \{1-(1-e^{-y})^{2d}-2(1-e^{-y})^{d}\big [1-(1-e^{-y})^{d}\big ]\Big \} \mathbf d y\\&\quad = \frac{1}{\ln 2} \int _{0}^{\infty } \Big [1-(1-e^{-y})^{2d}\Big ]\mathbf d y-\frac{2}{\ln 2} \int _{0}^{\infty } (1-e^{-y})^{d}\Big [1-(1-e^{-y})^{d}\Big ] \mathbf d y. \end{aligned}$$

Define $I(d)=\int _{0}^{\infty } \big [1-(1-e^{-y})^d \big ]\mathbf d y$, and $J(d)=\int _{0}^{\infty } (1-e^{-y})^{d}\big [1-(1-e^{-y})^{d}\big ] \mathbf d y$. Then,

$$\begin{aligned} \int _{0}^{\infty } \Big [1-(1-2^{-x})^d\Big ]^2 \mathbf d x=\frac{I(2d)-2J(d)}{\ln 2}. \end{aligned}$$

Next we calculate I(d) and J(d), respectively.

Observation 6.1

$I(d)=H_d$, for $d\ge 1$.

Proof

It is easy to directly calculate that $I(1)=1$. For $d>1$,

$$\begin{aligned} I(d)= & {} \int _{0}^{\infty } \Big [1-(1-e^{-y})^d \Big ]\mathbf d y\\= & {} \int _{0}^{\infty } \Big [1-(1-e^{-y})^{d-1}+(1-e^{-y})^{d-1}e^{-y}\Big ]\mathbf d y\\= & {} \int _{0}^{\infty } \Big [1-(1-e^{-y})^{d-1}\Big ]\mathbf d y+\int _{0}^{\infty } (1-e^{-y})^{d-1}e^{-y}{} \mathbf d y\\= & {} I(d-1)+\int _{0}^{\infty } (1-e^{-y})^{d-1}{} \mathbf d (1-e^{-y})\\= & {} I(d-1)+\left. \frac{1}{d}(1-e^{-y})^d\right| _{0}^{\infty }\\= & {} I(d-1)+\frac{1}{d}. \end{aligned}$$

Therefore, $I(d)=\sum _{i=1}^{d}\frac{1}{i}=H_d$. $\square$

Observation 6.2

$J(d)=H_{2d}-H_{d}$, for $d\ge 1$.

Proof

It is easy to directly calculate that $J(1)=1/2$. For $d>1$,

$$\begin{aligned} J(d)= & {} \int _{0}^{\infty } (1-e^{-y})^{d}\Big [1-(1-e^{-y})^{d}\Big ] \mathbf d y\\= & {} \int _{0}^{\infty } (1-e^{-y})(1-e^{-y})^{d-1}\Big [1-(1-e^{-y})^{d}\Big ] \mathbf d y\\= & {} \int _{0}^{\infty } (1-e^{-y})^{d-1}\Big [1-(1-e^{-y})^{d}\Big ] \mathbf d y\\&-\int _{0}^{\infty } e^{-y} (1-e^{-y})^{d-1}\Big [1-(1-e^{-y})^{d}\Big ] \mathbf d y\\= & {} \int _{0}^{\infty } (1-e^{-y})^{d-1}\Big [1-(1-e^{-y})^{d-1}+(1-e^{-y})^{d-1}e^{-y}\Big ] \mathbf d y\\&+\int _{0}^{\infty } (1-e^{-y})^{d-1}\Big [1-(1-e^{-y})^{d}\Big ] \mathbf d (e^{-y})\\= & {} J(d-1)+\int _{0}^{\infty } (1-e^{-y})^{2d-2}e^{-y}{} \mathbf d y\\&+\int _{0}^{\infty } (1-e^{-y})^{d-1}\Big [1-(1-e^{-y})^{d}\Big ] \mathbf d (e^{-y})\\= & {} J(d-1)-\int _{0}^{\infty } (1-e^{-y})^{2d-2}{} \mathbf d (e^{-y})+\int _{0}^{\infty } (1-e^{-y})^{d-1}{} \mathbf d (e^{-y})\\&-\int _{0}^{\infty } (1-e^{-y})^{2d-1}{} \mathbf d (e^{-y})\\= & {} J(d-1)+\left. \frac{(1-e^{-y})^{2d-1}}{2d-1}\right| _{0}^{\infty } -\left. \frac{(1-e^{-y})^{d}}{d}\right| _{0}^{\infty }+\left. \frac{(1-e^{-y})^{2d}}{2d}\right| _{0}^{\infty }\\= & {} J(d-1)+\frac{1}{2d-1}-\frac{1}{d}+\frac{1}{2d}\\= & {} J(d-1)+\frac{1}{2d-1}-\frac{1}{2d}. \end{aligned}$$

Therefore, $J(d)=\sum _{i=1}^{2d}\frac{(-1)^{i+1}}{i}=\sum _{i=1}^{2d}\frac{1}{i}-2\sum _{i=1}^{d}\frac{1}{2i}=H_{2d}-H_{d}$. $\Box$

From the above,

$$\begin{aligned} \int _{0}^{\infty } \Big [1-(1-2^{-x})^d\Big ]^2 \mathbf d x= & {} \frac{I(2d)-2J(d)}{\ln 2}=\frac{H_{2d}-2(H_{2d}-H_d)}{\ln 2}\\= & {} \frac{1}{\ln 2}(2H_{d}-H_{2d}), \end{aligned}$$

which establishes Claim 6.1.

Appendix B: Proof of Lemma 4.4

First, we write $E(2^{\lambda })$ as the following

$$\begin{aligned} E(2^{\lambda })= & {} \sum _{k=0}^{\infty }2^k\Pr (\lambda =k)\\= & {} \sum _{k=0}^{\infty }2^k\Big [\Pr (\lambda \ge k)-\Pr (\lambda \ge k+1)\Big ]\\= & {} \Pr (\lambda \ge 0)+\sum _{k=1}^{\infty }2^k\Pr (\lambda \ge k)-\sum _{k=0}^{\infty }2^k\Pr (\lambda \ge k+1)\\= & {} 1+\sum _{k=1}^{\infty }2^k\Pr (\lambda \ge k)-\frac{1}{2}\sum _{k_1=1}^{\infty }2^{k_1}\Pr (\lambda \ge k_1)\\= & {} 1+\frac{1}{2}\sum _{k=1}^{\infty }2^k\Pr (\lambda \ge k)\\= & {} 1+\frac{1}{2}\sum _{k=1}^{\infty }2^k\Big [1-(1-2^{-k})^d\Big ]^2. \end{aligned}$$

Thus,

$$\begin{aligned} E(2^{\lambda })= 1+\frac{1}{2}\sum _{k=1}^{\infty }2^k\Big [1-(1-2^{-k})^d\Big ]^2. \end{aligned}$$

(6)

We will prove an upper bound of 3d / 2 on $E(2^{\lambda })$ based on Equ. (6). For $d=1$, by Equ. (6) it is easy to calculate that $E(2^{\lambda })=3/2$, and so the lemma holds. Next we prove the lemma for integer $d>1$.

Note that for integers $k,d\ge 1$, clearly $2^k\big [1-(1-2^{-k})^d\big ]^2\le 2^k$, also

$$\begin{aligned} 2^k\Big [1-(1-2^{-k})^d\Big ]^2\le 2^k\Big [1-(1-2^{-k}d)\Big ]^2=2^{-k}d^2, \end{aligned}$$

where for the second inequality we use the fact that $(1-y)^d\ge 1-yd$ for $0\le y\le 1$ and $d\ge 1$.

Therefore, for any integer $\mu \ge 1$, we have

$$\begin{aligned} E(2^{\lambda })= & {} 1+\frac{1}{2}\sum _{k=1}^{\infty }2^k\Big [1-(1-2^{-k})^d\Big ]^2\\= & {} 1+\frac{1}{2}\sum _{k=1}^{\mu }2^k\Big [1-(1-2^{-k})^d\Big ]^2+\frac{1}{2}\sum _{k=\mu +1}^{\infty }2^k\Big [1-(1-2^{-k})^d\Big ]^2\\\le & {} 1+\frac{1}{2}\sum _{k=1}^{\mu } 2^k+\frac{1}{2}\sum _{k=\mu +1}^{\infty } 2^{-k}d^2\\= & {} 2^{\mu }+2^{-\mu -1}d^2. \end{aligned}$$

Next we show that for every integer $d>1$,

$$\begin{aligned} \min _{\mu \ge 1, ~\mu \in Z}\Big \{2^{\mu }+2^{-\mu -1}d^2\Big \}\le \frac{3d}{2}, \end{aligned}$$

which establishes Lemma 4.4.

Since

$$\begin{aligned} 2^{\mu }+2^{-\mu -1}d^2-\frac{3d}{2}=\frac{(2^{\mu }-d)(2^{\mu +1}-d)}{2^{\mu +1}}, \end{aligned}$$

for integer $d>1$ we can choose $\mu =\mu _0\ge 1$ such that $2^{\mu _0}\le d<2^{\mu _0+1}$, then from the above equality we have $2^{\mu _0}+2^{-\mu _0-1}d^2\le \frac{3d}{2}$.