The entropy of a distributed computation random number generation from memory interleaving

Abstract

We ask to what extent processes communicating through shared memory can extract randomness from their underlying scheduler, e.g., to generate random numbers for cryptographic applications. We introduce the quantitative notions of entropy rate and information capacity of a distributed algorithm. Whilst the entropy rate measures the Shannon information that may pass from a given scheduler to the processes executing the algorithm, the information capacity measures the optimal entropy rate over all possible schedulers. We present a general method for computing these quantities by classifying distributed algorithms according to their pattern of shared memory accesses. We then address the issue of effectively extracting, online, the information produced by the scheduler into a meaningful format at every process. We present Duez, an algorithm solving this problem with an optimal memory consumption. Putting these principles into practice, we introduce Co-RNG, a random number generator that leverages the unpredictability of modern processors state. The power of Co-RNG comes from its simplicity. No specialized hardware is required: two concurrent threads actively perform successive reads and writes to shared memory locations. Another thread collects the sequences of values read by these two threads and seeks to reconstruct the interleaving of read and write operations. The resulting (Markovian) interaction scheme is then used to produce random bits. This simplicity yields a transparent behavior. If the hardware exhibits enough entropy, then Co-RNG efficiently extracts random numbers from it. We successfully experimented Co-RNG on various idle as well as loaded platforms, from laptops and desktops featuring Intel Core processors, to servers with Intel Xeon and AMD Opteron. Co-RNG passes all state-of-the-art random number generator statistical test suites while being faster than current I/O sampling based methods by 2–3 orders of magnitude.

Appendix A: a description of Duez

In this section, we proceed to the detailed description of Duez. Each iteration of the detection loop splits in two parts. The conditional block at lines 14–31 rebuilds steps of schedules whose reconstruction stopped at a point where both processes are about to write. Symmetrically, the block at lines 32–46 rebuilds steps starting by a read for both processes.

Note that in both blocks, steps are pushed only as follows. Either an even number of steps of the same process are appended to the schedule (lines 17, 24, 35 and 39), or a step of each process is added (lines 27 and 42). Since both processes start by writing, it follows that, at any moment, processes are either both about to write or both about to read in the rebuilt schedule. Similarly, the variables $my\_steps$ and $her\_steps$ are updated at lines 18, 25, 28–29, 36, 40 and 43–44 to keep track of the number of steps rebuilt for each process. These variables are consequently always both even (when both processes are about to write) or both odd (when they are about to read).

Fig. 21

Round-trip from the point of view of $P_0$

Fig. 22

Possible schedules starting by write operations and pending detection

Fig. 23

Possible schedules starting by read operations and pending detection

Fig. 24

Final values of the journal variables of $P_0$

The structure of the two conditional blocks is very similar. Process $P_i$ first checks (lines 14 or 32) if a full round-trip (see Sect. 5.2) occurred in the schedule between the end of its last fully rebuilt round and the current round. This means that an ordered chain of events as displayed in Fig. 21 took place since the last of its fully rebuilt rounds. The index e computed at line 15 (resp. 33) captures the oldest entry in the journal of process $P_{1-i}$ concerning a round of process $P_i$ that has not been fully rebuilt yet (if both processes are about to read, the corresponding write may have been rebuilt already).

We now proceed to the analysis of the different conditional branches. To visualize the reconstruction loop, we present in Figs. 22 and 23 the trees of all possible schedules containing one round-trip from the point of view of $P_0$. The operations marked in blue form the communication round-trip.

We focus first on Fig. 22. This tree describes the possibilities when both processes are about to write, and corresponds to the conditional block at line 14. The conditional block reconstructs the sequence $(W_1R_1)^a (W_0R_0)^b (W_0 || W_1)$ in the branches A to D. The block reconstructs the sequence $(W_1R_1)^a (W_0R_0)^{b+1}$ in the other branches. The condition at line 19 matches the branch A, B, C, F, G, H, and, if $c \ne 0$, the branch D as well. The condition at line 26 matches A, B, C, D.

Figure 23 presents the tree of possibilities when both processes are about to read, which corresponds to the conditional block at line 32. The condition at line 37 matches the branches D.

The main technical difficulty in the algorithm is the computation of the several exponents (e.g., $(W_1R_1)^a$) from the different counters in the journal. We illustrate such a computation on the branch G of Fig. 22; the other cases being treated similarly. Figure 24 gives the values of the journal variables ($my\_jour$ and $her\_jour$) of $P_0$ at the end of the schedule G. The schedule G is divided in 11 segments. We assume that the exponents a and m are non-zero. The entry 1 of $my\_jour$ is recorded during the first round $W_0R_0$ of segment 1 since it is the first time that $P_0$ learns something new about $P_1$ (assuming $a \ne 0$). More precisely, at this moment, $P_0$ learns at his round $\frac{my\_steps}{2}$ that $P_1$ has completed $a-1$ more rounds since her last reconstructed round $\frac{her\_steps}{2}$. A similar argument shows that the entry 0 of $my\_jour$ is recorded right after the read $R_0$ of segments 5 and 11. The entry 2 of $her\_jour$ is recorded (by $P_1$) after the read $R_1$ of segment 4. The entry 1 of $her\_jour$ is recorded after the read $R_1$ of segment 7. The entry 0 of $her\_jour$ is recorded after the first read $R_1$ of segment 10.

After the last read of segment 11, process $P_0$ triggers a reconstruction loop since the condition at line 14 in Algorithm 2 is satisfied. The value e computed at line 15 is $e = 2$. The value $\alpha$ computed at line 16 is $\alpha = a$. Process $P_0$ then pushes (correctly) the segment $(W_1R_1)^a$, and increments the variable $her\_steps$ accordingly. The condition at line 19 is not satisfied, so that the value $\beta$ computed at line 20 is $\beta = b+1$. Process $P_0$ then pushes $(W_1R_1)^{b+1}$, and increments the variable $my\_steps$ accordingly. At this point, the conditions at lines 14 and 32 (checking that there is one round-trip) have become false: process $P_0$ needs to wait for another round-trip to complete before proceeding to reconstruct further steps.

Thanks to the fairness condition, there will always be enough round-trips for both processes to keep progressing in their reconstruction. Thus, Duez solves the reconstruction problem.