Analysis of Relationship Between Modes of Intercomputer Communications and Fault Types in Redundant Computer Systems

Abstract

This paper analyzes the reasons of appearance of non - Byzantine and Byzantine fault types in redundant computer systems. The proposed approach is based on analysis of the relationship between the modes of intercomputer communications and fault types. This analysis allows the users to design the redundant computer systems in such a way that Byzantine faults cannot appear. Consequently, designing the redundant computer systems, in which Byzantine faults cannot appear, allows the designers to increase the degree of reliability by preventing the masking of any forms of appearance of faults and by decreasing the time period of checkpoints. In addition, this approach decreases the cost of software and hardware involved in the execution of fault-tolerant procedures.

Appendix I: Examples

Example 1

Let us consider a Redundant Computer System (RCS) which consists of four computers (nodes) (Fig. 1 with $N=4$). Suppose that the fourth node is faulty and computational results of non-faulty nodes are “1”. If the fourth node sends the same values, namely logical “0”, to all others during the exchange by computational results it means that the fault type is Non-Byzantine.

As we see, vectors consist of the same values in all non-faulty nodes. However, if the fourth node sends different values, namely to the first and third nodes logical “0” and to the second node logical “1”, it means that the fault type is Byzantine.

As we see, vectors consist of different values in non-faulty nodes. The index of x (“0” or “1”) refers to values in faulty nodes.

Example 2

For explanation of the masking of fault forms, the Determinate Byzantine agreement protocol will be used. Suppose that $N=4$, $k=1$, consequently, $m=2$ (Table 3). Let us consider the examples for three cases: (1) there is no fault in RCS; (2) the Byzantine fault appears in RCS during the first intercomputer communication round and (3) the Byzantine fault appears in RCS during the second intercomputer communication round.

Example 2.1. First, let us consider the case when there is no fault in RCS. Suppose that computational results of nodes are the logical “1”.

In the first intercomputer communication round, nodes exchange the computational results. So after the first round, the following vectors are formed.

In the second intercomputer communication round, the $i^{th}$ node transmits to the $j^{th}$ node the computational result of the $n^{th}$ node, where i, j, $n= 1, 2, 3, 4$ and $i \ne j \ne n$. After the second round, the matrices of the computational results are formed.

After majority voting in columns, the final vectors are formed. Element $a_{nn}$ is chosen as majority of values in the $n^{th}$ column, where n = 1, 2, 3, 4.

As we see, all matrices consist of the same computational results and all nodes forms the same final vectors. This means that there is no fault in RCS.

Example 2.2. Now, let us consider the case when the Byzantine fault appears in the first intercomputer communication round. Suppose that the fourth node is faulty and computational results of nodes are the logical “1”, “1”, “1”, “x” (“0” or “1”).

Suppose that the faulty node sends to the first and third nodes the logical “0” and to the second node the logical “1” during the first intercomputer communication round. After the first round, the following vectors are formed.

After the second round, the following matrices and after majority voting in columns, the following final vectors are formed.

As we see, the matrices in all non-faulty nodes consist of different computational results. However, all non-faulty nodes form the same final vectors, where the fourth computational result differs from others. So the non-faulty nodes determine that the fourth node is faulty. In this case, the Determinate Byzantine agreement algorithm transforms the Byzantine fault type appearing in the first intercomputer communication round to Non-Byzantine type.

Example 2.3. Finally, let us consider the case when the Byzantine fault appears in the second intercomputer communication round. Suppose that the computational results of nodes are the logical “1”.

After the first round, the following vectors are formed.

Suppose that the Byzantine fault appears in the fourth node during the second intercomputer communication round. The faulty node will send different computational results to the non-faulty nodes. After the second round, the following matrices are formed.

After majority voting in columns, the following final vectors are formed.

As we see, the matrices in all non-faulty nodes consist of different computational results. Despite this, all non-faulty nodes form the same final vectors where all computational results are the same. The non-faulty nodes determine that there is no fault in RCS. This means that the Determinate Byzantine agreement algorithm masks the Byzantine fault type that appeared in the second round. The same situation will take place when the Non-Byzantine fault appears in RCS during the second intercomputational round. As result, the Byzantine agreement algorithms mask the Non-Byzantine and Byzantine fault appearance forms that occurred during the second and following rounds. Masking of faults is very dangerous because they can be accumulated and lead the system to failure.

Example 3

Let us show that the use of a Byzantine agreement algorithm (for example, the Determinate Byzantine agreement algorithm) cannot counteract the Byzantine fault in RCS with $N=3$ (Fig. 1 with $N=3$). Suppose that the third node is faulty and computational results of nodes are “1”, “1”, “x” (“0” or “1”).

Suppose that the faulty node sends to the first node the logical “0” and to the second node the logical “1” during the first intercomputer communication round. So after the first round, the following vectors are formed.

After the second round, the following matrices are formed.

After majority voting in columns, the following final vectors are formed. As we see, all non-faulty nodes form the different final vectors and could not determine the faulty node. So the Byzantine agreement algorithms cannot detect the Byzantine fault in RCS with $N=3$.

Example 4

Let us consider RCS where Non-Byzantine and Byzantine fault types may occur (Fig. 12).

Fig. 12.

Example for RCS using Protocol #II

Suppose that the $n^{th}$ ($n=1, 2, 3, 4$) computer (node) in RCS consists of Central Processor (CP), Input Processor (IP) and Output Processor (OP) as shown in Fig. 12.

Each CP controls its own computational process, computes its own computational result and executes its own fault-tolerant procedure on the basis of IDS which consists of computational results of all nodes in RCS. Each IP consists of 4 receivers (R1, R2, R3, and R4) which receive the computational results from the other nodes. Each OP consists of 4 transmitters (T1, T2, T3, and T4) which transmit the computational results to the other nodes.

On the one hand, in this protocol, OP of the $n^{th}$ node ($n=1, 2, 3, 4$) transmits the computational results to IP of the $i^{th}$ nodes ($i=1, 2, 3, 4$ and $n$ $\ne$ $i$) in parallel, by using three busses. Nodes execute this procedure in sequential order by using three busses in the time-sharing mode. For example:

• T1, T2 and T3 of OP of the $1^{st}$ node transmit the computational result in the instant of time $t_1$ to other nodes by using three busses in parallel;

• T1, T2 and T3 of OP of the $2^{nd}$ node transmit the computational result in instant of time $t_2$ to other nodes by using three busses in parallel;

• T1, T2 and T3 of OP of the $3^{rd}$ node transmit the computational result in instant of time $t_3$ to other nodes by using three busses in parallel;

• T1, T2 and T3 of OP of the $4^{th}$ node transmit computational result in instant of time $t_4$ to other nodes by using three busses in parallel.

On the other hand, in this protocol, IP of the $n^{th}$ node ($n=1, 2, 3, 4$) receives the computational results from OP of the $i^{th}$ nodes ($i=1, 2, 3, 4$ and $n$ $\ne$ $i$) in the time-sharing mode by using one bus. All nodes execute this procedure in parallel. For example:

• R1 of IP of the $1^{st}$ node receives the computational results from T1 of the $2^{nd}$, $3^{rd}$ and $4^{th}$ nodes in instants of times $t_1$, $t_3$ and $t_4$ accordingly by using one bus;

• R1 of IP of the $2^{nd}$ node receives the computational results from T1 of the $1^{st}$ node and from T2 of $3^{rd}$ and $4^{th}$ nodes in instants of times $t_1$, $t_3$ and $t_4$ accordingly by using one bus;

• R1 of IP of the $3^{rd}$ node receives the computational results from T2 of the $1^{st}$ and $2^{nd}$ nodes and from T3 of the $4^{th}$ node in instants of times $t_1$, $t_2$ and $t_4$ accordingly by using one bus;

• R1 of IP of the $4^{th}$ node receives the computational results from T3 of the $1^{st}$, $2^{nd}$ and $3^{rd}$ nodes in instants of times $t_1$, $t_2$ and $t_3$ accordingly by using one bus.

We assume that only one fault may occur in RCS in any instant of time (Sect. 2.4). According to this assumption, three cases may take place:

• (1) Fault occurs in IP (CP and OP are non-faulty) of the $n^{th}$ node. The functions of IP are to receive the computational results from the other nodes and to save them in its buffer memory. According to assumption if IP is faulty, CP and OP are non-faulty in the $n^{th}$ ($n=1, 2, 3, 4$) node.

  • Faulty IP may change the received correct computational results from the $i^{th}$ ($i=1, 2, 3, 4$ and $n \ne i$) nodes in the first and following communication rounds and save correct or incorrect computational results in its buffer memory.

  • Non-faulty CP computes its own correct computational result.

  • Non-faulty OP transmits the same (correct) computational result (which is computed in non-faulty CP and which was not received and changed by faulty IP) in the first communication round and the same (possibly correct or incorrect) computational results (which were received and may be changed by faulty IP in the first and following communication rounds) in the second and the following communication rounds to the other nodes.

  In this case, transmitters of non-faulty OP will transmit the same computational results (it is not important whether it is correct or incorrect) to other nodes during all communication rounds. Consequently, if a fault occurs in IP, the type of appeared faults in the above RCS structure is only Non-Byzantine.

• (2) Fault occurs in CP (IP and OP are non-faulty) of the $n^{th}$ node. The functions of CP are to control its own computational process, to compute its own computational result and to execute its own fault-tolerant procedure on the basis of IDS which consists of computational results of all nodes in RCS. According to our assumption, if CP is faulty, IP and OP are non-faulty in the $n^{th}$ node.

  • Non-faulty IP receives the same (correct) computational results from the other nodes in the first and following communication rounds.

  • Faulty CP may compute its own incorrect computational result.

  • Non-faulty OP transmits the same (may be correct or incorrect) computational result (which is computed in faulty CP of the $n^{th}$ node) in the first communication round and the same (correct) computational results (which were received by non-faulty IP in the first and the following communication rounds) in the second and following communication rounds.

  In this case, transmitters will also transmit the same computational results (it is not important whether it is correct or incorrect) to other nodes during all communication rounds. Consequently, if fault occurs in CP, the type of appeared faults in the above mentioned RCS structure is also only Non-Byzantine.

• (3) Fault occurs in OP (CP and IP are non-faulty) of the $n^{th}$ node. The main function of OP is to transmit the computational results to the other nodes. According to our assumption, if OP is faulty, IP and CP are non-faulty in the $n^{th}$ node.

  • Non-faulty IP receives the correct computational results from the other nodes in the first and following communication rounds.

  • Non-faulty CP computes its own correct computational result.

  • Faulty OP might transmit the different computational results to other nodes during all communication rounds because OP has multiple transmitters (in this case three of them are used) for transmission and one or more of them might be faulty and may change the transmitted values.

  Consequently, if fault occurs in OP, the type of appeared faults in the above RCS structure is Non-Byzantine or Byzantine.

Let us change the connections between nodes in RCS in Fig. 12 so that the Byzantine fault type could not occur (Fig. 13).

Fig. 13.

Example for RCS using Protocol #IV (Byzantine fault free protocol)

On the one hand, in this protocol, only one transmitter of OP of the $n^{th}$ node ($n=1, 2, 3, 4$) transmits the computational result to IP of all other nodes in the broadcast mode by using one bus simultaneously. Nodes execute this procedure in sequence. For example:

• T1 of OP of the $1^{st}$ node transmits the computational result in instant of time $t_1$ to other nodes in the broadcast mode by using one bus simultaneously;

• T1 of OP of the $2^{nd}$ node transmits the computational result in instant of time $t_2$ to other nodes in the broadcast mode by using one bus simultaneously;

• T1 of OP of the $3^{rd}$ node transmits the computational result in instant of time $t_3$ to other nodes in the broadcast mode by using one bus simultaneously;

• T1 of OP of the $4^{th}$ node transmits the computational result in instant of time $t_4$ to other nodes in the broadcast mode by using one bus simultaneously;

On the other hand, IP of the $n^{th}$ node ($n=1, 2, 3, 4$) receives the computational results from OP of the $i^{th}$ nodes ($i=1, 2, 3, 4$ and $n$ $\ne$ $i$) by using (N-1) bus in sequential order. All nodes execute this procedure in parallel. For example:

• R1, R2 and R3 of IP of the $1^{st}$ node receive the computational result from $2^{nd}$, $3^{rd}$ and $4^{th}$ nodes in instants of times $t_2$, $t_3$ and $t_4$ accordingly by using three different busses;

• R1, R2 and R3 of IP of the $2^{nd}$ node receive the computational result from $1^{st}$, $3^{rd}$ and $4^{th}$ nodes in instants of times $t_1$, $t_3$ and $t_4$ accordingly by using three different busses;

• R1, R2 and R3 of IP of the $3^{rd}$ node receive the computational result from $1^{st}$, $2^{nd}$ and $4^{th}$ nodes in instants of times $t_1$, $t_2$ and $t_4$ accordingly by using three different busses;

• R1, R2 and R3 of IP of the $4^{th}$ node receive the computational result from $1^{st}$, $2^{nd}$ and $3^{rd}$ nodes in instants of times $t_1$, $t_2$ and $t_3$ accordingly by using three different busses;

According to the assumption, three cases are also possible here. Discussions about the first and second cases are the same as for the previous protocol. The difference is in the third case in which the transmitter of faulty OP transmits the same computational results (it is not important whether correct or incorrect) because of the used broadcast mode.

Consequently, if a fault occurs in OP, the type of faults appearing in the RCS structure will only be Non-Byzantine.

As a result, we changed the connections between nodes and got RCS where only Non-Byzantine fault type might occur. Consequently, by changing connection modes between nodes we can block the occurrence of the Byzantine fault type in RCS.