An FPGA-Based Multiple-Weight-and-Neuron-Fault Tolerant Digital Multilayer Perceptron (Full Version)

Abstract

A method to implement a digital multilayer perceptron (DMLP) in an FPGA is proposed, where the DMLP is tolerant to simultaneous weight and neuron faults. It has been shown in [1] that a multilayer perceptron (MLP) which has successfully trained using the deep learning method is tolerant to multiple weight and neuron faults where the weight faults are between the hidden and output layers, and the neuron faults are in the hidden layer. Using this fact, a set of weights in the trained MLP is installed in an FPGA to cope with these faults. Further, the neuron faults in the output layer are detected or corrected using SECDED code. The above process is done as follows. The generator developed by us automatically outputs a VHDL source file which describes the perceptron using a set of weight values in the MLP trained by the deep learning method. The VHDL file obtained is input to the logic design software Quartus II of Altera Inc., and then, implemented in an FPGA. The process is applied to realizing fault-tolerant DMLPs for character recognitions as concrete examples. Then, the perceptrons to be made fault-tolerant and corresponding non-redundant ones not to be made fault-tolerant are compared in terms of not only reliability and fault rate but also hardware size, computing speed and electricity consumption. The data show that the fault rate of the fault-tolerant perceptron can be significantly decreased than that of the corresponding non-redundant one.

This paper is the full version of [2].

Itsuo Takanami—Retired.

Appendix A: Proof of Theorem 1

Let $X$ ($\hat{X}$) be the weighted sum of inputs to a neuron in the output layer when the MLP is fault free (the MLP has faulty neurons in the hidden layer and/or faulty weights). $X$ and $\hat{X}$ are given as follows.

$$\begin{aligned} X= \sum _{i\in I_M} w_ih_i \end{aligned}$$

(16)

where $I_M$ is the set of neurons in the hidden layer indices. Note that the index of the weight which is connected to the target neuron in the output layer is also represented as $i$.

$$\begin{aligned} \hat{X}&= \sum _{k \notin \hat{L}, k \notin \hat{J}}w_k \cdot h_k + \sum _{k \in \hat{L}, k \notin \hat{J}}w'_k \cdot h_k \nonumber \\&\quad + \sum _{k \notin \hat{L}, k \in \hat{J}}w_k \cdot h'_k + \sum _{k \in \hat{L}, k \in \hat{J}}w'_k \cdot h'_k \end{aligned}$$

(17)

where $w_i$ ($w'_i$) is the value of the $i$-th weight when it is healthy (faulty) and $h_i$, ($h'_i$) is the value of the $i$-th neuron in the hidden layer when it is healthy (faulty).

From the Eqs. (16) and (17),

$$\begin{aligned} \hat{X} -X&= \sum _{k \in \hat{L}, k \notin \hat{J}}(w'_k -w_k) \cdot h_k \end{aligned}$$

(18)

$$\begin{aligned}&\quad + \sum _{k \notin \hat{L}, k \in \hat{J}}w_k \cdot (h'_k -h_k) \end{aligned}$$

(19)

$$\begin{aligned}&\quad + \sum _{k \in \hat{L}, k \in \hat{J}}(w'_k \cdot h'_k-w_k \cdot h_k) \end{aligned}$$

(20)

In addition, the following equations are true because of Assumptions 2 and 3, and $h_k$ being 0 or 1.

$$-2 \cdot w_{max} \le (w'_k -w_k) \cdot h_k \le 2 \cdot w_{max}$$

$$\begin{aligned} -w_{max} \le w_k \cdot (h'_k -h_k) \le w_{max} \end{aligned}$$

$$-2 \cdot w_{max} \le (w'_k \cdot h'_k-w_k \cdot h_k) \le 2 \cdot w_{max}$$

Furthermore,

$$N_{18} + N_{20} = |\hat{L}|,$$

$$N_{19} + N_{20} = |\hat{J}|,$$

and

$$N_{20}=|\hat{J} \cap \hat{L}|,$$

where $N_{18}$, $N_{19}$, and $N_{20}$ are the number of the terms in the summations of Eqs. (18), (19), and (20), respectively. Therefore,

$$\begin{aligned} 2N_{18}+N_{19}+2N_{20}= & {} |\hat{J}|+2|\hat{L}|-|\hat{J} \cap \hat{L}| \\= & {} |\hat{J} \cup \hat{L}| + |\hat{L}| \nonumber \end{aligned}$$

Thus,

$$-w_{max}\cdot (|\hat{J} \cup \hat{L}| + |\hat{L}|)\le (\hat{X} - X)\le w_{max}\cdot (|\hat{J} \cup \hat{L}| + |\hat{L}|).$$

Then

$$X-w_{max}\cdot (|\hat{J} \cup \hat{L}| + |\hat{L}|)\le \hat{X}\le X + w_{max}\cdot (|\hat{J} \cup \hat{L}| + |\hat{L}|).$$

From this equation and the $DL$-$cond(N_d)$,

• $\hat{X}$ $\ge$ $w_{max}$ $\cdot$ $(N_d -$ $(|\hat{J} \cup \hat{L}| + |\hat{L}|))\ge 0$ (if $t_p = 1$),

• $\hat{X}$ $<$ $-$ $w_{max}$ $\cdot$ $(N_d -$ $(|\hat{J} \cup \hat{L}| + |\hat{L}|))< 0$ (if $t_p = 0$).

From these equations, the Eq. (5), and the use of the step function for calculating the output of each neuron, the theorem is proved.    $\Box$