Soft output decision convolutional (SONNA) decoders based on the application of neural networks

Abstract

The paper investigates principles of work and BER characteristics of a new soft decision algorithm for decoding convolutional codes that is based on neural network applications. The novelty of the algorithm is in its capability to generate soft output estimates of the message bits encoded. For this purpose the noise energy function, which is defined and used for the neural network decoding of convolutional codes, has been related to the well known log-likelihood function and the soft decision decoding rule has been defined and derived. The BER curves are obtained for this novel algorithm and then compared to the curve obtained by the Viterbi algorithm and the gradient descent algorithm. Based on the theoretical model a simulator of coding communication system has been developed and used to confirm theoretically expected results. It was found that the performances of the proposed soft decision decoder are comparable or better than the performances of the recurrent neural network decoder and decoders based on the Viterbi algorithm.

Introduction

Artificial neural networks (ANNs) have been applied in various fields of digital communications primarily due to their non-linear processing, possible parallel processing and efficient hardware implementations (Ibnkahla, 2000). Recently, substantial efforts have been made to apply ANNs in error control coding theory, initially, for block codes decoding (Bruck and Blaum, 1989), (Ciocoiu, 1996) and recently for convolutional (Wicker and Wang, 1996; Hamalainen and Henriksson, 1999a, Hamalainen and Henriksson, 1999b, Hamalainen and Henriksson, 2000; Berber et al., 2005), and turbo codes decoding (Buckley and Wicker, 1999). It was shown that the decoding problem could be formulated as a function minimization problem and the gradient descent algorithm was applied to solve this problem. The prototyping of neural network decoders was presented in (Salcic et al., 2006), and the algorithm was also implemented in hardware using floating—gate MOSFET circuits (Rantala et al., 2001).

It is to note that the theory developed until year 2000 was related to the 1/n rate codes, defined only for small values of n, with various constraint lengths. The general expressions needed for decoder development, presented in the form of the noise energy function, were derived for the 1/n-rate codes in Secker et al. (2003), Berber et al. (2005) and for the general k/n-rate code in Berber (2004). In this way the central problem of decoding based on neural networks application become solvable and reduced to a function minimization problem of the derived differentiable noise energy function.

The Viterbi algorithm is known to be asymptotically optimal in the maximum likelihood sense as can be seen from Appendix A, where a brief explanation of this algorithm is presented. However, because its complexity increases exponentially with the constraint length of the encoder, further research was undertaken to reduce this complexity (Zadeh and Soleymani, 2005b). For this purpose an adaptive algorithm was developed, called adaptive M-algorithm, which starts decoding with a reduced number of survivor paths and increases the number of paths depending on the error state in the channel that is controlled by CRC error detection code. Practical problems related to the increase of decoding speed of the Viterbi decoder are analysed in Tang and Parhi (2005) and three techniques are proposed to overcome these problems. New decoder architecture is proposed that is based on three existing techniques, lookahead, sliding block and parallel processing technique, which increase the speed of decoding with a relatively low clock frequency.

The soft output Viterbi algorithm (SOVA) is proved as a very good algorithm that improves significantly the coding gain achieved by the hard decision Viterbi algorithm. The algorithm is based on calculating the log-likelihood function (LLF) and making the estimates of message bits. If the LLF is available it is possible to include the estimates of the a priori probabilities and to develop turbo decoders based on the SOVA algorithm.

The intention of this paper is to extend the results presented in Berber (2004), Berber et al. (2005) by developing and simulating a soft output decoding algorithm based on neural networks application (SONNA). It will be shown by numerical calculations that the SONNA algorithm can be used to decode convolutional codes (Berber, 2004). For that purpose the LLF is derived and the algorithm for soft decoding is developed and simulated. According to this algorithm an iterative decoding procedure is implemented and simulated that starts with a specified set of message bits and finishes when the last estimated set of message bits is identical to the set of bits obtained in the previous iteration step. This algorithm is demonstrated on a $\frac{1}{2}$-rate convolutional code.