NEUROM: a ROM based RNS digital neuron

Abstract

In this work, a fast digital device is defined, which is customized to implement an artificial neuron. Its high computational speed is obtained by mapping data from floating point to integer residue representation, and by computing neuron functions through residue arithmetic operations, with the use of table look-up techniques. Specifically, the logic design of a residue neuron is described and complexity figures of area occupancy and time consumption of the proposed device are derived. The approach was applied to the logic design of a residue neuron with 12 inputs and with a Residue Number System defined in such a way as to attain an accuracy better than or equal to the accuracy of a 20-bit floating point system. The proposed design (NEUROM) exploits the RNS carry independence property to speed up computations, in addition it is very suitable for using look-up tables. The response time of our device is about 8×T_ACC, where T_ACC is the ROM access time. With a value of T_ACC close to the 10 ns allowed by the current ROM technology, the proposed neuron responds within 80 ns, NEUROM is therefore the neuron device proposed in the literature which allows for maximum throughput. Moreover, when a pipeline mode of operation is adopted, the pipeline delay can assume a value as low as about 14 ns. In the case study considered, the total amount of ROM is about 5.55 Mbits. Thus, using current technology, it is possible to integrate several residue neurons into a single VLSI chip, thereby enhancing chip throughput. The paper also discusses how this amount of memory could be reduced, at the expense of the response time.

Introduction

For many years, interest in Artificial Neural Networks (ANN) has been growing in a broad area of applications. A great deal of effort has been made to enhance ANN performance, especially in the framework of very complex applications with real time constraints, for example, real time image processing and artificial vision (Bello, 2000, Clark and Furth, 1999, Hammadou et al., 2000, Montufar-Chaveznava et al., 2001). As far as computing support for an advanced ANN emulation is concerned, the approaches proposed in the literature range from general purpose superscalar computers down to special parallel systems (Ming-Jung, Hau, & Vijayan, 2003), as well as to very fast special processors. There are several examples of the last approach in the literature. In (Clarkson, Ng, & Guan, 1993) a special VLSI chip is presented which implements a p-RAM. In (Bolouri, Morgan, & Gurney, 1994) a neural system, HyperNet, is proposed, which is based on five VLSI custom IC's. In (El-Mousa & Clarkson, 1996), a neurocomputer board is presented, which incorporates the pRAM-256 VLSI neural processor. Another interesting example is a board containing four SIMD-type processors named NEUR04, which can efficiently support a general neural network (Komori, Arima, Kondo, Tsubota, Tanaka, & Kyuma, 1997). In (Lau, 1996) high performance chips/systems for processing neural networks are presented, among which there are two fully digital implementations, namely the floating point SNAP (SIMD Numerical Array Processor) and the fixed point CNAPS (Connectionist Neural Adaptive Processor System). In (Hasan & Ng, 1997) the SLiFBAM, a parallel VLSI BAM processor, is described, which is based on bidirectional associative memories. There are many research papers related to hardware implementation of neural networks. However, in order to make significant comparisons, only those proposals that strictly relate to the goals of our work are considered in this paper.

We focus on the definition of a fast digital device that implements an artificial neuron. More specifically, with reference to a trained neural network, a neuron is considered in which inputs and weights are real numbers. Our approach is to enhance its computational speed implementing it by means of a physical device based on residue arithmetic units. The proposed approach exploits both the RNS carry independence property to speed up the computations, and the ability of the RNS to be implemented by means of lookup tables.

In Section 2 the fundamentals of a Residue Number System (RNS), which is the number representation system adopted in this work, are recalled.

In Section 3 the integer reduction problem in an artificial neuron is considered, and the aim is to find the integer range required to emulate neural computations that use floating point notation, without loss of accuracy.

In Section 4 the logic design of a table lookup structure for a residue neuron is described, while Section 5 outlines a case study concerning the design of a residue neuron covering the requirements of many neural applications in terms of both input number and variables ranges.

With regard to design issues, a 12-input neuron is considered, as well as a 20-bit floating point notation, with 16 bits of signed mantissa and 4 bits of two-complement exponent, is assumed for all variables modelling the neuron. With these values, several applications can achieve the required level of accuracy and a ROM-based implementation (NEUROM) yields a high time performance and a practicable area occupancy. In fact, we evaluated the response time of this residue neuron to be less than 80 ns, while a total memory amount of about 5.55 Mbits is required. It will be shown that NEUROM has a better throughput than other high performance devices proposed in the literature. Moreover, the feasibility of integrating several NEUROMs on the same chip will be discussed.

The number of NEUROM inputs can be increased, provided that the overall number of ROMs is increased as well, without ROM address space increasing. On the other hand, the upper bound to the address space expansion limits the size of RNS moduli.

Finally, when a pipeline mode of operation is adopted, we have evaluated that the pipeline delay can assume a value close to the time necessary to access a ROM.

Memory-based implementations in the field of Pattern Recognition were first proposed in (Bledsoe and Blisson, 1962, Bledsoe and Browning, 1959), and memory-based neurons theory was assessed in (Gurney, 1989) and (Gorse and Taylor, 1988, Gorse and Taylor, 1990a, Gorse and Taylor, 1990b, Gorse and Taylor, 1990c, Gorse and Taylor, 1991, Gorse et al., 1993, Gorse et al., 1997).

In the following a nomenclature of less known mathematical terms and symbols is given. However, symbols will generally also be defined in the text when they first occur.

⌊x⌋

is the greatest integer less than or equal to the real number x

⌈x⌉

is the least integer greater than or equal to the real number x

abs(x)

is the absolute value of x

$|n{|}_m$

with n and m integers, denotes the remainder of $n/m$ that is $n-m\lfloor n/m\rfloor$

[A,B]

is the set of numbers between A and B, with A<B, A and B included

[A,B)

is the set of numbers between A and B, with A<B, A included, B not included

{e_i: P(e_i)}

is the set of elements e_i that satisfy predicate P(e_i)

{e_i}, i=1,…,n

is a set of n elements e_i

{e₁, e₂, e₃}

is the set of elements e₁, e₂, e₃

↔

states a one-to-one correspondence between an integer and its RNS representation

sign(x)

sign of x

a^*

is the real number corresponding to the floating point representation of the real number a

≈

approximate equality