Abstract
Artificial neural networks employing stochastic arithmetic can under certain conditions outperform those based upon conventional radix arithmetic in reduced power dissipation, silicon area and improved fault tolerance. This paper describes limitations due to the inherent variance in the stochastic signals. We introduce and compare two stochastic multiplexing schemes, inter-count and intra-count multiplexing, for accumulating the total inputs to the artificial neurons.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Card, H. C.: Doubly stochastic Poisson processes in artificial neural learning, IEEE Trans. Neural Networks, 9 (1998), 229–231.
Gaines, B. R.: Stochastic computing systems, In: J. F. Tou (ed.), Advances in Information Systems Science, 2(2), NewYork: Plenum Press, 1969, pp. 37–172.
Tomlinson, M. S., Walker, D. J. and Sivilotti, M. A.: A digital neural network architecture for VLSI, Int. Joint Conf. on Neural Networks,Vol. 2, San Diego, pp. 545–550, 1990.
Van den Bout, D. E. and Miller III, T. K.: A digital architecture employing stochasticism for the simulation of Hopfield neural nets, IEEE Trans. Ccts and Syst, 36 (1989), 732–738.
Tomberg, J. E. and Kaski, K.: Pulse density modulation technique in VLSI implementation of neural network algorithms, IEEE J. Solid St. Ccts, 25 (1990), 1277–1286.
Dickson, J. A., McLeod, R. D. and Card, H. C.: Stochastic arithmetic implementations of neural networks with in situ learning, Proc. Int. Conf. on Neural Networks, pp. 711–716, San Francisco, 1993.
Shawe-Taylor, J., Jeavons, P. and van Daalen, M.: Probabilistic bit-stream neural chip: theory, Connection Science, 3 (1991), 317–328.
Zhao, J., Shawe-Taylor, J. and van Daalen, M.: Learning in stochastic bit-stream Neural networks, Neural Networks, 9 (1996), 991–998.
Brown, B. D. and Card, H. C.: Stochastic neural computation I and II, IEEE Trans. on Computers, Vol. 50, No. 9, pp. 891–920, Sept. 2001.
Burge, P. S., van Daalen, M. R., Rising, B. J. P. and Shawe-Taylor, J. S.: Stochastic bit-stream neural networks, In: W. Maass and C. M. Bishop (eds), Pulsed Neural Networks, Cambridge, MA: MIT Press, 1999.
Hortensius, P. D., McLeod, R. D. and Card, H. C.: Parallel random number generation for VLSI systems using cellular cutomata, IEEE Trans. on Computers, 38 (1989), 1466–14729.
Zhou, H., Card, H. C. and Bridges, G. E.: Parallel pseudorandom number generation in GaAs cellular automata for high speed circuit testing, J. Elect. Test. Theory Applns., 6 (1995), 325–330.
Hinton, G. E. and Sejnowski, T. J.: Learning and relearning in Boltzmann machines, In: D. E. Rumelhart and J. L. McClelland (eds), Parallel Distributed Processing: Explorations in Microstructure of Cognition, Cambridge, MA: MIT Press, 1986.
Hinton, G. E., Dayan, P., Frey, B. J. and Neal, R. M.: The w ake-sleep algorithm for unsupervised neural networks, Science, 268 (1995), 1158–1161.
Maass, W. and Bishop, C. M. (eds): Pulsed Neural Networks, Cambridge, MA: MIT Press, 1999.
Parzen, E.: Stochastic Processes, San Francisco: Holden-Day, 1962.
Trivedi, K. S.: Probability and Statistics with Reliability, Queueing, and Computer Science Applications, Englewood Cliffs: Prentice-Hall, 1982.
Teich, M. C. and Diament, P.: Flat counting distribution for triangular-modulated Poisson process, Phys. Lett., 30(A) (1969), 93–94.
Teich, M. C. and Card, H. C.: Photocounting distributions for exponentially-decaying sources, Optics Letters, 4 (1979), 146–148.
Bishop, C. M.: Neural Networks for Pattern Recognition, Oxford: Clarendon Press, 1995.
Card, H. C.: Compound binomial processes in neural integration, IEEE Trans. Neural Networks, 12, No. 6, (2001), in press.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Card, H.C. Input Multiplexing in Artificial Neurons Employing Stochastic Arithmetic. Neural Processing Letters 15, 1–8 (2002). https://doi.org/10.1023/A:1013805129793
Issue Date:
DOI: https://doi.org/10.1023/A:1013805129793