Abstract
This paper explores internal representation power of product units [1] that act as the functional nodes in the hidden layer of a multi-layer feedforward network. Interesting properties from using binary input provide an insight into the superior computational power of the product unit. Using binary computation problems of symmetry and parity as illustrative examples, we show that learning arbitrary complex internal representations is more achievable with product units than with traditional summing units.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Durbin and Rumelhart, D. E.: Product units: a computationally powerful and biologically plausible extension to backpropagation networks, Neural Computation 1 (1989), 133-142.
Giles, C. L., Miller, C. B., Chen, D., Chen, H. H., Sun, G. Z. and Lee, Y. C.: Learning and extracting finite state automata with second order recurrent neural networks, Neural Computation 4 (3), (1992), pp. 393-405.
Zeng, Z., Goodman, R. M. and Smyth, P.: Learning finite state machines with self-clustering recurrent networks, Neural Computation 5 (1993), 976-990.
Elman, J. L.: Distributed representations, simple recurrent networks, and grammatical structure, Machine Learning 7 (1991), 195-225.
Lin, C. T. and Lee, C. S.: Neural Fuzzy Systems, A Neuro-Fuzzy Synergismto Intelligent Systems, Prentice-Hall, 1996.
Rumelhart, D. E. and McClelland, J. L.: ParallelDistributed Processing, Explorations in the Microstructure of Cognition, Vol.1, MIT Press, Cambridge, 1986.
Minsky, M. and Papert, S.: Perceptrons, MIT Press, Cambridge, 1969.
Chen and Bastani, F.: ANN with two-dendrite neurons and its weight initialization, Proc. of Internat. Joint Conf. on Neural Networks, 1992, Vol. 3, pp.139-146.
Chung, P. C. and Krile, T. F.: Reliability characteristics of quadratic Hebbian-type associative memories in optical and electronic network implementations, IEEE Trans. Neural Networks 6(2) (1995), 357-367.
Wolpert, S. and Micheli-Tzanakou, E.: A Neuromime in VLSI, IEEE Trans. Neural Networks 7(2) (1996), 300-306.
Baldi, P. and Venkatesh, S.: Random interactions in higher order neural networks, IEEE Trans. Inf. Theory 39 (1993), 274-283.
Wang, J. H. and Jeng, M. D.: Performance characterization of product unit neural network associative memories, Proc. of 9th Internat. Conf. CAD/CAM, Robotics and Factories of the Future, Newark, NJ, August 1993.
Jang, J. S. R., Sun, C. T. and Mizutani, E.: Neuro-Fuzzy and Soft Computing, Prentice-Hall, 1997.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wang, JH., Yu, YW. & Tsai, JH. On the Internal Representations of Product Units. Neural Processing Letters 12, 247–254 (2000). https://doi.org/10.1023/A:1026534303563
Issue Date:
DOI: https://doi.org/10.1023/A:1026534303563