Skip to main content
SpringerLink
Log in

Multilayer Feedforward Neural Network Based on Multi-valued Neurons (MLMVN) and a Backpropagation Learning Algorithm

  • Original Paper
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

A multilayer neural network based on multi-valued neurons (MLMVN) is considered in the paper. A multi-valued neuron (MVN) is based on the principles of multiple-valued threshold logic over the field of the complex numbers. The most important properties of MVN are: the complex-valued weights, inputs and output coded by the kth roots of unity and the activation function, which maps the complex plane into the unit circle. MVN learning is reduced to the movement along the unit circle, it is based on a simple linear error correction rule and it does not require a derivative. It is shown that using a traditional architecture of multilayer feedforward neural network (MLF) and the high functionality of the MVN, it is possible to obtain a new powerful neural network. Its training does not require a derivative of the activation function and its functionality is higher than the functionality of MLF containing the same number of layers and neurons. These advantages of MLMVN are confirmed by testing using parity n, two spirals and “sonar” benchmarks and the Mackey–Glass time series prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aizenberg I, Aizenberg N, Vandewalle J (2000) Multi-valued and universal binary neurons: theory, learning, applications. Kluwer, Boston

    Google Scholar 

  2. Aizenberg I, Bregin T, Butakoff C, Karnaukhov V, Merzlyakov N, Milukova O (2002) Type of blur and blur parameters identification using neural network and its application to image restoration. In: Dorronsoro JR (eds) Lecture notes in computer science, 2415. Springer, Berlin Heidelberg New York, pp 1231–1236

    Google Scholar 

  3. Aizenberg I, Myasnikova E, Samsonova M, Reinitz J (2002) Temporal classification of Drosophila segmentation gene expression patterns by the multi-valued neural recognition method. J Math Biosci 176(1):145–159

    Article  MathSciNet  MATH  Google Scholar 

  4. Aizenberg NN, Ivaskiv Yu L, Pospelov DA (1971) About one generalization of the threshold function (in Russian). The reports of the Academy of Sciences of the USSR. Doklady Akademii Nauk SSSR 196:1287–1290

    Google Scholar 

  5. Aizenberg NN, Ivaskiv Yu L (1977) Multiple-valued threshold logic (in Russian). Naukova Dumka, Kiev

    Google Scholar 

  6. Aizenberg NN, Aizenberg IN (1992) CNN based on multi-valued neuron as a model of associative memory for gray-scale images. In: Proceedings of the second IEEE International workshop on cellular neural networks and their applications, Technical University Munich, Germany, 14–16 October, 1992, pp 36–41

  7. Aoki H, Kosugi Y (2000) An image storage system using complex-valued associative memory. In: Proceedings of the 15th international conference on pattern recognition, vol 2. IEEE Computer Society Press, pp 626–629

  8. Aoki H, Watanabe E, Nagata A, Kosugi Y (2001) image association for endoscopic positional identification using complex-valued associative memories. In: Mira J, Prieto A (eds) Bio-inspired applications of connectionism. Lecture notes in computer science, 2085. Springer, Berlin Heidelberg New York, pp 369–374

    Google Scholar 

  9. Chen J-H, Chen C-S (2002) Fuzzy kernel perceptron. IEEE Trans Neural Netw 13:1364–1373

    Article  Google Scholar 

  10. Cover TM (1965) Geometrical and statistical properties of systems of linear inequalities with application in pattern recognition. IEEE Trans Electron Comput 14:326–334

    Article  MATH  Google Scholar 

  11. Fahlman JD, Lebiere C (1987) Predicting the Mackey–Glass time series. Phys Rev Lett 59:845–847

    Article  MathSciNet  Google Scholar 

  12. Franco L, Cannas SA (2001) Generalization properties of modular networks: implementing the parity function. IEEE Trans Neural Netw 12:1306–1313

    Article  Google Scholar 

  13. Funahashi KI (1989) On the approximate realization of continuous mappings by neural networks. Neural Netw 2:183–192

    Article  Google Scholar 

  14. Fung H, Li LK (2001) Minimal feedforward parity networks using threshold gates. Neural Comput 13:319–326

    Article  MATH  Google Scholar 

  15. Georgiou GM, Koutsougeras C (1992) Complex domain backpropagation. IEEE Trans Circuits Syst CAS-II 39:330–334

    Article  MATH  Google Scholar 

  16. Gorman RP, Sejnowski TJ (1988) Analysis of hidden units in a layered network trained to classify sonar targets. Neural Netw 1:75–89

    Article  Google Scholar 

  17. Haykin S (1999) Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall, Englewood Cliff

    MATH  Google Scholar 

  18. Hecht-Nielsen R (1988) Kolmogorov mapping neural network existence theorem. In: Proceedings of the 1st IEEE international conference on neural networks, vol 3. IEEE Computer Society Press, pp 11–13

  19. Hecht-Nielsen R (1990) Neurocomputing. Addison Wesley, New York

    Google Scholar 

  20. Hirose A (ed) (2003) Complex valued neural networks. Theories and applications. World Scientific, Singapore

    MATH  Google Scholar 

  21. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward neural networks are universal approximators. Neural Netw 2:259–366

    Article  Google Scholar 

  22. Impagliazzo R, Paturi R, Saks ME (1997) Size-depth tradeoffs for threshold circuits. SIAM J Comput 26:693–707

    Article  MathSciNet  MATH  Google Scholar 

  23. Islam MM, Yao X, Murase K (2003) A constructive algorithm for training cooperative neural networks ensembles. IEEE Trans Neural Netw 14:820–834

    Article  Google Scholar 

  24. Jang J-SR (1993) ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans Syst, Man Cybern 23:665–685

    Article  Google Scholar 

  25. Jankowski S, Lozowski A, Zurada JM (1996) Complex-valued multistate neural associative memory. IEEE Trans Neural Netw 7:1491–1496

    Article  Google Scholar 

  26. Kim D, Kim C (1997) Forecasting time series with genetic fuzzy predictor ensemble. IEEE Trans Neural Netw 5:523–535

    Google Scholar 

  27. Kolmogorov AN (1957) On the representation of continuous functions of many variables by superposition of continuous functions and addition (in Russian). The Reports of the Academy of Sciences of the USSR. Doklady Akademii Nauk SSSR 114:953–956

    MathSciNet  MATH  Google Scholar 

  28. Lee S-H, Kim I (1994) Time series analysis using fuzzy learning. In: Proceedings of the international conference on neural information processing, Seoul, Korea, vol 6, pp 1577–1582

  29. Leung H, Haykin S (1991) The complex backpropagation algorithm. IEEE Trans Signal Process 39:2101–2104

    Article  Google Scholar 

  30. Mackey MC, Glass L (1977) Oscillation and chaos in physiological control systems. Science 197:287–289

    Article  Google Scholar 

  31. Mizutani E, Dreyfus SE, Jang J-SR (2000) On dynamic programming-like recursive gradient formula for alleviating hidden-node saturation in the parity problem. In: Proceedings of the international workshop on intelligent systems resolutions – the 8th Bellman continuum, Hsinchu, Taiwan, pp 100–104

  32. Mizutani E, Dreyfus SE (2002) MLP’s hidden-node saturations and insensitivity to initial weights in two classification benchmark problems: parity and two-spirals. In: Proceedings of the 2002 international joint conference on neural networks (IJCNN’02), pp 2831–2836

  33. Muezzinoglu MK, Guzelis C, Zurada JM (2003) A new design method for the complex-valued multistate hopfield associative memory. IEEE Trans Neural Netw 14:891–899

    Article  Google Scholar 

  34. Müller K-R, Mika S, Rätsch G, Tsuda K, Shölkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw 12:181–201

    Article  Google Scholar 

  35. Nitta T (1997) An extension of the backpropagation algorithm to complex numbers. Neural Netw 10:1391–1415

    Article  Google Scholar 

  36. Paul S, Kumar S (2002) Subsethood-product fuzzy neural inference system (SuPFuNIS). IEEE Trans Neural Netw 13:578–599

    Article  Google Scholar 

  37. Rumelhart DE, McClelland JL (1986) Parallel distributed processing: explorations in the microstructure of cognition. MIT Press, Cambridge

    Google Scholar 

  38. Russo M (2000) Genetic fuzzy learning. IEEE Trans Evol Comput 4:259–273

    Article  Google Scholar 

  39. Siegelman H, Sontag E (1991) Neural nets are universal computing devices. Research Report SYCON-91–08. Rutgers Center for Systems and Control. Rutgers University

  40. Vapnik V (1995) The nature of statistical learning theory. Springer, Berlin Heidelberg New York

    MATH  Google Scholar 

  41. Yao X, Liu Y (1997) A new evolutionary system for evolving artificial neural networks. IEEE Trans Neural Netw 8:694–713

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Information Sciences, Texas A&M University–Texarkana, P.O. Box 5518, 2600 N. Robison Rd., Texarkana, TX, 75505, USA

    Igor Aizenberg

  2. Department of Computer Science-1, University of Dortmund, 44221, Dortmund, Germany

    Claudio Moraga

Authors
  1. Igor Aizenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudio Moraga
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Igor Aizenberg.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aizenberg, I., Moraga, C. Multilayer Feedforward Neural Network Based on Multi-valued Neurons (MLMVN) and a Backpropagation Learning Algorithm. Soft Comput 11, 169–183 (2007). https://doi.org/10.1007/s00500-006-0075-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-006-0075-5

Keywords

Search

Navigation