Abstract
Neural computation in Clifford algebras, which include familiar complex numbers and quaternions as special cases, has recently become an active research field. As always, neurons are the atoms of computation. The paper provides a general notion for the Hessian matrix of Clifford neurons of an arbitrary algebra. This new result on the dynamics of Clifford neurons then allows the computation of optimal learning rates. A thorough discussion of error surfaces together with simulation results for different neurons is also provided. The presented contents should give rise to very efficient second–order training methods for Clifford Multi-layer perceptrons in the future.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Arena, P., Fortuna, L., Muscato, G., Xibilia, M.G.: Concepts in User Interfaces. LNCIS, vol. 234. Springer, Heidelberg (1998)
Arena, P., Fortuna, L., Occhipinti, L., Xibilia, M.G.: Neural networks for quaternion valued function approximation. In: IEEE Int. Symp. on Circuits and Systems, London, vol. 6, pp. 307–310. IEEE Computer Society Press, Los Alamitos (1994)
Baldi, P.F., Hornik, K.: Learning in linear neural networks: A survey. IEEE Transactions on Neural Networks 6(4), 837–858 (1995)
Buchholz, S.: A theory of neural computation with Clifford algebra. Technical Report Number 0504, Universität Kiel, Institut für Informatik (2005)
Buchholz, S., Le Bihan, N.: Optimal separation of polarized signals by quaternionic neural networks. In: EUSIPCO 2006. 14th European Signal Processing Conference, Florence, Italy (September 4-8, 2006)
Hirose, A.: Complex–Valued Neural Networks. Studies in Computational Intelligence, vol. 32. Springer, Heidelberg (2006)
LeCun, Y., Bottou, L., Orr, G.B., Müller, K.–R.: Efficient BackProp. In: Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 1524, Springer, Heidelberg (1998)
LeCun, Y., Kanter, I., Solla, S.A.: Eigenvalues of Covariance Matrices: Application to Neural–Network Learning. Physical Review Letters 66(18), 2396–2399 (1991)
Leung, H., Haykin, S.: The Complex Backpropagation Algorithm. IEEE Transactions on Signal Processing 3(9), 2101–2104 (1991)
Lounesto, P.: Clifford Algebras and Spinors. Cambridge University Press, Cambridge (1997)
Matsui, N., Isokawa, T., Kusamichi, H., Peper, F., Nishimura, H.: Quaternion neural network with geometrical operators. Journal of Intelligent & Fuzzy Systems 15(3–4), 149–164 (2004)
Porteous, I.R.: Clifford Algebras and the Classical Groups. Cambridge University Press, Cambridge (1995)
Rojas, R.: Neural Networks. Springer, Heidelberg (1996)
Yaglom, I.M.: Complex Numbers in Geometry. Academic Press, New York (1968)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Buchholz, S., Tachibana, K., Hitzer, E.M.S. (2007). Optimal Learning Rates for Clifford Neurons. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds) Artificial Neural Networks – ICANN 2007. ICANN 2007. Lecture Notes in Computer Science, vol 4668. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74690-4_88
Download citation
DOI: https://doi.org/10.1007/978-3-540-74690-4_88
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74689-8
Online ISBN: 978-3-540-74690-4
eBook Packages: Computer ScienceComputer Science (R0)