Abstract
It is commonly known that Hopfield Networks suffer from spurious states and from low storage capacity. To eliminate the spurious states Bistable Gradient Networks (BGN) introduce neurons with bistable behavior. The weights in BGN are calculated in analogy to those of Hopfield Networks, associated with Hebbian learning. Unfortunately, those networks still suffer from small storage capacity, resulting in high reconstruction errors when used to reconstruct noisy patterns. This paper proposes a new type of neural network consisting of neurons with a sigmoid hyperbolic tangent transfer function and a direct feedback. The feedback renders the neuron bistable. Furthermore, instead of using Hebbian learning which has some drawbacks when applied to overlapped patterns, we use the first order Contrastive Divergence (CD1) learning rule. We call these Networks Bistable Sigmoid Networks (BSN). When recalling patterns from the MNIST database the reconstruction error is zero even for high load providing no noise is applied. For an increasing noise level or an increasing amount of patterns the error rises only moderate.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Carreira-Perpiñán, M.A., Hinton, G.E.: On contrastive divergence learning. In: Cowell, R., Ghahramani, Z. (eds.) Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, 6-8 January 2005, Savannah Hotel, Barbados, pp. 33–40. The Society for Artificial Intelligence and Statistics (2005). ISBN 0-9727358-1-X
Chinarov, V., Menzinger, M.: Bistable gradient neural networks: their computational properties. In: Mira, J., Prieto, A. (eds.) IWANN 2001. LNCS, vol. 2084, pp. 333–338. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45720-8_38
Fischer, J., Lackner, S.: About learning in recurrent bistable gradient networks. CoRR abs/1608.08265 (2016), https://arxiv.org/abs/1608.08265
Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci. U.S.A. 79(8), 2554–2558 (1982). https://doi.org/10.1073/pnas.79.8.2554
McGraw, P.N., Menzinger, M.: Bistable gradient networks. I. Attractors and pattern retrieval at low loading in the thermodynamic limit. Phys. Rev. E Stat. Nonlinear Soft Matter Phys. 67(2), 16118 (2003). https://doi.org/10.1103/PhysRevE.67.016118
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Uschakow, S., Fischer, J., Ihme, T. (2019). Bistable Sigmoid Networks. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2019. Lecture Notes in Computer Science(), vol 11507. Springer, Cham. https://doi.org/10.1007/978-3-030-20518-8_49
Download citation
DOI: https://doi.org/10.1007/978-3-030-20518-8_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20517-1
Online ISBN: 978-3-030-20518-8
eBook Packages: Computer ScienceComputer Science (R0)