skip to main content
10.1145/3372422.3372443acmotherconferencesArticle/Chapter ViewAbstractPublication PagesciisConference Proceedingsconference-collections
research-article

UBAL: a Universal Bidirectional Activation-based Learning Rule for Neural Networks

Published: 07 February 2020 Publication History

Abstract

Artificial neural networks, in particular the deep end-to-end architectures trained by error backpropagation (BP), are currently the topmost used learning systems. However, learning in such systems is only loosely inspired by the actual neural mechanisms. Algorithms based on local activation differences were designed as a biologically plausible alternative to BP. We propose Universal Bidirectional Activation-based Learning, a novel neural model derived from contrastive Hebbian learning. Similarly to what is assumed about learning in the brain, our model defines a single learning rule that can perform multiple ways of learning via special hyperparameters. Unlike others, our model consists of mutually dependent, yet separate weight matrices for different directions of activation propagation. We show that UBAL can learn different tasks (such as pattern retrieval, denoising, or classification) with different setups of the learning hyperparameters. We also demonstrate the performance of our algorithm on a machine learning benchmark (MNIST). The experimental results presented in this paper confirm that UBAL is comparable with a basic version BP-trained multilayer network and the related biologically-motivated models.

References

[1]
Yoshua Bengio, Dong-Hyun Lee, Jorg Bornschein, Thomas Mesnard, and Zhouhan Lin. 2015. Towards biologically plausible deep learning. arXiv preprint arXiv:1502.04156 (2015).
[2]
Sylvain Chartier, Gyslain Giguere, and Dominic Langlois. 2009. A new bidirectional heteroassociative memory encompassing correlational, competitive and topological properties. Neural Networks 22, 5-6 (2009), 568--578.
[3]
Francis Crick. 1989. The recent excitement about neural networks. Nature 337, 6203 (1989), 129--132.
[4]
Peter Csiba and Igor Farkaš. 2015. Computational analysis of the Bidirectional Activation-based Learning in autoencoder task. In 2015 International Joint Conference on Neural Networks (IJCNN). IEEE, 1--6.
[5]
Igor Farkaš and Kristina Rebrová. 2013. Bidirectional activation-based neural network learning algorithm. In Proceedings of the International Conference on Artificial Neural Networks (ICANN), Sofia, Bulgaria.
[6]
Stephen Grossberg. 1987. Competitive learning: From interactive activation to adaptive resonance. Cognitive Science 11, 1 (1987), 23--63.
[7]
Geoffrey E Hinton. 1989. Deterministic Boltzmann learning performs steepest descent in weight-space. Neural Computation 1, 1 (1989), 143--150.
[8]
Geoffrey E Hinton and James L McClelland. 1988. Learning representations by recirculation. In Neural Information Processing Systems. 358--366.
[9]
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradientbased learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.
[10]
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. 2015. Difference target propagation. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 498--515.
[11]
Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. 2016. Random synaptic feedback weights support error backpropagation for deep learning. Nature Communications 7 (2016), 13276.
[12]
Kristina Malinovská, L'udovít Malinovský, and Igor Farkaš. 2018. Towards More Biologically Plausible Error-Driven Learning for Artificial Neural Networks. In International Conference on Artificial Neural Networks. Springer, 228--231.
[13]
Randall C O'Reilly. 1996. Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neural Computation 8, 5 (1996), 895--938.
[14]
Alexander G Ororbia and Ankur Mali. 2018. Biologically Motivated Algorithms for Propagating Local Target Representations. arXiv preprint arXiv:1805.11703 (2018).
[15]
Kristina Rebrová, Matej Pecháč, and Igor Farkaš. 2013. Towards a robotic model of the mirror neuron system. In Proceedings of the 3rd joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. Osaka, Japan.
[16]
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1986. Learning internal representations by error propagation. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Number 1. The MIT Press, Cambridge, MA, 318--362.
[17]
Benjamin Scellier and Yoshua Bengio. 2017. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in Computational Neuroscience 11 (2017), 24.
[18]
Xiaohui Xie and Sebastian H Seung. 2003. Equivalence of backpropagation and contrastive Hebbian learning in a layered network. Neural Computation 15, 2 (2003), 441--454.

Cited By

View all
  • (2024)Robotic Model of the Mirror Neuron System: A RevivalArtificial Neural Networks and Machine Learning – ICANN 202410.1007/978-3-031-72359-9_23(313-323)Online publication date: 18-Sep-2024
  • (2022)A connectionist model of associating proprioceptive and tactile modalities in a humanoid robot2022 IEEE International Conference on Development and Learning (ICDL)10.1109/ICDL53763.2022.9962195(336-342)Online publication date: 12-Sep-2022
  • (2021)Generative Properties of Universal Bidirectional Activation-Based LearningArtificial Neural Networks and Machine Learning – ICANN 202110.1007/978-3-030-86365-4_7(80-83)Online publication date: 14-Sep-2021

Index Terms

  1. UBAL: a Universal Bidirectional Activation-based Learning Rule for Neural Networks

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    CIIS '19: Proceedings of the 2019 2nd International Conference on Computational Intelligence and Intelligent Systems
    November 2019
    200 pages
    ISBN:9781450372596
    DOI:10.1145/3372422
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    In-Cooperation

    • Queensland University of Technology
    • City University of Hong Kong: City University of Hong Kong

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 February 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. activation propagation
    2. biological plausibility
    3. error-driven learning
    4. local learning rule

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    CIIS 2019

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)4
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Robotic Model of the Mirror Neuron System: A RevivalArtificial Neural Networks and Machine Learning – ICANN 202410.1007/978-3-031-72359-9_23(313-323)Online publication date: 18-Sep-2024
    • (2022)A connectionist model of associating proprioceptive and tactile modalities in a humanoid robot2022 IEEE International Conference on Development and Learning (ICDL)10.1109/ICDL53763.2022.9962195(336-342)Online publication date: 12-Sep-2022
    • (2021)Generative Properties of Universal Bidirectional Activation-Based LearningArtificial Neural Networks and Machine Learning – ICANN 202110.1007/978-3-030-86365-4_7(80-83)Online publication date: 14-Sep-2021

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media