Abstract
An agent acting in an environment aims to minimise uncertainties so that being attacked can be predicted, and rewards are not only found by chance. These events define an error signal which can be used to improve performance. In this paper we present a new algorithm where an error signal from a reflex trains a novel deep network: the error is propagated forwards through the network from its input to its output, in order to generate pro-active actions. We demonstrate the algorithm in two scenarios: a 1st-person shooter game and a driving car scenario, where in both cases the network develops strategies to become pro-active.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bennett, M.: The concept of long term potentiation of transmission at synapses. Prog. Neuriobiol. 60, 109–137 (2000)
Canolty, R.T., Knight, R.T.: The functional role of cross-frequency coupling. Trends Cogn. Sci. 14(11), 506–515 (2010). http://www.ncbi.nlm.nih.gov/pubmed/20932795
Grüsser, O.: Interaction of efferent and afferent signals in visual perception. A history of ideas and experimental paradigms. Acta Psychol. 63, 3–21 (1986)
Lillicrap, T.P., Cownden, D., Tweed, D.B., Akerman, C.J.: Random synaptic feedback weights support error backpropagation for deep learning. Nat. Commun. 7, 13276 (2016). http://www.ncbi.nlm.nih.gov/pubmed/27824044
Lindsay, G.W., Rigotti, M., Warden, M.R., Miller, E.K., Fusi, S.: Hebbian learning in a random network captures selectivity properties of the prefrontal cortex. J. Neurosci. Off. J, Soc. Neurosci. 37(45), 11021–11036 (2017). http://www.ncbi.nlm.nih.gov/pubmed/28986463
Malenka, R.C., Nicoll, R.A.: Long-term potentiation – a decade of progress? Science 285, 1870–1874 (1999)
Meunier, C.N.J., Chameau, P., Fossier, P.M.: Modulation of synaptic plasticity in the cortex needs to understand all the players. Front. Synaptic Neurosci. 9, 2 (2017). http://www.ncbi.nlm.nih.gov/pubmed/28203201
Mulkey, R.M., Malenka, R.C.: Mechanisms underlying induction of homosynaptic long-term depression in area ca1 of the hippocampus. Neuron 9(5), 967–975 (1992). http://www.ncbi.nlm.nih.gov/pubmed/1419003
Phillips, C.L.: Feedback Control Systems. Prentice-Hall International, London (2000)
Porr, B., von Ferber, C., Wörgötter, F.: ISO-learning approximates a solution to the inverse-controller problem in an unsupervised behavioural paradigm. Neural Comput. 15, 865–884 (2003)
Porr, B., Wörgötter, F.: Isotropic sequence order learning. Neural Comput. 15, 831–864 (2003)
Porr, B., Wörgötter, F.: What means embodiment for radical constructivists? Kybernetes, pp. 105–117 (2005)
Roelfsema, P.R., Holtmaat, A.: Control of synaptic plasticity in deep cortical networks. Nat. Rev. Neurosci. 19(3), 166–180 (2018). http://www.ncbi.nlm.nih.gov/pubmed/29449713
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. Bradford Books, MIT Press, Cambridge (1998)
Tejomurtula, S., Kak, S.: Inverse kinematics in robotics using neural networks. Inf. Sci. 116, 147–164 (1999)
von Uexküll, B.J.J.: Theoretical Biology. Kegan Paul, Trubner (1926)
Verschure, P., Coolen, A.: Adaptive fields: distributed representations of classically conditioned associations. Network 2, 189–206 (1991)
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992)
Wörgötter, F., Porr, B.: Temporal sequence learning, prediction and control - a review of different models and their relation to biological mechanisms. Neural Comput. 17, 245–319 (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Porr, B., Miller, P. (2018). Deep Feedback Learning. In: Manoonpong, P., Larsen, J., Xiong, X., Hallam, J., Triesch, J. (eds) From Animals to Animats 15. SAB 2018. Lecture Notes in Computer Science(), vol 10994. Springer, Cham. https://doi.org/10.1007/978-3-319-97628-0_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-97628-0_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97627-3
Online ISBN: 978-3-319-97628-0
eBook Packages: Computer ScienceComputer Science (R0)