Skip to main content

Advertisement

SpringerLink
Log in

Feedforward Networks: Adaptation, Feedback, and Synchrony

  • Published:
Journal of Nonlinear Science Aims and scope Submit manuscript

Abstract

We investigate the effect on synchrony of adding feedback loops and adaptation to a large class of feedforward networks. We obtain relatively complete results on synchrony for identical cell networks with additive input structure and feedback from the final to the initial layer of the network. These results extend the previous work on synchrony in feedforward networks by Aguiar et al. (Chaos 27:013103, 2017). We also describe additive and multiplicative adaptation schemes that are synchrony preserving and briefly comment on dynamical protocols for running the feedforward network that relates to unsupervised learning in neural nets and neuroscience.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. In dynamical systems theory, there are methods based on the Taken’s embedding theorem (Takens 1981) that allow reconstruction of complex dynamical systems from time-series data. However, these techniques seem not to be useful in data processing.

  2. For effective and efficient implementation of this approach, it is expedient to introduce some intra-layer inhibitory structures (for example Masquelier et al. 2009a; Lin et al. 2009).

References

  • Aguiar, M.A.D., Dias, A.: The lattice of synchrony subspaces of a coupled cell network: characterization and computation algorithm. J. Nonlinear Sci. 24(6), 949–996 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Aguiar, M.A.D., Dias, A., Ferreira, F.: Patterns of synchrony for feed-forward and auto-regulation feed-forward neural networks. Chaos 27, 013103 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Aguiar, M.A.D., Dias, A., Field, M.J.: Bifurcation, dynamics and feedback for adaptive feed-forward networks (2018) (in preparation)

  • Ashwin, P., Rodrigues, A.: Hopf normal form with \(S_N\) symmetry and reduction to systems of nonlinearly coupled phase oscillators. Phys. D: Nonlinear Phenom. 325, 14–24 (2016)

    Article  MATH  Google Scholar 

  • Ashwin, P., Swift, J.W.: The dynamics of n weakly coupled identical oscillators. J. Nonlinear Sci. 2(1), 69–108 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  • Belykh, V., Hasler, M.: Mesoscale and clusters of synchrony in networks of bursting neurons. Chaos 18, 016106 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Bick, C., Field, M.J.: Asynchronous networks and event driven dynamics. Nonlinearity 30(2), 558–594 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Bick, C., Field, M.J.: Asynchronous networks: modularization of dynamics theorem. Nonlinearity 30(2), 595–621 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Bick, C., Field, M.J.: Functional asynchronous networks: factorization of dynamics and function (MATEC Web of Conferences, 83 (2016), CSNDD 2016—International Conference on Structural Nonlinear Dynamics and Diagnosis, Marrakech, May 23–25) (2016)

  • Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)

    MATH  Google Scholar 

  • Caporale, N., Dan, Y.: Spike timing dependent plasticity: a Hebbian learning rule. Annu. Rev. Neurosci. 31, 25–36 (2008)

    Article  Google Scholar 

  • Chandra, S., Hathcock, D., Crain, K., Antonsen, T.M., Girvan, M., Ott, E.: Modeling the network dynamics of pulse-coupled neurons. Chaos 27(3), 033102 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Davey, B.A., Priestley, H.A.: Introduction to Lattices and Order. Cambridge University Press, Cambridge (1990)

    MATH  Google Scholar 

  • Ermentrout, G.B., Kopell, N.: Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM J. Appl. Math. 46, 233–253 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  • Field, M.J.: Heteroclinic networks in homogeneous and heterogeneous identical cell systems. J. Nonlinear Sci. 25(3), 779–813 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Gerstner, W., Kempter, R., van Hemmen, L.J., Wagner, H.: A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76–78 (1996)

    Article  Google Scholar 

  • Fricker, D., Miles, R.: EPSP amplification and the precision of spike timing in hippocampal neurons. Neuron 28, 559–569 (2000)

    Article  Google Scholar 

  • Goedeke, S., Diesmann, M.: The mechanism of synchronization in feed-forward neuronal networks. New J. Phys. 10, 015007 (2008)

    Article  Google Scholar 

  • Golubitsky, M., Postlethwaite, C.: Feed-forward networks, center manifolds, and forcing. Disc. Cont. Dyn. Sys. Ser. A 14(2), 2913–2935 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Golubitsky, M., Schaeffer, D.G., Stewart, I.N.: Singularities and Groups in Bifurcation Theory, Vol. II. Appl. Math. Sci. 69. Springer, New York (1988)

  • Golubitsky, M., Stewart, I.: Nonlinear dynamics of networks: the groupoid formalism. Bull. Am. Math. Soc. 43, 305–364 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Golubitsky, M., Stewart, I., Törok, A.: Patterns of synchrony in coupled cell networks with multiple arrows. SIAM J. Appl. Dyn. Syst. 4(1), 78–100 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2017)

    MATH  Google Scholar 

  • Hadley, B.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (2007)

    Google Scholar 

  • Hassoun, M.H.: Fundamentals of Artificial Neural Networks. MIT Press, Cambridge (1995)

    MATH  Google Scholar 

  • Hoppensteadt, F.C., Izhikevich, E.M.: Weakly Connected Neural Networks (Applied Mathematical Science), vol. 126. Springer, Berlin (1997)

    Book  Google Scholar 

  • Ito, J., Kaneko, K.: Self-organized hierarchical structure in a plastic network of chaotic units. Neural Netw. 13(3), 275–281 (2000)

    Article  Google Scholar 

  • Ito, J., Kaneko, K.: Spontaneous structure formation in a network of chaotic units with variable connection strengths. Phys. Rev. Lett. 88(2), 028701 (2002)

    Article  Google Scholar 

  • Kamei, H., Cock, P.J.A.: Computation of balanced equivalence relations and their lattice for a coupled cell network. SIAM J. Appl. Dyn. Syst. 12, 352–382 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Kaneko, K.: Relevance of dynamical clustering to biological networks. Phys. D 75, 55–73 (1994)

    Article  MATH  Google Scholar 

  • Kuramoto, Y.: Chemical Oscillations, Waves, and Turbulence. Springer, Berlin (1984)

    Book  MATH  Google Scholar 

  • Lee, H.L., Padmanabhan, V., Whang, S.: Information distortion in a supply chain: the Bullwhip effect. Manag. Sci. 43(4), 546–558 (1997)

    Article  MATH  Google Scholar 

  • Lin, K.K., Shea-Brown, E., Young, L.-S.: Spike-time reliability of layered neural oscillator networks. J. Comput. Neurosci. 27, 135–160 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Masquelier, T., Guyonneau, R., Thorpe, S.J.: Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PLoS ONE 3(1), e1377 (2008)

    Article  Google Scholar 

  • Masquelier, T., Guyonneau, R., Thorpe, S.J.: Competitive STDP-based spike pattern learning. Neural Comput. 21(5), 1259–1276 (2009)

    Article  MATH  Google Scholar 

  • Masquelier, T., Hugues, E., Deco, G., Thorpe, S.J.: Oscillations, phase-of-firing-coding, and spike timing-dependent plasticity: an efficient learning scheme. J. Neurosci. 29(43), 13484–13493 (2009)

    Article  Google Scholar 

  • Minsky, M., Papert, S.: Perceptrons: An Introduction to Computational Geometry. MIT Press, Cambridge (1969)

    MATH  Google Scholar 

  • Morrison, A., Diesmann, M., Gerstner, W.: Phenomenological models of synaptic plasticity based on spike timing. Biol. Cybern. 98(6), 459–478 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Panaggio, M.J., Abrams, D.M.: Chimera states: coexistence of coherence and incoherence in networks of coupled oscillators. Nonlinearity 28(3), R67 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Pedantic of Purley: Uncircling the Circle: Part I (On line article https://www.londonreconnections.com/2013/uncircling-circle-part-1/, in London Reconnections, October 3, 2013.)

  • Rink, B., Sanders, J.: Amplified Hopf bifurcations in feed-forward networks. SIAM J. Appl. Dyn. Syst. 12(2), 1135–1157 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386–408 (1958)

    Article  Google Scholar 

  • Sailamul, P., Jang, J., Paik, S.-B.: Synaptic convergence regulates synchronization-dependent spike transfer in feedforward neural networks. J. Comput. Neurosci. 43, 189–202 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  • Stewart, I.: The lattice of balanced equivalence relations of a coupled cell network. Math. Proc. Camb. Philos. Soc. 143(1), 165–183 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Stewart, I., Golubitsky, M., Pivato, M.: Symmetry groupoids and patterns of synchrony in coupled cell networks. SIAM J. Appl. Dyn. Syst. 2(4), 609–646 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Takens, F.: Detecting Strange Attractors in Turbulence (Lecture Notes in Mathematics 898). Springer, Berlin (1981)

    MATH  Google Scholar 

Download references

Acknowledgements

The authors acknowledge and appreciate the detailed comments of the referees which have helped improve the exposition and readability of the paper. The authors were partially supported by CMUP (UID/MAT/00144/2013), which is funded by FCT (Portugal) with national (MEC) and European structural funds through the programs FEDER, under the partnership agreement PT2020.

Author information

Author notes

  1. Michael Field

    Present address: Department of Mathematics, Rice University MS-136, 6100 Main St., Houston, TX, 77005, USA

Authors and Affiliations

  1. Faculdade de Economia, Centro de Matemática, Universidade do Porto, Rua Dr Roberto Frias, 4200-464, Porto, Portugal

    Manuela A. D. Aguiar

  2. Departamento de Matemática, Centro de Matemática, Universidade do Porto, Rua do Campo Alegre, 687, 4169-007, Porto, Portugal

    Ana Dias

  3. Department of Mathematics, Imperial College, South Kensington Campus, London, SW7 2AZ, UK

    Michael Field

Authors
  1. Manuela A. D. Aguiar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ana Dias
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Field
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana Dias.

Additional information

Communicated by Paul Newton.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aguiar, M.A.D., Dias, A. & Field, M. Feedforward Networks: Adaptation, Feedback, and Synchrony. J Nonlinear Sci 29, 1129–1164 (2019). https://doi.org/10.1007/s00332-018-9513-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00332-018-9513-7

Keywords

Mathematics Subject Classification

Search

Navigation