Abstract
In this paper, we discuss computational architectures that are motivated by connectionist patterns that occur in biochemical networks, and speculate about how this biochemical approach to connectionism might complement conventional neural approaches. In particular, we focus on three features of biochemical networks that make them distinct from neural networks: their diverse, complex nodal processes, their emergent organisation, and the dynamical behaviours that result from higher-order, self-modifying processes. We also consider the growing use of evolutionary algorithms in the design of connectionist systems, noting how this enables us to explore a wider range of connectionist architectures, and how the close relationship between biochemical networks and biological evolution can guide us in this endeavour.
Similar content being viewed by others
References
Adleman LM (1994) Molecular computation of solutions to combinatorial problems. Sci Agric 266(5187):1021–1024
Albert R, Othmer HG (2003) The topology of the regulatory interactions predicts the expression pattern of the segment polarity genes in drosophila melanogaster. J Theor Biol 223(1):1–18
Aldana M, Balleza E, Kauffman S, Resendiz O et al (2007) Robustness and evolvability in genetic regulatory networks. J Theor Biol 245(3):433–448
Alon Uri (2007) Network motifs: theory and experimental approaches. Nat Rev Genet 8(6):450–461
Andy A, De Lacy Costello B, Tetsuya A (2005) Reaction--diffusion computers. Elsevier, Amsterdam
Banzhaf W (2003) Artificial regulatory networks and genetic programming. In: Rick LR, Bill W (eds) Genetic programming theory and practice, Chap 4. Kluwer, Amsterdam, pp 43–62
Banzhaf W (2004) Artificial chemistries—towards constructive dynamical systems. Solid State Phenomena, 97/98:43–50
Berridge MJ (2012) Cell signalling biology: spatial and temporal aspects of signalling. Portland Press, Colchester
Bird A (2007) Perceptions of epigenetics. Nature 447(7143):396–398
Bolouri H (2008) Computational modeling of gene regulatory networks—a primer. Imperial College Press, London
Bull L (2012) Evolving Boolean networks on tunable fitness landscapes. IEEE Trans Evolut Comput 16(6):817–828
Bull L (2012) A simple computational cell: coupling Boolean gene and protein networks. Artif Life 18(2):223 – 236
Bull L (2013) Consideration of mobile DNA: new forms of artificial genetic regulatory networks. Nat Comput. doi:10.1007/s11047-013-9369-6
Bull L, Preen R (2009) On dynamical genetic programming: random Boolean networks in learning classifier systems. In: Leonardo V et al (eds) Genetic programming. Lecture notes in computer science, vol 5481. Springer, Berlin, pp 37–48
Chaté H, Losson J (1997) Non-trivial collective behavior in coupled map lattices: a transfer operator perspective. Phys D 103(1-4):51–72
Cedric RC, Bradley RC (2009) The biology of chromatin remodeling complexes. Ann Rev Biochem 78:273–304
Conrad M (1990) The geometry of evolution. BioSystems 24:61–81
Croft D, O’Kelly G, Wu G et al (2011) Reactome: a database of reactions, pathways and biological processes. Nucl Acids Res 39(suppl 1):D691–D697
de Ronde W, Tostevin F, ten Wolde PR (2011) Multiplexing biochemical signals. Phys Rev Lett 107(4):048101
Decraene J, Mitchell GG, McMullin B (2007) Evolving artificial cell signaling networks: perspectives and methods. In: Dressler F, Carreras I (eds) Advances in biologically inspired information systems. Springer, Heidelberg, pp 167–186
Dittrich P, Ziegler J, Banzhaf W (2001) Artificial chemistries—a review. Artif Life 7:225–275
Dubrova E, Teslenko M, Tenhunen H (2008) A computational scheme based on random Boolean networks. In: Priami C et al (eds) Transactions on computational systems biology X. Lecture notes in computer science, vol 5410. Springer, Berlin, pp 41–58
Duch W, Grabczewski K (2002) Heterogeneous adaptive systems. In: Proceedings of the 2002 international joint conference on neural networks, IJCNN’02, vol 1. IEEE, New York, pp 524–529
Duch W, Jankowski N (1999) Survey of neural transfer functions. Neural Comput Surv 2(1):163–212
Eccles JC, Ito M, Szentágothai J (1967) The cerebellum as a neuronal machine. Springer, Berlin
Egri-Nagy A, Dini P, Nehaniv CL, Schilstra MJ (2010) Transformation semigroups as constructive dynamical spaces. In: Colugnati FAB, Lia CRL, Saulo FAB (eds) Digital ecosystems. Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 67. Springer, Berlin, pp 245–265
Farmer JD (1990) A rosetta stone for connectionism. Phys D 42(13):153–187
Faulconbridge A, Stepney S, Miller JF, Leo S, Caves D (2012) RBN-World: a sub-symbolic artificial chemistry. In: Kampis G, Karsai I, Szathmáry E (eds) Advances in artificial life. Darwin Meets von Neumann, Part 1. Lecture notes in computer science, vol 5777. Springer, Berlin, pp 377–384
Fine P, Paolo E, Philippides A (2006) Spatially constrained networks and the evolution of modular control systems. In: Nolfi S et al (eds) From animals to animats 9. Lecture notes in computer science, vol 4095. Springer, Berlin, pp 546–557
Floreano D, Mondada F et al (1996) Evolution of plastic neurocontrollers for situated agents. In: From animals to animats IV: proceedings of the fourth international conference on simulation of adaptive behavior, vol 4. MIT Press, Cambridge
Fontana W (1992) Algorithmic chemistry. In: Langton CG, Taylor C, Farmer JD, Rasmussen S (eds) Artificial life II. Addison-Wesley, Redwood City, pp 159–210
Franke R, Theis FJ, Klamt S (2010) From binary to multivalued to continuous models: the lac operon as a case study. J Integr Bioinf 7(1):151
Fuente LA, Lones MA, Turner AP, Caves LS, Stepney S, Tyrrell AM (2013a) Adaptive robotic gait control using coupled artificial signalling networks, hopf oscillators and inverse kinematics. In: Proceedings fo 2013 IEEE congress on evolutionary computation (CEC 2013). IEEE, New York
Fuente LA, Lones MA, Turner AP, Caves LS, Stepney S, Tyrrell AM (2013b) Computational models of signalling networks for non-linear control. Biosystems 112(2):122–130
Funahashi K, Nakamura Y (1993) Approximation of dynamical systems by continuous time recurrent neural networks. Neural networks, 6(6):801–806
Gerstner W, Kistler WM (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, New York
Good MC, Zalatan JG, Lim WA (2011) Scaffold proteins: hubs for controlling the flow of cellular information. Sci Signal 332(6030):680
Hamann H, Schmickl T, Crailsheim K (2011) Coupled inverted pendulums: a benchmark for evolving decentral controllers in modular robotics. In: Proceedings of the 13th annual genetic and evolutionary computation conference, GECCO. ACM, New York, pp 195–202
Hancock J (2010) Cell signalling. Oxford University Press, Oxford
Hickinbotham S, Stepney S, Nellis A, Clarke T, Clark E, Pay M, Young P (2011) Embodied genomes and metaprogramming. In: Lenaerts T et al (eds) Proceedings of 11th European conference on the synthesis and simulation of living systems. Advances in artificial life, ECAL 2011. MIT Press, Cambridge, pp 334–341
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79(8):2554–2558
Hu T, Banzhaf W (2010) Evolvability and speed of evolutionary algorithms in light of recent developments in biology. J Artif Evolut Appl 2010:568375
Husbands P, Smith T, Jakobi N, O’Shea M (1998) Better living through chemistry: evolving GasNets for robot control. Connect Sci 10(3-4):185–210
Jaeger H (2003) Adaptive nonlinear system identification with echo state networks. In: Becker S, Thrun S, Obermayer K (eds) Advances in neural information processing systems, vol 15. MIT Press, Cambridge, pp 593–600
Jordan MI (1990) Attractor dynamics and parallelism in a connectionist sequential machine. In: Diederich J (ed) Artificial neural networks. IEEE, Piscataway, pp 112–127
Joyce AR, Palsson BØ (2006) The model organism as a system: integrating ‘omics’ data sets. Nat Rev Mol Cell Biol 7(3):198–210
Kauffman SA (1969) Metabolic stability and epigenesis in randomly constructed genetic nets. J Theor Biol 22(3):437–467
Kauffman SA (1993) The origins of order: self-organization and selection in evolution. Oxford University Press, Oxford
Khan MM, Lester DR, Plana LA, Rast A, Jin X, Painkras E, Furber SB (2008) SpiNNaker: mapping neural networks onto a massively-parallel chip multiprocessor. In: IEEE international joint conference on neural networks, IJCNN 2008. IEEE, New York, pp 2849–2856
Kholodenko BN (2006) Cell-signalling dynamics in time and space. Nat Rev Mol Cell Biol 7(3):165–176
Kim S, Coulombe PA (2010) Emerging role for the cytoskeleton as an organizer and regulator of translation. Nat Rev Mol Cell Biol 11(1):75–81
Kind T, Scholz M, Fiehn O (2009) How large is the metabolome? A critical analysis of data exchange practices in chemistry. PloS One 4(5):e5440
Kirschner M, Gerhart J (1998) Evolvability. Proc Natl Acad Sci USA 95:8420–8427
Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43(1):59–69
Komili S, Silver PA (2008) Coupling and coordination in gene expression processes: a systems biology view. Nat Rev Genet 9(1):38–48
Koonin EV, Wolf YI, Karev GP, Fernández P, Solé RV (2006) The role of computation in complex regulatory networks. In: Laws P (ed) Scale-free networks and genome biology. Molecular Biology Intelligence Unit. Springer, New York, pp 206–225
Kornberg RD (2007) The molecular basis of eukaryotic transcription. Proc Natl Acad Sci USA 104(32):12955–12961
Kota SK, Feil R (2010) Epigenetic transitions in germ cell development and meiosis. Dev Cell 19(5):675 – 686
Lacroix V, Cottret L, Thebault P, Sagot M-F (2008) An introduction to metabolic networks and their structural analysis. IEEE/ACM Trans Comput Biol Bioinf 5(4):594–617
Leier A, Kuo PD, Banzhaf W (2007) Analysis of preferential network motif generation in an artificial regulatory network model created by duplication and divergence. Adv Complex Syst 10(02):155–172
Levy ED, Landry CR, Michnick SW (2010) Signaling through cooperation. Sci Signal 328(5981):983
Lones MA (2003) Enzyme genetic programming: modelling biological evolvability in genetic programming. PhD thesis, Department of Electronics, University of York
Lones MA, Tyrrell AM (2004) Modelling biological evolvability: implicit context and variation filtering in enzyme genetic programming. BioSystems 76(1):229–238
Lones MA, Tyrrell AM, Stepney S, Caves LS (2010) Controlling complex dynamics with artificial biochemical networks. In: Esparcia-Alczar AI et al (eds) Proceedings of 2010 European conference on genetic programming (EuroGP 2010). Lecture notes in computer science, vol 6021. Springer, Berlin, pp 159–170
Lones MA, Tyrrell AM, Stepney S, Caves LSD (2011) Controlling legged robots with coupled artificial biochemical networks. In: Lenaerts T et al (eds) Proceedings of 11th European conference on the synthesis and simulation of living systems. Advances in artificial life ECAL 2011. MIT Press, Cambridge, pp 465–472
Lones MA, Smith SL, Tyrrell AM, Alty JE, Jamieson DRS (2012) Evolving computational dynamical systems to recognise abnormal human motor function. In: Lones MA et al (eds) Proceedings of 9th international Conference on information processing in cells and tissues, vol 7223. Lecture notes in computer science. Springer, Berlin, pp 177–182
Lones MA, Fuente LA, Turner AP, Caves LSD, Stepney S, Smith SL, Tyrrell AM (2013a) Artificial biochemical networks: evolving dynamical systems to control dynamical systems. IEEE Trans Evolut Comput (in press)
Lones MA, Smith SL, Alty JE, Lacy SE, Possin KL, Jamieson DRS, Tyrrell AM (2013b) Evolving classifiers to recognise the movement characteristics of Parkinson’s disease patients. IEEE Trans Evolut Comput (in press)
Lones MA, Smith SL, Tyrrell AM, Alty JE, Jamieson DRS (2013c) Characterising neurological time series data using biologically-motivated networks of coupled discrete maps. BioSystems 112(2):94–101
Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671
Maass W, Natschläger T, Markram H (2002) Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput 14(11):2531–2560
Mabu S, Hirasawa K, Hu J (2007) A graph-based evolutionary algorithm: genetic network programming (GNP) and its extension using reinforcement learning. Evolut Comput 15(3):369–398
MacLeod C, Capanni NF (2010) Artificial biochemical networks: a different connectionist paradigm. Artif Intell Rev 33(1):123–134
Marijuán PC (1995) Enzymes, artificial cells and the nature of biological information. BioSystems 35:167–170
May RM (1976) Simple mathematical models with very complicated dynamics. Nature 261:459–467
McAdams HH, Arkin A (1999) It’s a noisy business! genetic regulation at the nanomolar scale. Trends Genetics 15(2):65–69
McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5(4):115–133
Medler DA (1998) A brief history of connectionism. Neural Comput Surv 1:18–72
Meister G, Tuschl T (2004) Mechanisms of gene silencing by double-stranded RNA. Nat Environ Pollut Technol 431:343–349
Minsky M, Seymour P (1969) Perceptrons: an introduction to computational geometry. MIT Press, Cambridge
Mokhtar M, Halliday DM, Tyrrell AM (2007) Hippocampus neurons and place cells/place field representation to provide path navigation. In: International joint conference on neural networks, IJCNN 2007. IEEE, New York, pp 795 –800
Natarajan M, Lin KM, Hsueh RC, Sternweis PC, Ranganathan R (2006) A global analysis of cross-talk in a mammalian cellular signalling network. Nat Cell Biol 8(6):571–580
Newton AC (1995) Protein kinase C: structure, function, and regulation. J Biol Chem 270(48):28495–28498
Ohno S et al (1970) Evolution by gene duplication. George Allen & Unwin, London
Ohyama T (2005) DNA conformation and transcription. Molecular Biology Intelligence Unit. Springer, New York
Pǎun G (2000) Computing with membranes. J Comput Syst Sci 61(1):108–143
Poole AM, Philips MJ, Penny D (2003) Prokaryote and eukaryote evolvability. BioSystems 69:163–186
Porto-Pazos AB, Veiguela N, Mesejo P, Navarrete M, Alvarellos A, Ibáñez O, Pazos A, Araque A (2011) Artificial astrocytes improve neural network performance. PloS One 6(4):e19109
Rasmussen TP et al (2003) Embryonic stem cell differentiation: a chromatin perspective. Reprod Biol Endocrinol 1:100
Rebollo R, Romanish MT, Mager DL (2012) Transposable elements: an abundant and natural source of regulatory sequences for host genes. Annu Rev Genetics 46(1):21–42
Reil T (1999) Dynamics of gene expression in an artificial genome—implications for biological and artificial ontogeny. In: Proceedings of the 5th European conference on artificial life (ECAL’99). Lecture notes in artificial intelligence, vol 1674. Springer, Berlin, pp 457–466
Ricard J, Cornish-Bowden A (2005) Co-operative and allosteric enzymes: 20 years on. Eur J Biochem 166(2):255–272
Rosenblatt F (1958) The perceptron. Psychol Rev 65(6):386–408
Rumelhart DE, McClelland JL (1986) Parallel distributed processing: explorations in the microstructure of cognition. In: Foundations vol 1. MIT Press, Cambridge
Rumelhart DE, Hintont GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536
Shmulevich I, Dougherty ER, Zhang W (2002) From Boolean to probabilistic Boolean networks as models of genetic regulatory networks. Proc IEEE 90(11):1778–1792
Stanley KO (2007) Compositional pattern producing networks: a novel abstraction of development. Genetic Program Evolv Mach 8(2):131–162
Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evolut Comput 10(2):99–127
Szejka A, Drossel B (2007) Evolution of canalizing Boolean networks. Eur Phys J B 56(4):373–380
Teuscher C (2002) Turing’s connectionism: an investigation of neural network architectures. Springer, London
Tsuda S, Artmann S, Zauner K-P (2009) The phi-bot: a robot controlled by a slime mould. In: Artificial life models in hardware. Springer, London, pp 213–232
Tuci E, Quinn M, Harvey I (2002) Evolving fixed-weight networks for learning robots. In: Proceedings of the 2002 IEEE congress on evolutionary computation, CEC’02, vol 2. IEEE, New York, pp 1970–1975
Turing AM (1948) Intelligent machinery. Technical report, National Physical Laboratory, Teddington
Turing AM (1952) The chemical basis of morphogenesis. Philos Trans R Soc Lond B 237(641):37–72
Turner AP, Lones MA, Fuente LA, Stepney S, Caves LSD, Tyrrell AM (2013a) The artificial epigenetic network. In: Proceedings of 2013 IEEE symposium series on computational intelligence (SSCI 2013). IEEE, New York
Turner AP, Lones MA, Fuente LA, Tyrrell AM, Stepney S, Caves LSD (2013b) Controlling complex tasks using artificial epigenetic regulatory networks. BioSystems 112(2):56–62
Ulam S (1952) Random processes and transformations. In: Proceedings of international congress of mathematicians, vol 2. American Mathematical Society, Providence, pp 264–275
Vargas PA, Paolo EA, Husbands P (2007) Preliminary investigations on the evolvability of a non-spatial GasNet model. In: e Costa FA et al (eds) Advances in artificial life, vol 4648. Lecture notes in computer science. Springer, Berlin, pp 966–975
Volkert LG (2003) Enhancing evolvability with mutation buffering mediated through multiple weak interactions. Biosystems 69(2-3):127–142
Walker CC, Ashby WR (1966) On temporal characteristics of behavior in certain complex systems. Kybernetik 3(2):100–108
Wellen KE, Thompson CB (2012) A two-way street: reciprocal regulation of metabolism and signalling. Nat Rev Mol Cell Biol 13(4):270–276
Wilson MZ, Gitai Z (2013) Beyond the cytoskeleton: mesoscale assemblies and their function in spatial organization. Curr Opin Microbiol 16(2):177–183
Wittmann DM, Krumsiek J, Saez-Rodriguez J, Lauffenburger DA, Klamt S, Theis FJ (2009) Transforming Boolean models to continuous models: methodology and application to T-cell receptor signaling. BMC Syst Biol 3(1):98
Woodcock CL, Ghosh RP (2010) Chromatin higher-order structure and dynamics. Cold Spring Harb Perspect Biol 2(5):a000596
Yamauchi Brian M, Beer Randall D (1994) Sequential behavior and learning in evolved dynamical neural networks. Adapt Behav 2(3):219–246
Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447
Zandron C, Ferretti C, Mauri G (2001) Solving NP-complete problems using P systems with active membranes. In: Unconventional models of computation, UMC’2K. Springer, London, pp 289–301
Zanudo JGT, Aldana M, Martínez-Mekler G (2011) Boolean threshold networks: virtues and limitations for biological modeling. Inf Process Biol Syst 2:113–151
Acknowledgements
This research was supported by the EPSRC under the Grant “Artificial Biochemical Networks: Computational Models and Architectures” (ref. EP/F060041/1).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lones, M.A., Turner, A.P., Fuente, L.A. et al. Biochemical connectionism. Nat Comput 12, 453–472 (2013). https://doi.org/10.1007/s11047-013-9400-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11047-013-9400-y