Skip to main content
SpringerLink
Log in

Self-assembly of neural networks viewed as swarm intelligence

  • Published:
Swarm Intelligence Aims and scope Submit manuscript

Abstract

While self-assembly is a fairly active area of research in swarm intelligence, relatively little attention has been paid to the issues surrounding the construction of network structures. In this paper we extend methods developed previously for controlling collective movements of agent teams to serve as the basis for self-assembly or “growth” of networks, using neural networks as a concrete application to evaluate our approach. Our central innovation is having network connections arise as persistent “trails” left behind moving agents, trails that are reminiscent of pheromone deposits made by agents in ant colony optimization models. The resulting network connections are thus essentially a record of agent movements. We demonstrate our model’s effectiveness by using it to produce two large networks that support subsequent learning of topographic and feature maps. Improvements produced by the incorporation of collective movements are also examined through computational experiments. These results indicate that methods for directing collective movements can be adopted to facilitate network self-assembly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Arbuckle, D., & Requicha, A. (2004). Active self-assembly. In Proceedings of the IEEE international conference on robotics and automation (ICRA’04) (pp. 896–901). New York: IEEE.

    Google Scholar 

  • Astor, J. C., & Adami, C. (2000). A developmental model for the evolution of artificial neural networks. Artificial Life, 6, 189–218.

    Article  Google Scholar 

  • Bishop, J., Burden, S., Klavins, E., et al. (2005). Programmable parts: A demonstration of the grammatical approach to self-organization. In Proceedings of the IEEE international conference on intelligent robots and systems (IROS’05) (pp. 3684–3691). New York: IEEE.

    Google Scholar 

  • Bonabeau, E., Dorigo, M., & Theraulaz, G. (1999). Swarm intelligence: from natural to artificial systems. London: Oxford University Press.

    MATH  Google Scholar 

  • Cangelosi, A., Parisi, D., & Nolfi, S. (1994). Cell division and migration in a “genotype” for neural networks. Network: Computation in Neural Systems, 5, 497–515.

    Article  MATH  Google Scholar 

  • Chval, J. (2002). Evolving artificial neural networks by means of evolutionary algorithms with L-systems based encoding (Research Report). Prague, Czech Republic: Czech Technical University.

  • Delgado, A. (2000). Control of nonlinear systems using a self-organizing neural network. Neural Computing & Applications, 9(2), 113–123.

    Article  Google Scholar 

  • Deneubourg, J.-L., Goss, S., Franks, N., & Pasteels, J.-M. (1989). The blind leading the blind: modelling chemically mediated army ant raid patterns. Journal of Insect Behavior, 2, 719–725.

    Article  Google Scholar 

  • Deneubourg, J.-L., Aron, S., Goss, S., & Pasteels, J.-M. (1990). The self-organizing exploratory pattern of the Argentine ant. Journal of Insect Behavior, 3, 159–168.

    Article  Google Scholar 

  • Eggenberger, P. (1997). Creation of neural networks based on developmental and evolutionary principles. In W. Gerstner, A. Germond, M. Hasler, & J. Nicoud (Eds.), Proceedings of the international conference on artificial neural networks (ICANN’97) (pp. 337–342). Berlin/Heidelberg: Springer.

    Google Scholar 

  • Elizondo, E., Birkenhead, R., Góngora, M., et al. (2007). Analysis and test of efficient methods for building recursive deterministic perceptron neural networks. Neural Networks, 20, 1095–1108.

    Article  Google Scholar 

  • Fahlman, S., & Lebiere, C. (1990). The cascade-correlation learning architecture. In D. S. Touretzky (Ed.), Advances in neural information processing systems II (pp. 524–532). San Mateo: Morgan Kaufmann.

    Google Scholar 

  • Farkaš, I., & Miikkulainen, R. (1999). Modeling the self-organization of directional selectivity in the primary visual cortex. In D. Willshaw, & A. Murray (Eds.), Proceedings of the international conference on artificial neural networks (ICANN’99) (pp. 251–256). London: IEE.

    Chapter  Google Scholar 

  • Fleischer, K. (1995). A multiple-mechanism developmental model for defining self-organizing geometric structures (Dissertation). Pasadena, CA: California Institute of Technology.

  • Fleischer, K., & Barr, A. (1994). A simulation testbed for the study of multicellular development: the multiple mechanisms of morphogenesis. In C. G. Langton (Ed.), SFI studies in the science of complexity: Vol. XVII. Artificial life III (pp. 389–416). Reading: Addison-Wesley.

    Google Scholar 

  • Franks, N., Gomez, N., Goss, S., & Deneubourg, J.-L. (1991). The blind leading the blind in army ant raid patterns: testing a model of self-organization (Hymenoptera: Formicidae). Journal of Insect Behavior, 4, 583–607.

    Article  Google Scholar 

  • Frean, M. (1990). The upstart algorithm: a method for constructing and training feedforward neural networks. Neural Computation, 2, 198–209.

    Article  Google Scholar 

  • Goodhill, G., & Xu, J. (2005). The development of retinotectal maps: a review of models based on molecular gradients. Network, 16, 5–34.

    Article  Google Scholar 

  • Goodhill, G., Gu, M., & Urbach, J. (2004). Predicting axonal response to molecular gradients with a computational model of filopodial dynamics. Neural Computation, 16, 2221–2243.

    Article  MATH  Google Scholar 

  • Gracias, D., Tien, J., Breen, T., Hsu, C., & Whitesides, G. (2000). Forming electrical networks in three dimensions by self-assembly. Science, 289, 1170–1172.

    Article  Google Scholar 

  • Gross, R., Bonani, M., Mondala, F., & Dorigo, M. (2006). Autonomous self-assembly in swarm-bots. IEEE Transactions on Robotics, 22, 1115–1130.

    Article  Google Scholar 

  • Grove, E., & Fukuchi-Shimogori, T. (2003). Generating the cerebral cortical area map. Annual Review of Neuroscience, 26, 355–380.

    Article  Google Scholar 

  • Gruau, F. (1993). Genetic synthesis of modular neural networks. In S. Forest (Ed.), Proceedings of the 5th international conference on genetic algorithms (ICGA’93) (pp. 318–325). San Mateo: Morgan Kaufmann.

    Google Scholar 

  • Grushin, A., & Reggia, J. (2006). Stigmergic self-assembly of prespecified artificial structures in a constrained and continuous environment. Integrated Computer-Aided Engineering, 13, 289–312.

    Google Scholar 

  • Grushin, A., & Reggia, J. (2008). Automated design of distributed control rules for the self-assembly of pre-specified artificial structures. Robotics and Autonomous Systems, 56, 334–359.

    Article  Google Scholar 

  • Haessly, A., Sirosh, J., & Miikkulainen, R. (1995). A model of visually guided p lasticity of the auditory spatial map in the barn owl. In J.F. Lehman & J.D. Moore (Eds.), Proceedings of the 17th annual meeting of the cognitive science society (pp. 154–158). Hillsdale: Erlbaum.

    Google Scholar 

  • Haykin, S. (1999). Neural networks: a comprehensive foundation (2nd edn). New York: Prentice-Hall.

    MATH  Google Scholar 

  • Hentschel, H., & van Ooyen, A. (1999). Models of axon guidance and bundling during development. Proceedings of the Royal Society (London) B, 266, 2231–2238.

    Article  Google Scholar 

  • Honda, H. (2003). Competition between retinal ganglion axons for targets under the servomechanism model. Journal of Neuroscience, 23(1999), 10368–10377.

    Google Scholar 

  • Jones, C., & Matarić, M. (2003). From local to global behavior in intelligent self-assembly. In Proceedings of the IEEE international conference on robotics and automation (ICRA’03) (pp. 721–726). New York: IEEE.

    Google Scholar 

  • Jung, J., & Reggia, J. (2006). Evolutionary design of neural network architectures using a descriptive encoding language. IEEE Transactions on Evolutionary Computation, 10, 676–688.

    Article  Google Scholar 

  • Kalay, A., Parnas, H., & Shamir, E. (1995). Neuronal growth via hybrid system of self-growing and diffusion based grammar rules: I. Bulletin of Mathematical Biology, 57, 205–227.

    MATH  Google Scholar 

  • Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In Proceedings of the IEEE international conference on neural networks (pp. 1942–1948). New York: IEEE.

    Chapter  Google Scholar 

  • Kitano, H. (1990). Designing neural networks using genetic algorithms with graph generation system. Complex Systems, 4, 461–476.

    MATH  Google Scholar 

  • Klavins, E. (2007). Programmable self-assembly. IEEE Control Systems Magazine, 27, 43–56.

    Article  MathSciNet  Google Scholar 

  • Klavins, E., Ghrist, R., & Lipsky, D. (2004). Graph grammars for self-assembling robotic systems. In Proceedings of the IEEE international conference on robotics and automation (ICRA’04) (pp. 5293–5300). New York: IEEE.

    Google Scholar 

  • Kohonen, T. (2001). Self-organizing maps. New York: Springer.

    MATH  Google Scholar 

  • LeCun, Y., Denker, J., & Solla, S. (1990). Optimal brain damage. In D. Touretzky (Ed.), Advances in neural information processing systems II (pp. 598–605). San Mateo: Morgan Kaufmann.

    Google Scholar 

  • Lendasse, A., Verleysen, M., de Bodt, E., Gregoire, P., & Cottrell, M. (1998). Forecasting time-series by Kohonen classification. In M. Verleysen (Ed.), Proceedings of the 6th European symposium on artificial neural networks (ESANN’98) (pp. 221–226). Brussels: D-Facto public.

    Google Scholar 

  • Lopez-Bendito, G., & Molnar, Z. (2003). Thalamocortical development: how are we going to get there? Nature Reviews Neuroscience, 4, 276–289.

    Article  Google Scholar 

  • von der Malsburg, C. (1973). Self-organization of orientation sensitive cells in the striate cortex. Kybernetik, 14, 85–100.

    Article  Google Scholar 

  • Marchand, M., Golea, M., & Rujan, P. (1990). A convergence theorem for sequential learning in two-layer perceptrons. Europhysics Letters, 11(6), 487–492.

    Article  Google Scholar 

  • Nembrini, J., Reeves, N., Poncet, E., et al. (2005). Flying swarm intelligence for architectural research. In Proceedings of the IEEE swarm intelligence symposium (SIS’05) (pp. 225–232). New York: IEEE.

    Chapter  Google Scholar 

  • van Ooyen, A. (1994). Activity-dependent neural network development. Network: Computation in Neural Systems, 5, 401–423.

    Article  MATH  Google Scholar 

  • Pearson, J., Finkel, L., & Edelman, G. (1987). Plasticity in the organization of adult cerebral cortical maps: a computer simulation, based on neuronal group, selection. The Journal of Neuroscience, 7, 4209–4223.

    Google Scholar 

  • Prusinkiewicz, P., & Lindenmayer, A. (1990). The algorithmic beauty of plants. New York: Springer.

    MATH  Google Scholar 

  • Pulakka, K., & Kujanpa, V. (1998). Rough level path planning method for a robot using SOFM neural network. Robotica, 16, 415–423.

    Article  Google Scholar 

  • Reggia, J., & Martin, C. (2009). Self-assembly of a neural network. College Park, Univ. of Maryland, Dept. of Computer Science. http://www.cs.umd.edu/~reggia/martin.html.

  • Reynolds, C. (1987). Flocks, herds, and schools: a distributed behavioral model. Computer Graphics, 21(4), 25–34.

    Article  MathSciNet  Google Scholar 

  • Ritter, H., Martinetz, T., & Schulten, K. (1992). Neural computation and self-organizing maps. Reading: Addison-Wesley.

    MATH  Google Scholar 

  • Rodriguez, A., & Reggia, J. (2004). Extending self-organizing particle systems to problem solving. Artificial Life, 10, 379–395.

    Article  Google Scholar 

  • Rust, A., Adams, R., Schilstra, M., & Bolouri, H. (2003). Evolving computational neural systems using synthetic developmental mechanisms. In S. Kumar & P. Bentley (Eds.), On growth, form and computers (pp. 353–376). San Diego: Academic Press.

    Chapter  Google Scholar 

  • Spitzer, N. (2006). Electrical activity in early neuronal development. Nature, 444, 707–712.

    Article  Google Scholar 

  • Sutton, G., Reggia, J., Armentrout, S., & D’Autrechy, C. (1994). Cortical map reorganization as a competitive process. Neural Computation, 6, 1–13.

    Article  Google Scholar 

  • Vesanto, J. (1999). SOM-based data visualization methods. Intelligent Data Analysis, 3, 111–126.

    Article  MATH  Google Scholar 

  • Vesanto, J., & Alhoniemi, E. (2000). Clustering of the self-organizing map. IEEE Transactions on Neural Networks, 11(3), 586–600.

    Article  Google Scholar 

  • Werfel, J., & Nagpag, R. (2006). Extended stigmergy in collective construction. IEEE Intelligent Systems, 21, 20–28.

    Article  Google Scholar 

  • White, P., Zykov, V., Bongard, J., & Lipson, H. (2005). Three dimensional stochastic reconfiguration of modular robots. In Proceedings of robotics: science and systems (pp. 161–168). Cambridge: MIT Press.

    Google Scholar 

  • Whitesides, G., & Gzybowski, B. (2002). Self-assembly at all scales. Science, 295, 2418–2421.

    Article  Google Scholar 

  • Yates, P., Holub, A., McLaughlin, T., et al. (2004). Computational modeling of retinotopic map development to define contributions of EphA-EphrinA gradients, axon–axon interactions, and patterned activity. Journal of Neurobiology, 59, 95–113.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Maryland, College Park, MD, 20742, USA

    Charles E. Martin

  2. Department of Computer Science, University of Maryland, College Park, MD, 20742, USA

    James A. Reggia

Authors
  1. Charles E. Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James A. Reggia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles E. Martin.

Additional information

Supported in part by NSF Awards ITS-0325089 and DMS-0240049.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Cite this article

Martin, C.E., Reggia, J.A. Self-assembly of neural networks viewed as swarm intelligence. Swarm Intell 4, 1–36 (2010). https://doi.org/10.1007/s11721-009-0035-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11721-009-0035-7

Keywords

Search

Navigation