Abstract
There is a need for methods and tools that facilitate the systematic exploration of novel artificial neural network models. While significant progress has been made in developing concise artificial neural networks that implement basic models of neural activation, connectivity and plasticity, limited success has been attained in creating neural networks that integrate multiple diverse models to produce highly complex neural systems. From a problem-solving perspective, there is a need for effective methods for combining different neural-network-based learning systems in order to solve complex problems. Different models may be more appropriate for solving different subproblems, and robust, systematic methods for combining those models may lead to more powerful machine learning systems. From a neuroscience modelling perspective, there is a need for effective methods for integrating different models to produce more robust models of the brain. These needs may be met through the development of meta-model languages that represent diverse neural models and the interactions between different neural elements. A meta-model language based on attribute grammars, the Network Generating Attribute Grammar Encoding, is presented, and its capability for facilitating automated search of complex combinations of neural components from different models is discussed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Parks, R.W., Levine, D.S., Long, D.L.: Fundamentals of Neural Network Modeling: Neuropsychology and Cognitive Neuroscience. The MIT Press, Cambridge (1999)
http://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators
Demuth, H., Beale, M., Hagan, M.: Neural Network Toolbox: User’s Guide, Version 6. The Mathworks, Inc. (2010)
Aisa, B., Mingus, B., O’Reilly, R.: The emergent neural modeling system. Neural Networks 21, 1045–1212 (2008)
NeuroSolutions. v 6.0 by NeuroDimension, Inc., http://www.neurosolutions.com/products/ns/
Fischer, I., Hennecke, F., Bannes, C., Zell, A.: JavaNNS Java Neural Network Simulator User Manual Version 1.1. University of Tubingen Wilhelm-Schickard-Institute for Computer Science. Department of Computer Architecture, University of Tubingen Wilhelm-Schickard-Institute for Computer Science (2001)
Simbrain, Y.J.: A visual framework for neural network analysis and education. In: Lorenz, S., Egelhaaf, M. (eds.) Interactive Educational Media for the Neural and Cognitive Sciences; Brains, Minds & Media 3, bmm1411(2008)
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: An update. SIGKDD Explorations 11 (1) (2009)
Beeman, D., Wang, Z., Edwards, M., Bhalla, U., Cornelis, H., Bower, J.: The GENESIS 3.0 Project: a universal graphical user interface and database for research, collaboration, and education in computational neuroscience. BMC Neuroscience 8(Suppl. 2), P4 (2007)
Carnevale, N.T., Hines, M.L.: The NEURON Book. Cambridge University Press, Cambridge (2006)
Dudani, N., Ray, S., George, S., Bhalla, U.S.: Multiscale modeling and interoperability in MOOSE. BMC Neuroscience 10(Suppl. 1), P54 (2010)
Kock, G., Serbedzija, N.B.: Object-oriented and functional concepts in artificial neural network modelling. In: Intl. Joint Conference on Neural Networks, vol. 1, pp. 923–927 (1993)
Hopp, H., Prechelt, L.: CuPit-2 - A Parallel Language for Neural Algorithms: Language Reference and Tutorial. Technical Report 4/97, Institut fur Programmstrukturen und Datenorganisation. Karlsruhe, Germany (1997)
Strey, A.: EpsiloNN - A Tool for the Abstract Specification and Parallel Simulation of Neural Networks. Systems Analysis - Modelling - Simulation (SAMS) 34(4) (1999)
Linden, A., Tietz, C.: Combining multiple neural network paradigms and applications using SESAME. In: Intl. Joint Conference on Neural Networks, vol. 2, pp. 528–533 (1992)
Rubtsov, D., Butakov, S.: Application of XML for neural network exchange. Computer Standards and Interfaces,24(4), p. 311-322 (2002)
Gorchetchnikov and INCF Multiscale Modeling Taskforce. NineML – a description language for spiking neuron network modeling: the user layer. BMC Neuroscience 11(Suppl. 1), P71 (2010)
Raikov and INCF Multiscale Modeling Taskforce. NineML – a description language for spiking neuron network modeling: the abstraction layer. BMC Neuroscience, 11(Suppl. 1) P66 (2010)
Data Mining Group. PMML version 4.0 (2009), http://www.dmg.org/pmml-v4-0.html
Miller, G.F., Todd, P.M., Hegde, S.U.: Designing neural networks using genetic algorithms. In: Third Intl Conf. on Genetic Algorithms and Their Applications, pp. 379–384. Morgan Kaufmann, San Francisco (1989)
Polani, D., Uthmann, T.: Adaptation of Kohonen feature map topologies by genetic algorithms. In: Männer, R., Manderick, B. (eds.) Parallel Problem Solving from Nature 2, pp. 421–429. Elsevier, Amsterdam (1992)
Schaffer, J.D., Caruana, R.A., Eshelman, L.J.: Using genetic search to exploit the emergent behavior of neural networks. Physica D 42, 244–248 (1990)
Jacob, C.: Genetic L-system programming. In: Davidor, Y., Männer, R., Schwefel, H.-P. (eds.) PPSN 1994. LNCS, vol. 866, pp. 334–343. Springer, Heidelberg (1994)
Kitano, H.: Designing neural networks using genetic algorithms with graph generation system. Complex Systems 4, 461–476 (1990)
Gruau, F.: Automatic definition of modular neural networks. Adaptive Behavior 3(2), 151–183 (1995)
Bengio, S., Bengio, Y., Cloutier, J.: Use of genetic programming for the search of a new learning rule for neural networks. In: First Conference on Evolutionary Computation, pp. 324–327 (1994)
Tsoulosa, I., Gavrilis, D., Glavas, E.: Neural network construction and training using grammatical evolution. Neurocomputing 72, 269–277 (2008)
Mouret, J., Doncieux, S.: MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars. Evolutionary Intelligence 1, 187–207 (2008)
Hussain, T.S., Browse, R.A.: Network generating attribute grammar encoding. In: IEEE International Joint Conference on Neural Network, pp. 431–436 (1998)
Hussain, T.S., Browse, R.A.: Evolving neural networks using attribute grammars. In: 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks, pp. 37-42 (2000)
Hussain, T.S.: Attribute Grammar Encoding of the Structure and Behaviour of Artificial Neural Networks. Ph.D. Thesis, Queen’s University (2003)
Hussain, T.S.: Generic Neural Markup Language: Facilitating the Design of Theoretical Neural Network Models. In: Proceedings of the 2004 IEEE International Joint Conference on Neural Networks (IJCNN), pp. 235–242. Institute of Electrical and Electronics Engineers, Inc., Piscataway (2004)
Knuth, D.E.: The semantics of context-free languages. Mathematical Systems Theory 2(2), 127–145 (1968)
Deransart, P., Lorho, B., Jourdan, M.: Attribute Grammars: Definitions, Systems and Bibliography. LNCS, vol. 323. Springer, Heidelberg (1988)
Vellacott, O.R.: A framework of hierarchy for neural theory. In: Second International Conference on Artificial Neural Networks, pp. 237–241 (1991)
Gleeson, P., Steuber, V., Silver, R.A.: NeuroConstruct: A Tool for Modeling Networks of Neurons in 3D Space. Neuron 54(2), 219–235 (2007)
Mcgarry, K., Wermter, S., Macintyre, J.: Hybrid neural systems: from simple coupling to fully integrated neural networks. Neural Computing Surveys 2, 62–93 (1999)
Shavlik, J.W.: A Framework for Combining Symbolic and Neural Learning, Technical Report 1123, Computer Sciences Department, University of Wisconsin - Madison (November 1992)
Hussain, T.S.: Explicit learning relationships within neural systems. In: Workshop on Achieving Functional Integration of Diverse Neural Models, held at 2005 International Joint Conference on Neural Networks, p.16 (2005)
Iversen, A., Taylor, N.K., Brown, K.: Integrating Neural Network Strategies for Discrimination, Recognition and Clustering. In: Workshop on Achieving Functional Integration of Diverse Neural Models, held at 2005. Intl. Joint Conference on Neural Networks, p. 4 (2005)
de Kamps, M.: Large scale brain simulations are not a technical problem. In: Workshop on Achieving Functional Integration of Diverse Neural Models, held at 2005 International Joint Conference on Neural Networks, p. 2 (2005)
de Kamps, M., Baier, V., Drever, J., Dietz, M., Mosenlechner, L., van der Velde, F.: The state of MIIND. Neural Networks 21(8), 1164–1181 (2008)
Smieja, F.J., Mühlenbein, H.: Reflective Modular Neural Network Systems. Technical Report: GMD #633, German National Research Centre for Computer Science (1992)
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Computation 3, 79–87 (1991)
Hecht-Nielsen, R.: Neurocomputing. Addison-Wesley, Reading (1990)
Oblinger, D.: Bootstrapped learning: Creating the electronic student that learns from natural instruction. Defense Advanced Research Projects Agency briefing (2006), http://www.darpa.mil/ipto/programs/bl/docs/AAAI_Briefing.pdf (August 19, 2010)
Burstein, M., Brinn, M., Cox, M., Hussain, T., Laddaga, R., McDermott, D., McDonald, D., Tomlinson, R.: An architecture and language for the integrated learning of demonstrations. In: Workshop on Acquiring Planning Knowledge via Demonstration. held at the Twenty-Second National Conference on Artificial Intelligence (2007)
Burstein, M., Laddaga, R., McDonald, D., Cox, M., Benyo, B., Robertson, P., Hussain, T., Brinn, M., McDermott, D.: POIROT: integrated learning of web service procedures. In: Cohn, A. (ed.) Proceedings of the 23rd National Conference on Artificial intelligence, vol. 3, pp. 1274–1279. AAAI Press, Chicago (2008)
Koch, C., Segev, I.: Methods in Neuronal Modeling. From Ions to Networks, 2nd edn. MIT Press, Cambridge (1998)
Beeman, D.: Introduction to Realistic Neural Modeling. In: Bower, J.M., Beeman, D. (eds.) Special Issue on Realistic Neural Modeling - WAM-BAMM 2005 Tutorials; Brains, Minds 1, bmm218, (2005)
Jaeger, D.: Realistic single cell modeling – from experiment to simulation. In: Bower, J.M., Beeman, D. (eds.) Special Issue on Realistic Neural Modeling WAM-BAMM 2005 Tutorials; Brains, Minds 1, bmm222 (2005)
Carpenter, G.A., Grossberg, S.: ART 2: Self-organization of stable category recognition codes for analog input patterns. Applied Optics 26(23), 4919–4930 (1987)
Murre, J.M.J., Phaf, R.H., Wolters, G.: CALM: Categorizing and Learning Module. Neural Networks 5, 55–82 (1992)
Mendao, M.: Hormonally moderated neural control. In: Hudlicka, E., Canamero, L. (eds.) Architectures for Modeling Emotion: Cross Disciplinary Foundations: Papers from the 2004 Spring Symposium, pp. 92–95. American Assoc. Artificial Intelligence, Menlo Park (2004)
Liu, B., Ding, Y., Wang, J.: Intelligent network control system inspired from neuroendocrine-immune system. In: Chen, Y., Zhang, D., Deng, H., Xiao, Y. (eds.) Proceedings of the 6th International Conference on Fuzzy Systems and Knowledge Discovery, vol. 7, pp. 136–140. IEEE Press, Piscataway (2009)
Århem, P., Klement, G., Blomberg, C.: Channel density regulation of firing patterns in a cortical neuron model. Biophysical Journal 90(12), 4392–4404 (2006)
Mouret, J., Doncieux, S., Girard, B.: Importing the computational neuroscience toolbox into neuro-evolution-application to basal ganglia. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, pp. 587–594. ACM Press, New York (2010)
Floreano, D., Epars, Y., Zufferey, J., Mattiussi, C.: Evolution of spiking neural circuits in autonomous mobile robots. Intl. Journal of Intelligent Systems 21(9), 1005–1024 (2006)
Hebb, D.O.: The Organization of Behavior: A Neuropsychological Theory. Wiley, New York (1949)
Grillner, S.: The motor infrastructure: from ion channels to neuronal networks. Nature Reviews Neuroscience 4, 573–586 (2003)
Song, S., Miller, K.D., Abbott, L.F.: Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience 3(9), 919–926 (2000)
Kempter, R., Gerstner, W., van Hemmen, J.L.: Hebbian learning and spiking neurons. Physical Review E 59(4), 4498–4514 (1999)
Bray, T., Paoli, J., Sperberg-McQueen, C.M., Maler, E., Yergeau, F. (eds.): Extensible Markup Language (XML) 1.0 (5th edn.) (August 30, 2010), http://www.w3.org/TR/REC-xml/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Hussain, T.S. (2011). A Meta-Model Perspective and Attribute Grammar Approach to Facilitating the Development of Novel Neural Network Models. In: Jankowski, N., Duch, W., Gra̧bczewski, K. (eds) Meta-Learning in Computational Intelligence. Studies in Computational Intelligence, vol 358. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20980-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-20980-2_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20979-6
Online ISBN: 978-3-642-20980-2
eBook Packages: EngineeringEngineering (R0)