Skip to main content
SpringerLink
Log in

Self Organisation and Modal Learning: Algorithms and Applications

  • Chapter
Handbook on Neural Information Processing

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 49))

Abstract

Modal learning in neural computing [33] refers to the strategic combination of modes of adaptation and learning within a single artificial neural network structure. Modes, in this context, are learning methods that are transferable from one learning architecture to another, such as weight update equations. In modal learning two or more modes may proceed in parallel in different parts of the neural computing structure (layers and neurons), or they occupy the same part of the structure, and there is a mechanism for allowing the neural network to switch between modes.

From a theoretical perspective any individual mode has inherent limitations because it is trying to optimise a particular objective function. Since we cannot in general know a priori the most effective learning method or combination of methods for solving a given problem, we should equip the system (the neural network) with the means to discover the optimal combination of learning modes during the learning process. There is potential to furnish a neural system with numerous modes. Most of the work conducted so far concentrates on the effectiveness of two to four modes. The modal learning approach applies equally to supervised and unsupervised (including self organisational) methods. In this chapter, we focus on modal self organisation.

Examples of modal learning methods include the Snap-Drift Neural Network (SDNN) [5, 25, 28, 33, 32] which toggles its learning between two modes, an adaptive function neural network, in which adaptation applies simultaneously to both the weights and to the shape of the individual neuron activation functions, and the combination of four learning modes, in the form of Snap-drift ADaptive FUnction Neural Network [17, 18, 33]. In this chapter, after reviewing modal learning in general, we present some examples methods of modal self organisation. Self organisation is taken in the broadest context to include unsupervised methods.We review the simple unsupervised modalmethod called snap-drift [5, 25, 28, 32], which combines Learning Vector Quantization [21, 22, 23, 37] with a ’Min’ or Fuzzy AND method. Snap-drift is then applied to the Self-Organising Map [34]. The methods are utilised in numerous real-world problems such as grouping learners’ responses to multiple choice questions, natural language phrase recognition and pattern classification on well known datasets. Algorithms, dataset descriptions, pseudocode and Matlab code are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alpaydin, E., Kaynak, C.: Cascading Classifiers. Kybernetika 34, 369–374 (1998)

    MATH  Google Scholar 

  2. Burge, P., Shawe-Taylor, J.: An Unsupervised Neural Network Approach to Profiling the Behavior of Mobile Phone Users for Use in Fraud Detection. Journal of Parallel and Distributed Computing 61(7), 915–925 (2001)

    Article  MATH  Google Scholar 

  3. Carpenter, G.A., Grossberg, S.: Adaptive resonance theory. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, Second Edition, 2nd edn., pp. 87–90. MIT Press, Cambridge (2003)

    Google Scholar 

  4. Dafoulas, G.A.: The role of feedback in online learning communities. In: Fifth IEEE International Conference on Advanced Learning Technologies, pp. 827–831 (2005)

    Google Scholar 

  5. Donelan, H., Pattinson, C., Palmer-Brown, D.: The Analysis of User Behaviour of a Network Management Training Tool using a Neural Network. Journal of Systemics, Cybernetics and Informatics 3(5) (2006)

    Google Scholar 

  6. Dotan, Y., Intrator, N.: Multimodality exploration by an unsupervised projection pursuit neural network. IEEE Transactions on Neural Networks 9(3), 464–472 (1998)

    Article  Google Scholar 

  7. Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis, p. 218. John Wiley & Sons, New York (1973)

    MATH  Google Scholar 

  8. Ekpenyong, F., Palmer-Brown, D., Brimicombe, A.: Extracting road information from recorded GPS data using snap-drift neural network. Neurocomputing 73, 24–36 (2009)

    Article  Google Scholar 

  9. Fernandez Aleman, J.L., Palmer-Brown, D., Jayne, C.: Effects of Response Driven Feedback in Computer Science Learning. IEEE Transactions on Education 99 (2010), doi:10.1109/TE.2010.2087761

    Google Scholar 

  10. Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annual Eugenics 7, Part II, 179–188 (1936); also in Contributions to Mathematical Statistics. John Wiley, NY (1950)

    Article  Google Scholar 

  11. Forina, M., Lanteri, S., Armanino, C., et al.: PARVUS–an extendible package for data exploration, classification and correlation. Institute of Pharmaceutical and Food Analysis and Technologies, Via Brigata Salerno, 16147 Genoa, Italy (1991)

    Google Scholar 

  12. Garside, R., Leech, G., Varadi, T.: Manual of Information to Accompany the Lancaster Parsed Corpus. University of Oslo (1987)

    Google Scholar 

  13. Gupta, L., McAvoy, M.: Investigating the prediction capabilities of the simple recurrent neural network on real temporal sequences. Pattern Recognition 33(i12), 2075–2081 (2000)

    Article  MATH  Google Scholar 

  14. Hebb, D.O.: The organization of behavior. Wiley & Sons, New York (1949)

    Google Scholar 

  15. Higgins, E., Tatham, L.: Exploring the potential of Multiple Choice Questions in Assessment. Learning & Teaching in Action 2(1) (2003)

    Google Scholar 

  16. Horton, P., Nakai, K.: A Probablistic Classification System for Predicting the Cellular Localization Sites of Proteins. Intelligent Systems in Molecular Biology, 109–115 (1996)

    Google Scholar 

  17. Kang, M., Palmer-Brown, D.: A Multilayer ADaptive FUnction Neural Network (MADFUNN) for Analytical Function Recognition. IJCNN (2006); part of the IEEE World Congress on Computational Intelligence, WCCI 2006, Vancouver, BC, Canada, pp. 1784–1789 (2006)

    Google Scholar 

  18. Kang, M., Palmer-Brown, D.: A Modal Learning Adaptive Function Neural Network Applied to Handwritten Digit Recognition. Information Sciences 178(20), 3802–3812 (2008)

    Article  Google Scholar 

  19. Kaynak, C.: Methods of Combining Multiple Classifiers and Their Applications to Handwritten Digit Recognition. MSc Thesis, Institute of Graduate Studies in Science and Engineering, Bogazici University (1995)

    Google Scholar 

  20. Kohonen, T.: Self-organised formation of topologically correct feature maps. Biological Cybernetics 43 (1982)

    Google Scholar 

  21. Kohonen, T.: Learning Vector Quantisation. Helsinki University of Technology, Laboratory of Computer and Information Science, Report TKK-F-A-601 (1986)

    Google Scholar 

  22. Kohonen, T.: Learning Vector Quantisation. Neural Networks 1, 303 (1988)

    Article  Google Scholar 

  23. Kohonen, T.: Self-Organisation and Asssociative Memory, 3rd edn. Springer, Heilderberg (1989)

    Book  Google Scholar 

  24. Kohonen, T.: Improved Versions of Learning Vector Quantization. In: Proc. of IJCNN 1990, Washington, DC, vol. 1, pp. 545–550 (1990)

    Google Scholar 

  25. Lee, S.W., Palmer-Brown, D., Roadknight, C.M.: Performance guided Neural Network for Rapidly Self Organising Active Network Management. Neurocomputing 61C, 5–20 (2004a)

    Article  Google Scholar 

  26. Lee, S.W., Palmer-Brown, D., Roadknight, C.M.: Reinforced Snap Drift Learning for Proxylet Selection in Active Computer Networks. In: Proc. of IJCNN 2004, Budapest, Hungary, vol. 2, pp. 1545–1550 (2004b)

    Google Scholar 

  27. Lee, S.W., Palmer-Brown, D.: Snap-drift learning for phrase recognition. In: Proc. IEEE IJCNN 2005, Montréal, Québec, Canada, vol. 1, pp. 588–592 (2005)

    Google Scholar 

  28. Lee, S.W., Palmer-Brown, D.: Phonetic Feature Discovery in Speech Using Snap-Drift Learning. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006, Part II. LNCS, vol. 4132, pp. 952–962. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  29. Mayberry III, M.R., Miikkulainen, R.: SARDSRN: A Neural Network ShiftReduce Parser. In: Proc. of the 16th IJCAI, Stockholm, Sweden, pp. 820–825 (1999)

    Google Scholar 

  30. MacKay, D.J.C.: Bayesian methods for supervised neural networks. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, pp. 144–149. MIT Press, Cambridge (1998)

    Google Scholar 

  31. Palmer-Brown, D., Tepper, J., Powell, H.: Connectionist Natural Language Parsing. Trends in Cognitive Sciences 6(10), 437–442 (2002)

    Article  Google Scholar 

  32. Palmer-Brown, D., Draganova, C., Lee, S.W.: Snap-Drift Neural Network for Selecting Student Feedback. In: Proc. IJCNN 2009, Atlanta, USA, pp. 391–398 (2009)

    Google Scholar 

  33. Palmer-Brown, D., Lee, S.W., Draganova, C., Kang, M.: Modal Learning Neural Networks. WSEAS Transactions on Computers 8(2), 222–236 (2009)

    Google Scholar 

  34. Palmer-Brown, D., Draganova, C.: Snap-Drift Self Organising Map. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds.) ICANN 2010, Part II. LNCS, vol. 6353, pp. 402–409. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  35. Palmer-Brown, D., Draganova, C.: Recurrent Snap Drift Neural Network for Phrase Recognition. In: WCCI 2010 IEEE World Congress on Computational Intelligence, IJCNN 2010, Barcelona, Spain, pp. 3445–3449 (2010)

    Google Scholar 

  36. Payne, A., Brinkman, W.-P., Wilson, F.: Towards Effective Feedback in e-Learning Packages: The Design of a Package to Support Literature Searching, Referencing and Avoiding Plagiarism. In: Proceedings of HCI 2007 Workshop: Design, Use and Experience of e-Learning Systems, pp. 71–75 (2007)

    Google Scholar 

  37. Ritter, H., Kohonen, T.: Self-Organizing Semantic Maps. Biological Cybernetics 61, 241–254 (1989)

    Article  Google Scholar 

  38. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagation errors. Nature 323, 533–536 (1986)

    Article  Google Scholar 

  39. Rushton, J.N.: Natural Language Parsing using Simple Neural Networks. In: Proc. of MLMTA, Las Vegas, Nevada, pp. 137–141 (2003)

    Google Scholar 

  40. Tepper, J., Powell, H.M., Palmer-Brown, D.: A Corpus based Connectionist Architecture for Large scale Natural Language Parsing. Connection Science 14(2), 93–114 (2002)

    Article  Google Scholar 

  41. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)

    Book  MATH  Google Scholar 

  42. Webb, A.R., Lowe, D.: A hybrid optimisation strategy for adaptive feed-forward layered newtorks. RSRE Memorandum 4193, Royal Signals and Radar Establishemnt, St Andrews Road, Malvern, UK (1988)

    Google Scholar 

  43. Widrow, B., Hoff, M.E.: Institute of Radio Engineers WESCON Convention Record, Adaptive switching circuits, pp. 96–104. Institute of Radio Engineers, New York (1960)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. London Metropolitan University, 166-220 Holloway Road, London, N7 8DB, UK

    Dominic Palmer-Brown & Chrisina Jayne

Authors
  1. Dominic Palmer-Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chrisina Jayne
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dominic Palmer-Brown .

Editor information

Editors and Affiliations

  1. , Dipto. Ingegneria dell'Informazione, Università degli Studi di Siena, Via Roma 56, Siena, 53100, Italy

    Monica Bianchini

  2. Fac. Ingegneria, Dipto. Ingegneria dell'Informazione, Università Siena, Via Roma 56, Siena, 53100, Italy

    Marco Maggini

  3. University of Canberra, School of Electrical and Information, Adjunct Professor, Mawson Lakes Campus, ACT, 2601, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Palmer-Brown, D., Jayne, C. (2013). Self Organisation and Modal Learning: Algorithms and Applications. In: Bianchini, M., Maggini, M., Jain, L. (eds) Handbook on Neural Information Processing. Intelligent Systems Reference Library, vol 49. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36657-4_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-36657-4_11

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-36656-7

  • Online ISBN: 978-3-642-36657-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation