Skip to main content
SpringerLink
Log in

A generalized reinforcement learning scheme for random neural networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

We propose a reinforcement learning scheme for random neural networks (RNN) that update network weights based on the relative reward obtained by past actions. This new learning scheme achieves improved performance in random environments under certain conditions, in particular, when the number of actions is small. We also explore hybrid learning structures, where a RNN functions as a meta decision unit that learns how to best delegate the action decision to a second layer of decision units. An extensive simulation study demonstrates the performance of these learning methods applied to minimizing response times in a distributed task assignment scenario. Results are compared to the cognitive packet networks’ reinforcement learning scheme, a reward-only variation, and several non-learning methods. The study reveals that no single method can produce the best performance results in all cases. However, both the generalized reinforcement learning scheme and the application of meta decision units can help to achieve improved performance in specific cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Aiello G, Gaglio S, Re G, Storniolo P, Urso A (2005) The random neural network model for the on-line multicast problem. In: Apolloni B, Marinaro M, Tagliaferri R (eds) Biological and artificial intelligence environments. Springer, Dordrecht, pp 157–164

    Chapter  Google Scholar 

  2. Atalay V, Gelenbe E (1992) Parallel algorithm for colour texture generation using the random neural network model. Int J Pattern Recognit Artif Intell 06(02n03):437–446. doi:10.1142/S0218001492000266

    Article  Google Scholar 

  3. Atalay V, Gelenbe E, Yalabik N (1992) The random neural network model for texture generation. Int J Pattern Recognit Artif Intell 06(01):131–141. doi:10.1142/S0218001492000072

    Article  Google Scholar 

  4. Bakircioglu H, Gelenbe E, Koccak T (1998) Image processing with the random neural network model. Elektrik 5(1):65–77

    Google Scholar 

  5. Basterrech S, Rubino G (2013) A more powerful random neural network model in supervised learning applications. In: 2013 international conference on soft computing and pattern recognition, SoCPaR 2013, Hanoi, Vietnam, December 15–18, 2013, pp 201–206. doi:10.1109/SOCPAR.2013.7054127

  6. Basterrech S, Mohammed S, Rubino G, Soliman M (2011) Levenbergmarquardt training algorithms for random neural networks. Comput J 54(1):125–135

    Article  Google Scholar 

  7. Bi H, Akinwande O (2015) Multi-path routing in emergency with health-aware classification. In: 2015 2nd international conference on information and communication technologies for disaster management (ICT-DM), pp 109–115. doi:10.1109/ICT-DM.2015.7402026

  8. Bi H, Desmet A, Gelenbe E (2013) Routing emergency evacuees with cognitive packet networks. Springer, Cham, pp 295–303

    Google Scholar 

  9. Busoniu L, hatska RB, Schutter BD (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern C Appl Rev 38(2):156–172. doi:10.1109/TSMCC.2007.913919

    Article  Google Scholar 

  10. Cancela H, Robledo F, Rubino G (2004) A grasp algorithm with rnn based local search for designing a wan access network. Electron Notes Discrete Math 18(Complete):59–65. doi:10.1016/j.endm.2004.06.010

    Article  MathSciNet  MATH  Google Scholar 

  11. Cramer C, Gelenbe E, Bakircioglu H (1996) Low bit rate video compression with neural networks and temporal sub-samp ling. Proc IEEE 84(10):1529–1543

    Article  Google Scholar 

  12. Cramer C, Gelenbe E, Gelenbe P (1998) Image and video compression. IEEE Potentials 17(1):29–33

    Article  Google Scholar 

  13. Cramer CE, Gelenbe E (2000) Video quality and traffic QoS in learning-based subsampled and receiver-interpolated video sequences. IEEE J Sel Areas Commun 18(2):150–167

    Article  Google Scholar 

  14. Gelenbe E (1989) Random neural networks with positive and negative signals and product form solution. Neural Comput 1(4):502–510

    Article  Google Scholar 

  15. Gelenbe E (1990) Stability of the random neural network model. Neural Comput 2:239–247

    Article  Google Scholar 

  16. Gelenbe E (1992) Une généralisation probabiliste du probleme SAT. Comptes Rendus de l’Académie des Sciences 309 Série II 313:339–3422

    MathSciNet  MATH  Google Scholar 

  17. Gelenbe E (1993) Learning in the recurrent random neural network. Neural Comput 5(1):154–164. doi:10.1162/neco.1993.5.1.154

    Article  Google Scholar 

  18. Gelenbe E, Batty F (1992) Minimum cost graph covering with the random neural network. Pergamon, New York, pp 139–147

    Google Scholar 

  19. Gelenbe E, Kocak T (2000) Area-based results for mine detection. IEEE Trans Geosci Remote Sens 38(1):12–24

    Article  Google Scholar 

  20. Gelenbe E, Lent R (2004) Power-aware ad hoc cognitive packet networks. Ad Hoc Netw J 2(3):205–216

    Article  Google Scholar 

  21. Gelenbe E, Koubi V, Pekergin F (1993) Dynamical random neural network approach to the traveling salesman problem. In: Proceedings of the IEEE symposium on systems, man and cybernetics, pp 630–635

  22. Gelenbe E, Feng Y, Ranga K, Krishnan R (1996) Neural networks for volumetric MR imaging of the brain. In: Proceedings of international workshop on neural networks for identification, control, robotics and signal/image processing, pp 194–202. doi:10.1109/NICRSP.1996.542760

  23. Gelenbe E, Ghanwani A, Srinivasan V (1997a) Improved neural heuristics for multicast routing. IEEE J Sel Areas Commun 15(2):147–155

    Article  Google Scholar 

  24. Gelenbe E, Ghanwani A, Srinivasan V (1997b) Improved neural heuristics for multicast routing. IEEE J Sel Areas Commun 15(2):147–155. doi:10.1109/49.552065

    Article  Google Scholar 

  25. Gelenbe E, Seref E, Xu Z (1999) Towards networks with intelligent packets. In: Proceedings of the fourteenth international symposium on computer and information sciences, pp 1–11

  26. Gelenbe E, Hussain KF, Abdelbaki H (2000) Random neural network texture model. doi:10.1117/12.382903

  27. Gelenbe E, Lent R, Xu Z (2001a) Design and performance of cognitive packet networks. Perform Eval 46(2,3):155–176

    Article  MATH  Google Scholar 

  28. Gelenbe E, Seref E, Xu Z (2001b) Simulation with learning agents. Proc IEEE 89(2):148–157. doi:10.1109/5.910851

    Article  Google Scholar 

  29. Gelenbe E, Lent R, Nunez A (2004) Self-aware networks and QoS. Proc IEEE 9(92):1478–1489

    Article  Google Scholar 

  30. Gelenbe E, Timotheou S, Nicholson D (2010) Random neural network for emergency management. In: Workshop on grand challenges in modeling, simulation and analysis for homeland security (MSAHS’10), Washington, DC

  31. Georgiopoulos M, Li C, Kocak T (2011) Learning in the feed-forward random neural network: a critical review. Perform Eval 68(4):361–384

    Article  MATH  Google Scholar 

  32. Hagan MT, Menhaj MB (1994) Training feedforward networks with the Marquardt algorithm. Trans Neural Netw 5(6):989–993. doi:10.1109/72.329697

    Article  Google Scholar 

  33. Halici U (1997) Reinforcement learning in random neural networks for cascaded decisions. Biosystems 40(1):83–91. doi:10.1016/0303-2647(96)01633-4

    Article  Google Scholar 

  34. Lent R (2007) Linear QoS goals of additive and concave metrics in ad hoc cognitive packet routing. IEEE Trans Syst Man Cybern 6(36):255–1260

    Google Scholar 

  35. Loukas G, Öke G (2007a) A biologically inspired pired denial of service detector using the random neural network. In: IEEE international conference on mobile adhoc and sensor systems, 2007. MASS 2007, pp 1–6

  36. Loukas G, Öke G (2007b) Likelihood ratios and recurrent random neural networks in detection of denial of service attacks. In: Proceedings of international symposium of computer and telecommunication systems (SPECTS ’07), San Diego, CA, USA

  37. Lu R, Shen Y (2005) Image segmentation based on random neural network model and gabor filters. In: 2005 IEEE engineering in medicine and biology 27th annual conference, pp 6464–6467. doi:10.1109/IEMBS.2005.1615979

  38. Martinelli G, Mascioli FMF, Panella M, Rizzi A (2002) Extended random neural networks. In: Proceedings of the 13th Italian workshop on neural nets-revised papers. WIRN VIETRI 2002. Springer, London, pp 75–82. http://dl.acm.org/citation.cfm?id=645961.674072

  39. Mohamed S, Rubino G (2002) A study of real-time packet video quality using random neural networks. IEEE Trans Circuits Syst Video Technol 12(12):1071–1083. doi:10.1109/TCSVT.2002.806808

    Article  Google Scholar 

  40. Narendra KS, Member S, Thathachar MAL (1974) Learning automata–a survey. IEEE Trans Syst Man Cybern 4:323–334

    Article  MathSciNet  MATH  Google Scholar 

  41. Öke G, Loukas G (2007) A denial of service detector based on maximum likelihood detection and the random neural network. Comput J 50(6):717–727

    Article  Google Scholar 

  42. Öke G, Loukas G, Gelenbe E (2007) Detecting denial of service attacks with bayesian classifiers and the random neural network. In: Fuzzy Systems Conference, 2007. FUZZ-IEEE 2007. IEEE International, pp 1–6

  43. Rubino G, Tirilly P, Varela M (2006) Evaluating users’ satisfaction in packet networks using random neural networks. In: Kollias SD, Stafylopatis A, Duch W, Oja E (eds) Artificial neural networks – ICANN 2006. ICANN 2006. Lecture notes in computer science, vol 4131. Springer, Berlin, Heidelberg

  44. Russell SJ, Norvig P (2003) Artificial intelligence: a modern approach, 2nd edn. Pearson Education, London

    MATH  Google Scholar 

  45. Serrano W, Gelenbe E (2016) An intelligent internet search assistant based on the random neural network. Springer, Cham, pp 141–153. doi:10.1007/978-3-319-44944-9_13

    Book  Google Scholar 

  46. Stafylopatis A, Likas A (1992) Pictorial information retrieval using the random neural network. IEEE Trans Softw Eng 18(7):590–600. doi:10.1109/32.148477

    Article  Google Scholar 

  47. Sutton RS, Barto AG (1998) Introduction to reinforcement learning, 1st edn. MIT Press, Cambridge

    MATH  Google Scholar 

  48. Zhong Y, Sun D, Wu J (2005) Dynamical random neural network approach to a problem of optimal resource allocation. In: Proceedings of the 8th international conference on artificial neural networks: computational intelligence and bioinspired systems. IWANN’05. Springer, Berlin, pp 1157–1163

Download references

Author information

Authors and Affiliations

  1. University of Houston, Houston, TX, USA

    Ricardo Lent

Authors
  1. Ricardo Lent
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ricardo Lent.

Ethics declarations

Conflict of interest

The author declares that he has no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lent, R. A generalized reinforcement learning scheme for random neural networks. Neural Comput & Applic 31, 2699–2716 (2019). https://doi.org/10.1007/s00521-017-3223-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-017-3223-1

Keywords

Search

Navigation