Abstract
This paper deals with Quality of Service-Routing (QoS-routing) in telecommunication networks with irregular traffics. The prediction of congestion’s problems in the networks in real time is quite difficult, making the effectiveness of ”traditional” methodologies based on analytical models questionable. An adaptive routing algorithm called Q-Neural Routing is proposed here. Compared to standard Q-Routing, the Q-value is approximated by a reinforcement learning based neural network of a fixed size, allowing the learner to incorporate various parameters such as local queue size and time of day, into its distance estimation. Moreover, each router uses an on line learning module to optimize the path in terms of average packet delivery time, by taking into account the waiting queue states of neighboring routers. The performance of the proposed algorithm is evaluated experimentally with OPNET simulator for different levels of load and compared to standard RIP and Q-Routing algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Habib, I.: Application of Neurocomputing in Traffic Management of ATM Networks. Proceeding of the IEEE 84(10) (October 1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning. MIT Press, Cambridge (1997)
Mellouk, A., Gallinari, P.: Discriminative training for improved neural prediction system. In: IEEE Int. Acoustic, Speech and Signal Processing (1993)
Levin, E.: Hidden Control architecture modeling of non-queuear time invariant systems and its applications. IEEE Trans. Neur. Networks (January 1993)
Boyan, J.A., Littman, M.L.: Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach. In: Cowan, Tesauro, Alspector (eds.) Advances in Neural Information Processing Systems 6 (1994)
Kumar, S., Miikkualainen, R.: Dual reinforcement Q-routing: an on-queue adaptive routing algorithm. In: Proceedings of Neural Networks in Engineering (1997)
Nevers, J.E., Leitao, M.J., Almeida, L.B.: Neural Networks in B-ISDN Flow Control: ATM Traffic Prediction or Network Modeling. IEEE Com. Magazine (October 1995)
Lemaire, V., Clérot, F.: Estimation of the Blocking probabilities in an ATM Network Node Using Artificial Neural Networks for Connection Admission Control. In: International Tel. traffic Congress, Edinburgh, vol. 16 (1999)
Aussem, A., Rouxel, S., Marie, R.: Neural-based Queuing System Modeling for Service Quality Estimation in Communication Networks. In: ICANN (1999)
Malkin, G.: RIP version2: Carrying Additional Information, RFC 1388 RFC (1993)
Bellman, R.: On a routing problem. Quarterly of Applied Mathematics 16(1), 87–90 (1958)
Wang, Z., Crowcroft, J.: QoS Routing for Supporting Resource Reservation. IEEE Journal on Selected Areas in Communications (September 1996)
Watkins, C.J., Dayan, P.: Q-Learning. Machine Learning 8, 279–292 (1989)
Jacobson, V.: Congestion Avoidance of Network Traffic, Computer Communication. Review 18(4), 314–329 (1988)
Feng, W., Kandlur, D., Saha, D., Shin, K.: Understanding TCP Dynamics in an Integrated Services Internet. In: NOSSDAV 1997 (1997)
Shenker, S., Partridge, C., Guerin, R.: Specification of guaranteed quality of service, RFC2212 (septembre 1997)
Turner, J.: New directions in communications (or which way to the information age). IEEE Communications Magazine 24(10) (1986)
Zhang, L., Deering, S., Estrin et, D., Zappala, D.: RSVP: A New Resource ReSerVation Protocol. IEEE Network 7(5), 8–18 (1993)
Bernet, Y.: Requirements of Diff-serv Boundary Routers, IETF Internet Draft (1998)
Nichols, K., Blake, S.: Differentiated Services Operational Model and Definitions, IETF Internet Draft (1998)
Moy, J.: OSPF Version 2, RFC2328, IETF (1998)
Crawley, E., Nair, R., Rajagopalan, B., Sandick, H.: A Framework for QoS-based Routing in the Internet, RFC2386, IETF (August. 1998)
Rosen, E., Viswanathan, A., Callon, R.: Multiprotocol Label Switching Architecture, Internet Draft draft-ietf-mpls-arch-06.txt, IETF (August 1999)
Stallings, W.: MPLS. Internet Protocol Journal 4(3) (September 2001)
Partridge, C.: A proposed flow specification, RFC1363, IETF (September 1992)
Goetz, P., Kumar, S., Miikkulainen, R.: On-Line Adaptation of a Signal Predistorter through Dual Reinforcement Learning. In: Proc. Machine Learning. Proceedings of the 13th Annual Conference, Bari, Italy (1996)
Kumar, S., Miikkualainen, R.: Confidence-based Q-routing: an on-queue adaptive routing algorithm. In: Proceedings of Neural Networks in Engineering (1998)
Costa, L.H.M.K., Fdida, S., Duarte, O.C.M.B.: A Scalable Algorithm for Link-state QoS-based Routing with Three Metrics. In: Proc. of IEEE ICC 2001, Helsink, Filand (2001)
Subramanian, D., Druschel, P., Chen, J.: Ants and reinforcement learning: A case study in routing in dynamic networks. In: Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, vol. 2, pp. 832–839 (1997)
Wang, H., Shen, C., Shin, K.: Adaptive-Weighted Packet Scheduling for Premium Service. In: IEEE International Conference on Communications (ICC 2001), pp. 1846–1850 (2001)
Gelenbe, E., Lent, R., Xu, Z.: Towards networks with cognitive packets. In: Proc. IEEE MASCOTS Conference, San Francisco, CA, August 29-September 1, pp. 3–12 (2000) ISBN 0-7695-0728-X
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hoceini, S., Mellouk, A., Amirat, Y. (2004). Neural Net Based Approach for Adaptive Routing Policy in Telecommunication Networks. In: Mammeri, Z., Lorenz, P. (eds) High Speed Networks and Multimedia Communications. HSNMC 2004. Lecture Notes in Computer Science, vol 3079. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-25969-5_32
Download citation
DOI: https://doi.org/10.1007/978-3-540-25969-5_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22262-0
Online ISBN: 978-3-540-25969-5
eBook Packages: Springer Book Archive