Summary
Call admission control in access network has become an interesting topic for the research community due to its potential applicability in broadband wireless systems. Admission control problem can be formulated as Markov decision process (MDP) and has proven to deliver optimal policies for blocking and dropping probabilities in wireless networks. This however typically requires the model to know the system dynamics in advance. One approach to solving MDPs considers letting CAC agent interact with the environment and learn by ”trial and error” to choose optimal actions - thus Reinforcement Learning algorithms are applied. Abstraction and generalization techniques can be used with RL algorithms to solve MDPs with large state space. In this paper authors decribe and evaluate a MDP formulated problem to find optimal Call Admission Control policies for WiMAX networks with adaptive modulation and coding. We consider two classes of service (BE and UGS-priority) and a variable capacity channel with constant error bit rate. Hierarchical Reinforcement Learning (HRL) techniques are applied to find optimal policies for multi-task CAC agent. In addition this article validates several neural network training algorithms to deliver a training algorithm suitable for the CAC agent problem.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Putterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley and Sons Inc. Publication, Hoboken (2005)
Nasser, N., Hassanein, H.: An optimal and fair call admission control policy for seamless handoff in multimedia wireless networks with QoS guarantees. In: Global Telecommunications Conference, vol. 6, pp. 3926–3930 (2004)
Yang, X., Feng, G.: Cost Minimization for Admission Control in Bandwidth Asymmetry Wireless Networks. In: IEEE International Conference on Communications, pp. 5484–5489 (2007)
Wenlong, N., Wei, L., Alam, M.: Determination of optimal call admission control policy in wireless networks. IEEE Transactions on Wireless Communications 8, 1038–1044 (2009)
Nordstrom, E., Carlstrom, J.: Call admission control and routing for integrated CBR/VBR and ABR services: a Markov decision approach. In: IEEE Proceedings, pp. 71–76 (1999)
Senouci, S., Beylot, A., Pujolle, G.: Call Admission Control in Cellular Networks: A Reinforcement Learning Solution. International Journal of Network Management 13, 89–103 (2004)
Mitchell, T.M.: Machine Learning. McGraw-Hill Science-Engineering-Math, New York (1997)
Pietrabissa, A., Delli, P.F.: MDP call control in variable capacity communication networks. In: 18th Mediterranean Conference on Control and Automation, pp. 483–488 (2010)
Ren-Hung, H., Kurose, J.F., Towsley, D.: MDP routing in ATM networks using virtual path concept. In: 13th Proceedings IEEE Networking for Global Communications, pp. 1509–1517 (1994)
IEEE 802.16. Part 16: Air Interface for Broadband Wireless Access Systems (2009)
Yu, K., Wang, X., Sun, S., Zhang, L., Wu, X.: A Statistical Connection Admission Control Mechanism for Multiservice IEEE 802.16 Network. In: Vehicular Technology Conference, pp. 1–5 (2009)
Pietrabissa, A.: Reinforcement learning call control in variable capacity links. In: 18th Mediterranean Conference on Control and Automation, pp. 933–938 (2010)
Vu, X.-T., Nguyen, D.-T., Vu, T.A.: An finite-state Markov channel model for ACM scheme in WiMAX. In: Region 10 Conference, pp. 1–6 (2009)
Flizikowski, A., Hołubowicz, W., Przybyszewski, M., Grzegorzewski, S.: Admission control and system capacity assessment of WiMAX with ACM and nb-LDPC codes - simulation study with ViMACCS ns2 patch. In: The International Conference on Advanced Information Networking and Applications, Perth (2009)
Flizikowski, A., Przybyszewski, M., Majewski, M., Kozik, R.: Evaluation of guard channel admission control schemes for IEEE 802.16 with integrated nb-LDPC codes. In: International Conference on Ultra Modern Telecommunications, St.-Petersburg
Frontiers of Mobile and Wireless Communication. In: Proceedings of the IEEE 6th Circuits and Systems Symposium, vol. 2, pp. 549–552 (2004)
Barto, A., Mahadevan, S.: Recent Advances in Hierarchical Reinforcement Learning. Discrete Event Dynamic Systems: Theory and Applications 13, 343–379 (1999)
Sutton, R., Precup, D., Singh, S.: Between MDPs and Semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112, 181–211 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Flizikowski, A., Majewski, M., Przybyszewski, M. (2011). Evaluation of Optimal Resource Management Policies for WiMAX Networks with AMC: A Reinforcement Learning Approach. In: Choraś, R.S. (eds) Image Processing and Communications Challenges 3. Advances in Intelligent and Soft Computing, vol 102. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23154-4_50
Download citation
DOI: https://doi.org/10.1007/978-3-642-23154-4_50
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23153-7
Online ISBN: 978-3-642-23154-4
eBook Packages: EngineeringEngineering (R0)