Skip to main content
SpringerLink
Log in

Optimization of user behavior based handover using fuzzy Q-learning for LTE networks

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

In LTE networks, handover optimization is necessary to enhance the users’ satisfaction. Specifically, users using real time traffic need to experience continuous connectivity. Hence, radio link failures (RLFs) severely affect their quality of experience. Decreasing the RLFs for non-real time users is not as urgent as the case of real time users. On the other hand, a total network collapse can happen in case of too much unnecessary handovers (ping-pongs). In this work, fuzzy Q-learning is used to optimize the two contradictory handover problems, which are RLFs and ping-pongs. The former needs to decrease Handover Margin (HOM) to reduce the too late handover, and the latter needs to increase the HOM to reduce the unnecessary signaling. In the developed algorithm, the users in the network are divided into four categories, according to their speed and the data traffic used. This increases the satisfaction of some users, while keeping the overall handover problems within acceptable limits. For each category of users, fuzzy Q-learning is applied with a different initial candidate fuzzy actions. The proposed technique shows the best performance for each category of users in terms of the most preferred metric, either decreasing RLF or decreasing ping-pongs, for this category of users in comparison with two other literature techniques, or without using any optimization technique. Moreover, the algorithm is robust against changes in the number of users in the system, as it maintains the best solution when the number of users is halved or even doubled.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. 3GPP (2011) Evolved Universal Terrestrial Radio Access Network (E-UTRAN); self-configuring and self-optimizing network (SON) use cases and solutions. TR 36.902, 3rd Generation Partnership Project (3GPP). http://www.3gpp.org/ftp/Specs/html-info/36902.htm.

  2. Zhang, H., Wen, X., Wang, B., Zheng, W., & Lu, Z. (2009). A novel self-optimizing handover mechanism for multi-service provisioning in LTE-advanced. In International conference on research challenges in computer science, 2009 (ICRCCS’09), pp. 221–224. doi:10.1109/ICRCCS.2009.64.

  3. Alonso-Rubio, J. (2010). Self-optimization for handover oscillation control in LTE. In 2010 IEEE Network Operations and Management Symposium (NOMS), pp. 950–953. doi:10.1109/NOMS.2010.r5488335.

  4. Khan, M., Rahman, M., Raahemifar, K., Misic, J., & Misic, V. (2014). Self-optimizing control parameters for minimizing ping-pong handover in long term evolution (LTE). In 27th Biennial symposium on communications (QBSC), pp. 118–122. doi:10.1109/QBSC.2014.6841197.

  5. Balan, I., Jansen, T., Sas, B., Moerman, I., & Kurner, T. (2011). Enhanced weighted performance based handover optimization in LTE. In Future network and mobile summit (FutureNetw), pp. 1–8.

  6. Lobinger, A., Stefanski, S., Jansen, T., & Balan, I. (2011). Coordinating handover parameter optimization and load balancing in LTE self-optimizing networks. In 2011 IEEE 73rd vehicular technology conference (VTC Spring), pp. 1–5. doi:10.1109/VETECS.2011.5956561.

  7. Jansen, T., Balan, I., Stefanski, S., Moerman, I., & Kurner, T. (2011). Weighted performance based handover parameter optimization in LTE. In 2011 IEEE 73rd vehicular technology conference (VTC Spring), pp. 1–5. doi:10.1109/VETECS.2011.5956572.

  8. Jansen, T., Balan, I., Turk, J., Moerman, I., & Kurner, T.(2010). Handover parameter optimization in LTE self-organizing networks. In 2010 IEEE 72nd vehicular technology conference fall (VTC 2010-Fall), pp. 1–5. doi:10.1109/VETECF.2010.5594245.

  9. Kitagawa, K., Komine, T., Yamamoto, T., & Konishi, S. (2011). A handover optimization algorithm with mobility robustness for LTE systems. In 2011 IEEE 22nd international symposium on personal, indoor and mobile radio communications (PIMRC), pp. 1647–1651. doi:10.1109/PIMRC.2011.6139784.

  10. Berenji, H. (1994). Fuzzy Q-learning: A new approach for fuzzy dynamic programming. In Proceedings of 3rd IEEE conference on fuzzy systems, 1994. IEEE world congress on computational intelligence, Vol. 1, pp. 486–491. doi:10.1109/FUZZY.1994.343737.

  11. Glorennec, P. (1994). Fuzzy Q-learning and dynamical fuzzy Q-learning. In Proceedings of the 3rd IEEE conference on fuzzy systems, 1994. IEEE World Congress on Computational Intelligence, Vol. 1, pp. 474–479. doi:10.1109/FUZZY.1994.343739.

  12. Glorennec, P., & Jouffe, L. (1997). Fuzzy Q-learning. In Proceedings of the 6th IEEE international conference on fuzzy systems, Vol. 2, pp. 659–662. doi:10.1109/FUZZY.1997.622790.

  13. Razavi, R., Klein, S., & Claussen, H. (2010). A fuzzy reinforcement learning approach for self-optimization of coverage in LTE networks. Bell Labs Technical Journal, 15(3), 153–175. doi:10.1002/bltj.20463.

    Article  Google Scholar 

  14. Chen, Y. H., Chang, C. J., & Huang, C. Y. (2009). Fuzzy Q-learning admission control for WCDMA/WLAN heterogeneous networks with multimedia traffic. IEEE Transactions on Mobile Computing, 8(11), 1469–1479. doi:10.1109/TMC.2009.65.

    Article  Google Scholar 

  15. Galindo-Serrano, A., & Giupponi, L. (2011). Downlink femto-to-macro interference management based on fuzzy Q-learning. In International symposium on modeling and optimization in mobile, ad hoc and wireless networks (WiOpt), pp. 412–417. doi:10.1109/WIOPT.2011.5930054.

  16. Simsek, M., & Czylwik, A. (2012). Improved decentralized fuzzy Q-learning for interference reduction in heterogeneous LTE-networks. In Proceedings of the 17th international OFDM workshop 2012 (InOWo’12), pp. 1–6

  17. Xu, Y., Li, L., Soong, B.H., & Li, C. (2014). Fuzzy Q-learning based vertical handoff control for vehicular heterogeneous wireless network. In IEEE international conference on communications (ICC), pp. 5653–5658. doi:10.1109/ICC.2014.6884222.

  18. Munoz, P., Barco, R., Ruiz-Aviles, J., de la Bandera, I., & Aguilar, A. (2013a). Fuzzy rule-based reinforcement learning for load balancing techniques in enterprise LTE femtocells. IEEE Transactions on Vehicular Technology, 62(5), 1962–1973. doi:10.1109/TVT.2012.2234156.

    Article  Google Scholar 

  19. Munoz, P., Barco, R., & de la Bandera, I. (2013b). Optimization of load balancing using fuzzy Q-learning for next generation wireless networks. Expert Systems with Applications, 40(4), 984–994. doi:10.1016/j.eswa.2012.08.071.

    Article  Google Scholar 

  20. Klein, A., Kuruvatti, N., Schneider, J., & Schotten, H .(2013). Fuzzy Q-learning for mobility robustness optimization in wireless networks. In 2013 IEEE globecom workshops (GC Wkshps), pp. 76–81. doi:10.1109/GLOCOMW.2013.6824965.

  21. Munoz, P., Barco, R., & de la Bandera, I. (2015). Load balancing and handover joint optimization in LTE networks using fuzzy logic and reinforcement learning. Computer Networks, 76, 112–125. doi:10.1016/j.comnet.2014.10.027.

    Article  Google Scholar 

  22. Hegazy, R. D., & Nasr, O. A. (2015). A user behavior based handover optimization algorithm for LTE networks. In 2015 IEEE wireless communications and networking (WCNC), pp. 1255–1260. doi:10.1109/WCNC.2015.7127649.

  23. 3GPP (2007) Physical layer aspect for evolved Universal Terrestrial Radio Access (UTRA). TR 25.814, 3rd Generation Partnership Project (3GPP). http://www.3gpp.org/ftp/Specs/html-info/25814.htm.

  24. 3GPP (2013) Evolved Universal Terrestrial Radio Access (E-UTRA); radio resource control (RRC); protocol specification. TS 36.331, 3rd Generation Partnership Project (3GPP). http://www.3gpp.org/ftp/Specs/html-info/36331.htm.

  25. 3GPP (2014) Evolved Universal Terrestrial Radio Access (E-UTRA) and Evolved Universal Terrestrial Radio Access (E-UTRAN); overall description; stage 2. TS 36.300, 3rd Generation Partnership Project (3GPP). http://www.3gpp.org/ftp/Specs/html-info/36300.htm

  26. Dayan, P., & Watkins, C. (2001). Reinforcement learning, encyclopedia of cognitive (science ed.). London: MacMillan Press.

    Google Scholar 

  27. Watkins, C., & Dayan, P. (1992). Technical note. In: Sutton, R. (Ed.), Reinforcement learning, The Springer International Series in engineering and computer science, Vol. 173, pp 55–68. New York: Springer. doi:10.1007/978-1-4615-3618-5_4.

  28. Chen, G., & Pham, T. T. (2005). Introduction to fuzzy systems. Boca Raton: CRC Press.

    MATH  Google Scholar 

  29. Li, J., Zeng, J., Su, X., Luo, W., & Wang, J. (2012). Self-optimization of coverage and capacity in lte networks based on central control and decentralized fuzzy Q-learning. International Journal of Distributed Sensor Networks. doi:10.1155/2012/878595.

  30. Galindo-Serrano, A., & Giupponi, L. (2014). Self-organized femtocells: A fuzzy Q-learning approach. Wireless Networks, 20(3), 441–455. doi:10.1007/s11276-013-0609-6.

    Article  Google Scholar 

  31. François-Lavet, V., Fonteneau, R., & Ernst, D. (2015). How to discount deep reinforcement learning: Towards new dynamic strategies. arXiv preprint arXiv:151202011.

  32. Piro, G., Grieco, L., Boggia, G., Capozzi, F., & Camarda, P. (2011). Simulating lte cellular systems: An open-source framework. IEEE Transactions on Vehicular Technology, 60(2), 498–513. doi:10.1109/TVT.2010.2091660.

    Article  Google Scholar 

  33. Rada-Vilela, J. (2014). Fuzzylite: A Fuzzy Logic Control Library. http://www.fuzzylite.com.

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communications Engineering, Cairo University, Giza, Egypt

    Rana D. Hegazy, Omar A. Nasr & Hanan A. Kamal

Authors
  1. Rana D. Hegazy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Omar A. Nasr
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hanan A. Kamal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rana D. Hegazy.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hegazy, R.D., Nasr, O.A. & Kamal, H.A. Optimization of user behavior based handover using fuzzy Q-learning for LTE networks. Wireless Netw 24, 481–495 (2018). https://doi.org/10.1007/s11276-016-1348-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-016-1348-2

Keywords

Search

Navigation