Skip to main content
SpringerLink
Log in

A Q Learning and Fuzzy Q Learning Approach for Optimization of Interference Constellations in Femto–Macro Cellular Architecture in Downlink

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

The reduction in the cost and enhancement in the network coverage and capacity are the main objectives in the establishment of mobile networks. These objectives were the key force behind the idea of femtocells deployment. But, there are many technical issues for the femtocells deployment with already existing macro network. Cross-tier interference is one of the main challenge that must have to be resolved for smooth operation of macro–femto network. This paper gives self-optimizing and self-healing technique that utilizes Q-learning and fuzzy Q-learning algorithm for the objectives of enhancement in the network capacity and spectral efficiency. In our proposed scheme, each macro base station acts as an agent which interacts with its local environment (all the femtocells and mobile stations under its coverage area), gathers the information and takes the suitable actions correspondingly. For the objective of controlling cross-tier interference, macrousers are rescheduled in such an intelligent way that performance of the femtousers, located on the overlapped spectral portion, is not degraded. The simulation results confirm our proposed approach to improve the network capacity and spectral efficiency as well as sharp convergence, which designates its capability to meet the self organizing network requirements set by 3GPP.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. 3GPP TS 32.500 (2009). Self-organizing networks (SON): Concepts and requirements (release 9), V9.0.0.

  2. 3GPP TR 36.902 (2011). E-UTRA; Self-configuring and self-optimizing network (SON) use cases and solutions (Release 9), V9.3.1.

  3. Han, Q., Ma, K., Liu, Z., & Guan, X. (2012). Power control based on maximum power adaptation in two-tier femtocell networks. Wireless Personal Communications, 70(1), 331–351.

    Article  Google Scholar 

  4. Saquib, N., Hossain, E., Le, L. B., & Kim, D. I. (2012). Interference management in OFDMA femtocell networks: Issues and approaches. IEEE Wireless Communications, 19(3), 86–95.

    Article  Google Scholar 

  5. Dohler, M., Giupponi, L., Galindo-Serrano, A., & Blasco, P. (2010). Docitive networks: A novel framework beyond cognition. IEEE Communications Society, Multimdia Communications TC, E-Letter, 5(1), 1–3.

    Google Scholar 

  6. Giupponi, L., Galindo-Serrano, A., Blasco, P., & Dohler, M. (2010). Docitive networks: An emerging paradigm for dynamic spectrum management. IEEE Wireless Communications Magazine 2010, Special Issue on Dynamic Spectrum Management, 17(4), 47–54.

    Google Scholar 

  7. Shahid, A., Aslam, S., Kim, H. S., & Lee, K. G. (2015). A docitive Q-learning approach towards joint resource allocation and power control in self-organised femtocell networks. Transaction on Emerging Telecommunications Technologies, 26(2), 216–230.

    Article  Google Scholar 

  8. Zheng, Z., Dowhuszko, A., & Hmlinen, J. (2013). Interference management for LTE-Advanced Het-Nets: stochastic scheduling approach in frequency domain. Transaction on Emerging Telecommunications Technologies, 24(1), 4–17.

    Article  Google Scholar 

  9. Galindo-Serrano, A., Giupponi, L., & Dohler, M. (2010). Cognition and docition in OFDMA-based femtocell networks. In IEEE global telecommunications conference (GLOBECOM 2010) (pp. 1–6).

  10. Kim, T., & Lee, T. (2008) Throughput enhancement of macro and femto networks by frequency reuse and pilot sensing. In 2008 Proceeding of IEEE international performance, computing and communications conference (IPCCC 2008) (pp. 390–394).

  11. Poongup, L., Taeyoung, L., Jangkeun, J., & Jitae S. (2010) Interference management in LTE femtocell systems using fractional frequency reuse. In 12th International conference on advanced communication technology (Vol. 2, pp. 1047–1051).

  12. Oh, C., Chung, M. Y., Choo, H., & Lee, T. (2013). Resource allocation with partitioning criterion for macro-femto overlay cellular networks with fractional frequency reuse. Wireless Personal Communications, 68(2), 417–432.

    Article  Google Scholar 

  13. Simsek, M., Bennis, M., & Czylwiki, A. (2012). Dynamic Inter-cell interference coordination in HetNets: A reinforcement learning approach. In Wireless networking symposium—Globecom.

  14. Ghaffar, R., & Ho, P.-H. (2013). Cross-tier interference mitigation in Femto–macro cellular architecture in downlink. In 2013 IEEE international conference on communications (ICC 2013) (pp. 5015–5020).

  15. 3GPP TR 36.913. Technical specification group radio access network. In Requirements for further advancements for evolved universal terrestrial radio access (EUTRA).

  16. Sutton, R., & Barto, A. (1998). Reinforcement learning: An introduction. Cambridge: MIT Press.

    Google Scholar 

  17. Matignon, L., Laurent, G., & Le Fort-Piat, N. (2007). Hysteretic Q-learning: An algorithm for decentralized reinforcement learning in cooperative multi-agent teams. In IEEE international conference on intelligent robots and systems, IROS 2007 (pp. 64–69).

  18. Naeeni, A.F. Advanced multi-agent fuzzy reinforcement learning. Master of Science in Computer Engineering, Dalarna University, Rödavägen 3, S-781 88 Borlänge, Sweden.

  19. Naeeni, A.F. (2008). X2 application protocol (X2AP) (release 8)”. 3GPP TS 36.423 V8.2.0 (2008-06).

  20. Park, S., Seo, W., Kim, Y., Lim, S., & Hong, D. (2010). Beam subset selection strategy for interference reduction in two-tier femtocell networks. IEEE Transactions on Wireless Communications, 9(11), 3440–3449.

    Article  Google Scholar 

  21. Simsek, M., & Czylwik, A. (2012). Decentralized Q-learning of LTE-femtocells for interference reduction in heterogeneous networks using cooperation. In International ITG workshop on smart antennas (WSA).

  22. Galindo-Serrano, A., & Giupponi, L. (2014). Self-organized femtocells: A fuzzy Q-learning approach. Wireless Networks, 20(3), 441–455.

    Article  Google Scholar 

  23. Dirani,  M., & Altman, Z. (2011). Self-organizing networks in next generation radio access networks: Application to fractional power control. Computer Networks, 55(2), 431–438.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, MCS, National University of Sciences and Technology, Islamabad, Pakistan

    Muhammad Waseem Akhtar & Imran Rashid

  2. Intel Mobile Communications, Santa Clara, CA, USA

    Rizwan Ghaffar

Authors
  1. Muhammad Waseem Akhtar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rizwan Ghaffar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Imran Rashid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Waseem Akhtar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Akhtar, M.W., Ghaffar, R. & Rashid, I. A Q Learning and Fuzzy Q Learning Approach for Optimization of Interference Constellations in Femto–Macro Cellular Architecture in Downlink. Wireless Pers Commun 88, 797–817 (2016). https://doi.org/10.1007/s11277-016-3206-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-016-3206-z

Keywords

Search

Navigation