Skip to main content
SpringerLink
Log in

Q-learning based heterogenous network self-optimization for reconfigurable network with CPC assistance

  • Published:
Science in China Series F: Information Sciences Aims and scope Submit manuscript

Abstract

With the irreversible trend of the convergence and cooperation among heterogeneous networks, there emerge some important issues for network evolution. One of them is to reconfigure network elements such as cellular base stations (BSs) or access points (APs) of wireless local area networks (WLANs) according to the real-time network environment, in order to maximize the cooperation gain of different networks. In this paper, we consider cognitive pilot channel (CPC) as an assistant to enable cooperation among heterogeneous networks. Based on the widely used reinforcement learning algorithm, this paper has proposed the heterogeneous network self-optimization algorithm (HNSA) to solve the adaptation problem in reconfigurable systems. In the algorithm, distributed agents perform reinforcement learning, and make decisions cooperatively with the help of CPC in order to reduce the system blocking rate and improve network revenue. Finally our simulation proves the anticipated goal is achieved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wysocki T A, Dadejand A Wysocki B J, et al. Advanced wired and wireless networks. Multimedia Syst Appl, 2004, 26: 39–56, 81–104

    Google Scholar 

  2. Demestichas P, Dimitrakopoulos G, Tsagkaris K, et al. Reconfigurations selection in cognitive, beyond 3G, radio infrastructures. In: Proc Int Conf Cognitive Radio Oriented Wireless Networks and Commun, Mykonos Island, June 2006. 1–5

  3. Bogenfeld E, Gaspard I. Self-x in radio access networks. E3 White Paper v1.0. 22 Dec, 2008

  4. Gandalf Homepage: http://www.celtic-gandalf.org/

  5. European Commission. The Future of the Internet. Belgium, 2008

  6. Next Generation Mobile Network. Next generation mobile networks beyond HSPA & EVDO. White Paper, Release 3.0. December 2006. http://www.ngmn.org/

  7. IEEE. IEEE 802.16m system requirements (SRD). IEEE 802.16m, IEEE 802.16m-07/002r8. 15th Jan, 2009

  8. IEEE. Draft IEEE 802.16m system description document (SDD). IEEE 802.16m, IEEE 802.16m-08/003r7. 7th Feb, 2009

  9. 3GPP. Self-organizing networks (SON), concepts and requirements. 3GPP TS 32.500 v8.0.0, 3GPP Telecommunication Management, Release 8. Dec 2008

  10. 3GPP. Self-configuring and self-optimizing network use cases and solutions. 3GPP TR 36.902 v1.0.1, 3GPP TSG RAN and E-UTRAN, Release 8. Sep 2008

  11. Ghosh D, Sharman R, Rao H R, et al. Self-healing systemssurvey and synthesis. Decis Supp Syst, 2007, 42(4): 2164–2185

    Article  Google Scholar 

  12. Perez-Romero J, Sallent O, Agusti R, et al. A novel on-demand cognitive pilot channel enabling dynamic spectrum allocation. 2nd ed. In: IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, Dublin, April 2007. 46–54

  13. European Commission, the seventh Frame Programme (FP7). ICT-2007-216248 E3 (End-to-End Efficiency) Project. http://www.ict-e3.eu/.

  14. European Commission, the sixth Frame Programme (FP6). IST-2003-507995 E2R (End-to-End Reconfigurability) Project. http://e2r.motlabs.com/.

  15. European Commission, the sixth Frame Programme (FP6). Dynamic network planning and management. E2R White Paper. Aug, 2005

  16. Moessner K, Rodriguez V, Dimitrakopoulos G, et al. Dynamic radio resource allocation strategies and time scales. In: SDR Forum Technical Conf. Orange County, Nov, 2005

  17. Demestichas P, Dimitrakopoulos G, Bourse D, et al. Dynamic planning and management of reconfigurable systems. In: Proc IEEE Int Symp New Frontiers in Dynamic Spectrum Access Networks, Baltimore, Nov 2005. 568–572

  18. Zhang Y, Tang T, Chen J, et al. Autonomic joint session admission control using reinforcement learning. J Beijing Univ Posts Telecom, 2007, 30(4): 5–9

    MathSciNet  Google Scholar 

  19. Huang B, Cao G, Wang Z. Reinforcement learning theory, algorithms and application. J Hebei University Tech, 2006, 35(6): 34–38

    MATH  Google Scholar 

  20. Gao Y, Chen S, Lu X. A review of reinforcement learning. Acta Automa Sin, 2004, 30(1): 86–100

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of Universal Wireless Communications, Ministry of Education, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    ZhiYong Feng, Li Tan & Ping Zhang

  2. China Mobile Group Beijing Co., Ltd., Beijing, 100140, China

    LiTao Liang

Authors
  1. ZhiYong Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. LiTao Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ping Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to ZhiYong Feng.

Additional information

Supported by the E3 Project within Community’s Seventh Framework Program (Grant No. FP7-ICT-2007-216248), the National Natural Science Foundation of China (Grant Nos. 60832009, 60632030) and the National Basic Research Program of China (Grant No. 2009CB320406)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Feng, Z., Liang, L., Tan, L. et al. Q-learning based heterogenous network self-optimization for reconfigurable network with CPC assistance. Sci. China Ser. F-Inf. Sci. 52, 2360–2368 (2009). https://doi.org/10.1007/s11432-009-0223-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-009-0223-5

Keywords

Search

Navigation