Skip to main content
SpringerLink
Log in

Managing Femto to Macro Interference without X2 Interface Support through POMDP

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

We present a self-organized downlink power control for interference management when Home eNodeBs (HeNBs) work in co-channel operation with the macrocell system. The main novelty with regards to previous works is that we provide a completely autonomous framework, considering 3GPP release 11 hypothesis of non availability of X2 interface between evolved NodeBs (eNBs) and HeNBs. In this situation, the HeNB has to make autonomous decisions without receiving any feedback from the macro network. We model the HeNBs as a multiagent system where each node is an independent agent able to learn through Reinforcement Learning (RL) techniques a downlink power allocation policy for different interference situations. To deal with the lack of information in the scenario, we rely on the theory of Partially Observable Markov Decision Process (POMDP). POMDP works on the basis of a set of beliefs that the HeNB builds considering the impact it causes to the macrocell system. To gather this system performance information, we propose that HeNBs use spatial interpolation techniques, such as ordinary Kriging. Results show that the proposed approach allows HeNBs to autonomously learn a power allocation policy to coexist with the macro network, in a 3GPP compliant fashion, and without introducing overhead signaling in the system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Saunders SR, Carlaw S, Giustina A, Bhat RR, Rao VS, Siegberg R (2009) Femtocells: oppotunities and challenges for business and technology. Wiley, Great Britain

    Google Scholar 

  2. 3GPP (2008) Telecommunication management; self-organizing networks (SON); concepts and requirements (release 8). 3GPP TS 32.500

  3. Zhang J, de la Roche G (2010) Femtocells: technologies and deployment. Wiley, New York

    Book  Google Scholar 

  4. He G, Betz S, Debbah M (2009) Game-theoretic deployment design of small-cell OFDM networks. In: Proc. of the 3rd ICST/ACM international workshop on game theory in communication networks, Pisa, Italy

    Google Scholar 

  5. Altman E, Koole G (1993) Stochastic scheduling games with markov decision arrival processes. J Comput Math Appl 26(6):141–148

    Article  MathSciNet  MATH  Google Scholar 

  6. Fudenberg D, Levine DK (1998) The theory of learning in games, vol 1. MIT Press, Cambridge, MA

    Google Scholar 

  7. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge, MA

    Google Scholar 

  8. Sycara KP (1998) Multiagent systems. AI Mag 19(2):79–92

    Google Scholar 

  9. Galindo-Serrano A, Giupponi L (2010) Distributed Q-learning for interference control in OFDMA-based femtocell networks. In: Proc. of the IEEE 71th vehicular technology conference (VTC spring 2010), Taipei, Taiwan, 16–19 May 2010

  10. Galindo-Serrano A, Giupponi L, Auer G (2011) Distributed femto-to-macro interference management in multiuser OFDMA networks. In: Proc. of IEEE 73rd vehicular technology conference (VTC2011-Spring), workshop on broadband femtocell technologies, Budapest, Hungary, 15–18 May 2011

  11. Galindo-Serrano A, Giupponi L, Dohler M (2010) Cognition and docition in OFDMA-based femtocell networks. In: Proceedings of IEEE global communications conference (IEEE GLOBECOM), Miami, USA, 6–10 Dec 2010

  12. Galindo-Serrano A, Giupponi L (2011) Downlink femto-to-macro interference management based on fuzzy Q-learning. In: Proc. of the third IEEE international workshop on indoor and outdoor femto cells (IOFC’11), Princeton, USA, 13 May 2011

  13. 3GPP (2011) X2 application protocol (X2AP) (release 10). 3GPP TS 36.423 V10.1.0 (2010-03)

  14. 3GPP (2007) X2 general aspects and principles (release 8). 3GPP TS 36.420 V8.0.0 (2007-12)

  15. 3GPP (2010) Evolved universal terrestrial radio access (E-UTRA); physical layer procedures (release 9). 3GPP TS 36.213 V 9.2.0 (2010-06)

  16. Oh S, Kim H, Ryu B, Park N (2011) Inbound mobility management on LTE-Advanced femtocell topology using X2 interface. In: Proc. of 2011 20th international conference on computer communications and networks (ICCCN), Maui, Hawaii, 31 July–4 Aug 2011

  17. Liu W, Hu C, Wei D, Peng M, Wang W (2010) An overload indicator & high interference indicator hybrid scheme for inter-cell interference coordination in LTE system. In: Proc. of the 2010 3rd IEEE international conference on broadband network and multimedia technology (IC-BNMT), Beijing, China, 26–28 Oct 2010, pp 514–518

  18. Combes R, Haddad ZAM, Altman E (2011) Self-optimizing strategies for interference coordination in OFDMA networks. In: Proc. of the 2011 IEEE international conference on communications workshops (ICC), Kyoto, Japan, 5–9 June 2011

  19. 3GPP (2012) Evolved universal terrestrial radio access (E-UTRA); overall description; stage 2 (release 11). 3GPP TS 36.300 V11.1.0 (2012-03)

  20. Littman ML, Cassandra AR, Kaelbling LP (1995) Learning policies for partially observable environments: scaling up. In: Proc. of the twelfth international conference on machine learning, Tahoe City, California, USA, pp 362–370

  21. Udenze A, McDonald-Maier K (2008) Partially observable markov decision process for transmitter power control in wireless sensor networks. In: Proc. of the bio-inspired learning and intelligent systems for security BLISS ’08. ECSIS Symposium on, Edinburgh, 4–6 Aug 2008, pp 101–106

  22. Zhao Q, Tong L, Swami A, Chen Y (2007) Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: a POMDP framework. IEEE J Sel Areas Commun 25:589–600

    Article  Google Scholar 

  23. Unnikrishnan J, Veeravalli VV (2008) Dynamic spectrum access with learning for cognitive radio. In: Proc. of the signals, systems and computers, 2008 42nd Asilomar Conference on, Pacific Grove, CA, 26–29 Oct 2008, pp 103–107

  24. Ma R-T, Hsu Y-P, Feng K-T (2009) A POMDP-based spectrum handoff protocol for partially observable cognitive radio networks. In: Proc. of the 2009 IEEE conference on wireless communications & networking conference (WCNC’09). IEEE Press, Budapest, Hungary, pp 1331–1336

    Google Scholar 

  25. Rathnasabapathy, B. & Gmytrasiewicz, P (2003). Formalizing multi-agent POMDPs in the context of network routing. In, Proc. of the 36th Hawaii international conference on system sciences (HICSS’03), vol 9. Los Alamitos: IEEE Computer Society Press.

    Google Scholar 

  26. Karmokar AK, Djonin DV, Bhargava VK (2006) POMDP-based coding rate adaptation for type-I hybrid ARQ systems over fading channels with memory. IEEE Trans Wirel Commun 5(12):3512–3523

    Article  Google Scholar 

  27. Bohling G (2005) Kriging. C&PE 940

  28. The BeFEMTO (2010) Website. http://www.ict-befemto.eu/

  29. D2.1 (2010) Description of baseline reference systems, use cases, requirements, evaluation and impact on business model. EU FP7-ICT BeFEMTO project

  30. D4.1 (2010) Preliminary SON enabling & multi-cell RRM techniques for networked femtocells. EU FP7-ICT BeFEMTO project

  31. 3GPP, 3GPP TS 25.467 V8.1.0., UTRAN architecture for 3G home nodeB (stage 2). Tech. Rep. (2009)

  32. Cai Y, Ding XCP, Jin X, Moorut RP (2010) Mangement interference from femtocells. Patent, US, 12/536,125, 11 Feb 2010

  33. Riihijärvi J, Mähönen P, Wellens M, Gordziel M (2008) Characterization and modelling of spectrum for dynamic spectrum access with spatial statistics and random fields. In: Proceedings of the IEEE 19th international symposium on personal, indoor and mobile radio communications, (PIMRC 2008)

  34. Cressie NA (1993) Statistics for spatial data. Wiley, Great Britain

    Google Scholar 

  35. Bellman R (1957) Dynamic programming. Princeton Univ. Press, Princeton, NJ

    MATH  Google Scholar 

  36. Panait L, Luke S (2005) Cooperative multi-agent learning: the state of the art. Auton Agent Multi-Agent Syst 3(11):383–434

    Google Scholar 

  37. Harmon ME, Harmon SS (2000) Reinforcement learning: a tutorial. [online]. Available: http://www.nbu.bg/cogs/events/2000/Readings/Petrov/rltutorial.pdf

  38. Watkins CJ, Dayan P (1992) Technical note: Q-learning. Mach Learn 8:279–292

    MATH  Google Scholar 

  39. Murphy KP (2000) A survey of POMDP solution techniques. Technical report, UC Berkeley [online]. Available: http://www.cs.ubc.ca/murphyk/papers.html

  40. Ref. TCI6489. [online]. Available: http://www.ti.com/litv/pdf/sprt522a

  41. Ref. TCI6487 [online]. Available: http://www.ti.com/corp/docs/landing/tci6487/flash.htm

  42. TMS320c6416 fixed-point digital signal processor. [online]. Available: http://focus.ti.com/lit/ds/symlink/tms320c6416t.pdf

  43. 3GPP, 3GPP TSG RAN WG4 (radio) meeting 51: Simulation assumptions and parameters for FDD HeNB RF requirements. Tech. rep., 4–8 May 2009

  44. Salo J, Del Galdo G, Salmi J, Kyösti P, Milojevic M, Laselva D, Schneider C (2005) MATLAB implementation of the 3GPP spatial channel model (3GPP TR 25.996). Online, http://www.tkk.fi/Units/Radio/scm/

  45. 3GPP, 3GPP TR (2010) 36.921 evolved universal terrestrial radio access (E-UTRA); FDD home eNode B (HeNB) radio frequency (RF) requirements analysis. 3GPP, Tech. rep.

Download references

Acknowledgements

This work has been partially funded by the Spanish Research Council under SOFOCLES grant (TEC2010-21100), by European Union under ICT project ICT-4-248523 BeFEMTO, ACROPOLIS-NoE (ICT-2009.1.1), COST Action IC0902.

Author information

Authors and Affiliations

  1. Centre Tecnològic de Telecomunicacions de Catalunya (CTTC), Parc Mediterrani de la Tecnologia, Av. Carl Friedrich Gauss 7, Barcelona, 08860, Spain

    Ana Galindo-Serrano & Lorenza Giupponi

Authors
  1. Ana Galindo-Serrano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lorenza Giupponi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana Galindo-Serrano.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Galindo-Serrano, A., Giupponi, L. Managing Femto to Macro Interference without X2 Interface Support through POMDP. Mobile Netw Appl 17, 633–647 (2012). https://doi.org/10.1007/s11036-012-0394-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-012-0394-y

Keywords

Search

Navigation