Abstract
This study investigates the problem of joint channel and power allocation in stochastic underwater acoustic communication networks. And the multiarmed bandit theory is employed to model this problem which includes unknown variables. This study presents two two-tier learning algorithms, which do not need any prior environment information. In the upper learning, the user plays the predicted best strategy and learns the actual played strategy. In the lower learning, “outdated virtual learning information,” which can be obtained by the information of actual played strategy, is learned. The two-tier actual–virtual learning enormously enriches the learning information and effectively improves the learning ability. And multidimensional learning method is presented to ease the difficulty caused by the coupling of joint strategy. With the evolution of learning time, the emphasis of learning transfers from the channel power sub-strategy dimension to the entire power strategy dimension. Due to the specific learning manner, the algorithms have high tolerance about delay and non-complete information. Simulation results show high performance and adaptability of the proposed learning algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akyildiz IF, Pompili D, Melodia T (2004) Challenges for efficient communication in underwater acoustic sensor networks. Acm Sigbed Rev 1(2):3–8
Akyildiz IF, Pompili D, Melodia T (2005) Underwater acoustic sensor networks: research challenges. Ad Hoc Netw 3(3):257–279
Aval YM, Wilson SK, Stojanovic M (2015) On the achievable rate of a class of acoustic channels and practical power allocation strategies for ofdm systems. IEEE J Oceanic Eng 40(4):785–795
Bubeck S, Cesabianchi N (2012) Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Found Trends Mach Learn 5(1):101–112
Coucheney P, Khawam K, Cohen J (2015) Multi-armed bandit for distributed inter-cell interference coordination. In: IEEE International Conference on Communications, pp 3323–3328
Gai Y, Krishnamachari B (2012) Online learning algorithms for stochastic water-filling. In: Information Theory and Applications Workshop, pp 352–356
Gai Y, Krishnamachari B (2014) Distributed stochastic online learning policies for opportunistic spectrum access. IEEE Trans Signal Process 62(23):6184–6193
Gai Y, Krishnamachari B, Jain R (2010) Learning multiuser channel allocations in cognitive radio networks: A combinatorial multi-armed bandit formulation. In: New Frontiers in Dynamic Spectrum, 2010 IEEE Symposium on, pp 1–9
Guruacharya S, Niyato D, Dong IK, Hossain E (2013) Hierarchical competition for downlink power allocation in ofdma femtocell networks. IEEE Trans Wireless Commun 12(4):1543–1553
Han G, Jiang J, Sun N, Shu L (2015) Secure communication for underwater acoustic sensor networks. IEEE Commun Mag 53(8):54–60
Han J, Zhang L, Leus G (2016a) Partial fft demodulation for mimo-ofdm over time-varying underwater acoustic channels. IEEE Signal Process Lett 23(2):282–286
Han S, Li X, Liu Z, Guan X (2016b) Hierarchical-game-based algorithm for downlink joint subchannel and power allocation in ofdma femtocell networks. J Netw Comput Appl 73:44–56
Jornet JM, Stojanovic M, Zorzi M (2010) On joint frequency and power allocation in a cross-layer protocol for underwater acoustic networks. IEEE J Oceanic Eng 35(4):936–947
Kumar P, Trivedi VK, Kumar P (2015) Recent trends in multicarrier underwater acoustic communications. In: Underwater Technology, pp 1–8
Leinhos HA (1996) Capacity calculations for rapidly fading communications channels. IEEE J Oceanic Eng 21(2):137–142
Li X, Liu J, Yan L, Han S, Guan X (2017) Relay selection in underwater acoustic cooperative networks: a contextual bandit approach. IEEE Commun Lett 21(2):382–385
Liu J, Wang Z, Zuba M, Peng Z, Cui JH, Zhou S (2014) Da-sync: a dopplerassisted time-synchronization scheme for mobile underwater sensor networks. IEEE Trans Mob Comput 13(3):582–595
Maghsudi S, Staczak S (2015) Joint channel selection and power control in infrastructureless wireless networks: a multiplayer multiarmed bandit framework. IEEE Trans Veh Technol 64(10):4565–4578
Morvari F, Ghasemi A (2016) Two-stage resource allocation for random access m2m communications in lte network. IEEE Commun Lett 20(5):982–985
Ng DWK, Lo ES, Schober R (2016) Multiobjective resource allocation for secure communication in cognitive radio networks with wireless information and power transfer. IEEE Trans Veh Technol 65(5):3166–3184
Polprasert C, Ritcey JA, Stojanovic M (2011) Capacity of ofdm systems over fading underwater acoustic channels. IEEE J Oceanic Eng 36(4):514–524
Qarabaqi P, Stojanovic M (2013) Statistical characterization and computationally efficient modeling of a class of underwater acoustic communication channels. IEEE J Oceanic Eng 38(4):701–717
Roy S, Duman TM, Mcdonald V, Proakis JG (2007) High-rate communication for underwater acoustic channels using multiple transmitters and space time coding: receiver structures and experimental results. IEEE J Oceanic Eng 32(3):663–688
Shum KW, Leung KK, Chi WS (2007) Convergence of iterative waterfilling algorithm for gaussian interference channels. IEEE J Sel Areas Commun 25(6):1091–1100
Son K, Lee S, Yi Y, Song C (2011) Refim: a practical interference management in heterogeneous wireless access networks. IEEE J Sel Areas Commun 29(6):1260–1272
Sozer EM, Stojanovic M, Proakis JG (2000) Underwater acoustic networks. IEEE J Oceanic Eng 25(1):72–83
Su Y, Zhu Y, Mo H, Cui JH, Jin Z (2015) A joint power control and rate adaptation mac protocol for underwater sensor networks. Ad Hoc Netw 26:36–49
Tuna G, Gungor VC (2017) A survey on deployment techniques, localization algorithms, and research challenges for underwater acoustic sensor networks. Int J Commun Syst. https://doi.org/10.1002/dac.3350
Walree PAV (2013) Propagation and scattering effects in underwater acoustic communication channels. IEEE J Oceanic Eng 38(4):614–631
Wang W, Kwasinski A, Niyato D, Han Z (2016) Learning for robust routing based on stochastic game in cognitive radio networks. arXiv preprint arXiv:160308664
Wen M, Cheng X, Yang L, Li Y, Cheng X, Ji F (2016) Index modulated ofdm for underwater acoustic communications. IEEE Commun Mag 54(5):132–137
Funding
This study was funded by the Natural Science Foundation of China (Grant No. 61571387 and No. 61473247).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Han, S., Li, X., Yan, L. et al. MAB-based two-tier learning algorithms for joint channel and power allocation in stochastic underwater acoustic communication networks. Soft Comput 23, 7181–7192 (2019). https://doi.org/10.1007/s00500-018-3357-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-018-3357-9