Skip to main content
Springer Nature Link
Log in

Learning to Solve Decision Problems Over Two Timescales: An Application to 5G Puncturing

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

One of the biggest innovations on 5G and beyond is the support of three different services with particular delay and bandwidth requirements, such as Massive Machine Type Communications (MMTC), enhanced Mobile Broad Band (eMBB) and Ultra-Reliable Low Latency Communications (URLLC). In order to achieve these multiple service requirements, all users have to share resources over the 5G Orthogonal Frequency-Division Multiple Access (OFDMA) frame. One of the strategies proposed by the 5G standard is puncturing, which allows the scheduler to assign eMBB services on a timescale, and on a shorter timescale to preemptively overwrite part of the eMBB assignment when a URLLC user arrives. The optimization of puncturing poses a challenging problem: the optimal allocation depends on traffic arriving over different timescales, which forces the scheduler to make allocation decisions without knowledge of future users’ demands, all while having to satisfy several strong constraints. This kind of multiple timescales optimization with restrictions is also to be found in many interesting problems, such as energy management. We propose a learning mechanism where the system learns offline the optimal allocation according to the network state. This learned estimation is then used online to determine the optimal allocation. Through simulations, we verify that the proposed learning strategy provides results close to the optimal policy, improving state of the art proposals for puncturing schemes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

All data generated and code can be found at https://gitlab.fing.edu.uy/ai45g/puncturing-ai45g.

Notes

  1. All code can be found at https://gitlab.fing.edu.uy/ai45g/puncturing-ai45g.

References

  1. Kaur, J., Khan, M. A., Iftikhar, M., Imran, M., & Emad UL Haq, Q. (2021). Machine learning techniques for 5g and beyond. IEEE Access, 9, 23472–23488. https://doi.org/10.1109/ACCESS.2021.3051557

    Article  Google Scholar 

  2. Morocho-Cayamcela, M. E., Lee, H., & Lim, W. (2019). Machine learning for 5g/b5g mobile and wireless communications: Potential, limitations, and future directions. IEEE Access, 7, 137184–137206.

    Article  Google Scholar 

  3. Klautau, A., Batista, P., González-Prelcic, N., Wang, Y., & Heath, R.W. (2018) 5g mimo data for machine learning: Application to beam-selection using deep learning. In: 2018 Information Theory and Applications Workshop (ITA), pp. 1–9. IEEE

  4. Huang, H., Guo, S., Gui, G., Yang, Z., Zhang, J., Sari, H., & Adachi, F. (2019). Deep learning for physical-layer 5g wireless techniques: Opportunities, challenges and solutions. IEEE Wireless Communications, 27(1), 214–222.

    Article  Google Scholar 

  5. Hasan, M.K., Shahjalal, M., Islam, M.M., Alam, M.M., Ahmed, M.F., & Jang, Y.M. (2020) The role of deep learning in noma for 5g and beyond communications. In: 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), pp. 303–307. IEEE

  6. Li, J., Liu, M., Xue, Z., Fan, X., & He, X. (2020). Rtvd: A real-time volumetric detection scheme for ddos in the internet of things. IEEE Access, 8, 36191–36201.

    Article  Google Scholar 

  7. Wang, Y., Li, P., Jiao, L., Su, Z., Cheng, N., Shen, X. S., & Zhang, P. (2016). A data-driven architecture for personalized qoe management in 5g wireless networks. IEEE Wireless Communications, 24(1), 102–110.

    Article  Google Scholar 

  8. Martin, A., Egaña, J., Flórez, J., Montalbán, J., Olaizola, I. G., Quartulli, M., Viola, R., & Zorrilla, M. (2018). Network resource allocation system for qoe-aware delivery of media services in 5g networks. IEEE Transactions on Broadcasting, 64(2), 561–574. https://doi.org/10.1109/TBC.2018.2828608

    Article  Google Scholar 

  9. Ji, H., Park, S., Yeo, J., Kim, Y., Lee, J., & Shim, B. (2018). Ultra-reliable and low-latency communications in 5g downlink: Physical layer aspects. IEEE Wireless Communications, 25(3), 124–130.

    Article  Google Scholar 

  10. 3GPP (2018) Etsi tr 138 912 5g; study on new radio (nr) access technology. ETSI version 15.0.0(Release 15)

  11. Popovski, P., Trillingsgaard, K. F., Simeone, O., & Durisi, G. (2018). 5g wireless network slicing for embb, urllc, and mmtc: A communication-theoretic view. IEEE Access, 6, 55765–55779.

    Article  Google Scholar 

  12. Pedersen, K., Pocovi, G., Steiner, J., & Maeder, A. (2018). Agile 5g scheduler for improved e2e performance and flexibility for different network implementations. IEEE Communications Magazine, 56(3), 477–490.

    Article  Google Scholar 

  13. Pedersen, K., Pocovi, G., Steiner, J., & Maeder, A. (2018). Agile 5g scheduler for improved e2e performance and flexibility for different network implementations. IEEE Communications Magazine, 56(3), 210–217. https://doi.org/10.1109/MCOM.2017.1700517

    Article  Google Scholar 

  14. Pocovi, G., Pedersen, K. I., & Mogensen, P. (2018). Joint link adaptation and scheduling for 5g ultra-reliable low-latency communications. IEEE Access, 6, 28912–28922. https://doi.org/10.1109/ACCESS.2018.2838585

    Article  Google Scholar 

  15. Esswie, A.A., & Pedersen, K.I. (2018) Multi-user preemptive scheduling for critical low latency communications in 5g networks. In: 2018 IEEE Symposium on Computers and Communications (ISCC), pp. 00136–00141. https://doi.org/10.1109/ISCC.2018.8538471

  16. Bairagi, A.K., Munir, M.S., Alsenwi, M., Tran, N.H., & Hong, C.S. (2019) A matching based coexistence mechanism between embb and urllc in 5g wireless networks. In: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing. SAC ’19, pp. 2377–2384. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3297280.3297513.

  17. Alsenwi, M., Tran, N. H., Bennis, M., Pandey, S. R., Bairagi, A. K., & Hong, C. S. (2021). Intelligent resource slicing for embb and urllc coexistence in 5g and beyond: A deep reinforcement learning based approach. IEEE Transactions on Wireless Communications, 20(7), 4585–4600. https://doi.org/10.1109/TWC.2021.3060514

    Article  Google Scholar 

  18. Elsayed, M., & Erol-Kantarci, M. (2019) Ai-enabled radio resource allocation in 5g for urllc and embb users. In: 2019 IEEE 2nd 5G World Forum (5GWF), pp. 590–595. https://doi.org/10.1109/5GWF.2019.8911618

  19. Almekhlafi, M., Arfaoui, M. A., Elhattab, M., Assi, C., & Ghrayeb, A. (2022). Joint resource allocation and phase shift optimization for ris-aided embb/urllc traffic multiplexing. IEEE Transactions on Communications, 70(2), 1304–1319. https://doi.org/10.1109/TCOMM.2021.3127265

    Article  Google Scholar 

  20. Anand, A., De Veciana, G., & Shakkottai, S. (2018) Joint scheduling of urllc and embb traffic in 5g wireless networks, pp. 1970–1978. https://doi.org/10.1109/INFOCOM.2018.8486430

  21. Otsuka, H., Tian, R., & Senda, K. (2019). Transmission performance of an ofdm-based higher-order modulation scheme in multipath fading channels. Journal of Sensor and Actuator Networks, 8, 19. https://doi.org/10.3390/jsan8020019

    Article  Google Scholar 

  22. Carpentier, P., Chancelier, J.-P., de Lara, M., & Rigaut, T. (2019) Algorithms for two-time scales stochastic optimization with applications to long term management of energy storage. working paper or preprint

  23. Xu, Y., Gui, G., Gacanin, H., & Adachi, F. (2021). A survey on resource allocation for 5g heterogeneous networks: Current research, future trends, and challenges. IEEE Communications Surveys Tutorials, 23(2), 668–695. https://doi.org/10.1109/COMST.2021.3059896

    Article  Google Scholar 

  24. Zhang, D., Qiao, Y., She, L., Shen, R., Ren, J., & Zhang, Y. (2019). Two time-scale resource management for green internet of things networks. IEEE Internet of Things Journal, 6(1), 545–556. https://doi.org/10.1109/JIOT.2018.2842766

    Article  Google Scholar 

  25. Chen, T., Zhang, X., You, M., Zheng, G., & Lambotharan, S. (2022). A gnn-based supervised learning framework for resource allocation in wireless iot networks. IEEE Internet of Things Journal, 9(3), 1712–1724. https://doi.org/10.1109/JIOT.2021.3091551

    Article  Google Scholar 

  26. Bao, Z., Zhou, Q., Yang, Z., Yang, Q., Xu, L., & Wu, T. (2015). A multi time-scale and multi energy-type coordinated microgrid scheduling solution-part i: Model and methodology. IEEE Transactions on Power Systems, 30(5), 2257–2266. https://doi.org/10.1109/TPWRS.2014.2367127

    Article  Google Scholar 

  27. Liu, Z., Wu, Q., Ma, K., Shahidehpour, M., Xue, Y., & Huang, S. (2019). Two-stage optimal scheduling of electric vehicle charging based on transactive control. IEEE Transactions on Smart Grid, 10(3), 2948–2958. https://doi.org/10.1109/TSG.2018.2815593

    Article  Google Scholar 

  28. Bengio, Y., Lodi, A., & Prouvost, A. (2021). Machine learning for combinatorial optimization: A methodological tour d’horizon. European Journal of Operational Research, 290(2), 405–421. https://doi.org/10.1016/j.ejor.2020.07.063

    Article  MathSciNet  MATH  Google Scholar 

  29. Eisen, M., Zhang, C., Chamon, L. F. O., Lee, D. D., & Ribeiro, A. (2019). Learning optimal resource allocations in wireless systems. IEEE Transactions on Signal Processing, 67(10), 2775–2790. https://doi.org/10.1109/TSP.2019.2908906

    Article  MathSciNet  MATH  Google Scholar 

  30. Aitchison, J. (1982). The statistical analysis of compositional data. Journal of the Royal Statistical Society Series B Methodological, 44(2), 139–177.

    MathSciNet  MATH  Google Scholar 

  31. Zhang, Q., & Kassam, S. A. (1999). Finite-state markov model for rayleigh fading channels. IEEE Transactions on Communications, 47(11), 1688–1692. https://doi.org/10.1109/26.803503

    Article  Google Scholar 

  32. Diamond, S., & Boyd, S. (2016). CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Research, 17(83), 2909–2913.

    MathSciNet  MATH  Google Scholar 

  33. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.

    MathSciNet  MATH  Google Scholar 

  34. Chollet, F., et al. (2015). Keras. https://keras.io

Download references

Funding

This work is partially funded by the Agencia Nacional de Investigación e Innovación (ANII) project ‘Artificial Intelligence for 5G networks’ (FMV_1_2019_1_155700). Martín Randall’s PhD on which this article is inscribed has the support of a scholarship granted by the Agencia Nacional de Investigación e Innovación (ANII).

Author information

Authors and Affiliations

  1. Facultad de Ingeniería, Instituto de Ingeniería Eléctrica, Universidad de la República, J. Herrera y Reissig 565, Montevideo, 11200, Uruguay

    Martin Randall, Gonzalo Belcredi, Pablo Belzarena & Federico Larroca

Authors
  1. Martin Randall
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Gonzalo Belcredi
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Pablo Belzarena
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Federico Larroca
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by MR, supported by GB, and advised by PB and FL. The first draft of the manuscript was written by MR, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Martin Randall.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Randall, M., Belcredi, G., Belzarena, P. et al. Learning to Solve Decision Problems Over Two Timescales: An Application to 5G Puncturing. Wireless Pers Commun 132, 2603–2623 (2023). https://doi.org/10.1007/s11277-023-10735-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-023-10735-3

Keywords