Skip to main content

Advertisement

SpringerLink
Log in

Deep Q-learning based sparse code multiple access for ultra reliable low latency communication in industrial wireless networks

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

Sparse code multiple access (SCMA) is a technology that allows for extremely low latency and high reliability in modern wireless communication networks. Moreover, due to the sparse layout of its codebooks, SCMA competence for multiple access techniques leads the way for futuristic ultra reliable low latency communications (URLLC), which aims for high reliability with low latency. Therefore, a deep Q learning-based SCMA, called Q-SCMA, is proposed, in which codebooks are adaptively constructed to minimize bit error rate (BER), maximizing throughput within the constraints of reliability and latency, i.e., supporting URLLC. Therefore, the rewards for Q-SCMA are formulated such that, while imposing a latency constraint, they also provide excellent reliability. The performance of the proposed approach is evaluated in terms of BER, loading factor, level of interference, bit error probability, throughput, latency, and computational complexity. It is also analyzed and compared with contemporary approaches, conventional SCMA (C-SCMA) and deep neural network SCMA (D-SCMA), for average white Gaussian noise as well Rayleigh fading channel, and for industrial wireless network environment modified Rician fading channel is considered. Simulation outcomes show considerable performance disparities between the proposed approach and D-SCMA along with C-SCMA. This underlines the requirement for using a deep Q learning model as a performance indicator for developing URLLC supporting SCMA for industrial wireless networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

Not applicable.

References

  1. Chen, Y., Zhu, Q., Feng, C., & Li, X. (2021). Analysis of backlog and delay in downlink power-domain non-orthogonal multiple access wireless networks. Computer Communications, 166, 26–39.

    Article  Google Scholar 

  2. Basharat, M., Ejaz, W., Naeem, M., Khattak, A. M., & Anpalagan, A. (2018). A survey and taxonomy on nonorthogonal multiple-access schemes for 5G networks. Transactions on Emerging Telecommunications Technologies, 29(1), 3202.

    Article  Google Scholar 

  3. Mahmood, A., Zeeshan, M., & Ashraf, T. (2021). A new hybrid CDMA-NOMA scheme with power allocation and user clustering for capacity improvement. Telecommunication Systems, 78(2), 225–237.

    Article  Google Scholar 

  4. Kumar, S., Yadav, P., Kaur, M., & Kumar, R. (2022). A survey on IRS NOMA integrated communication networks. Telecommunication Systems, 80(2), 277–302.

    Article  Google Scholar 

  5. Taherzadeh, M., Nikopour, H., Bayesteh, A., & Baligh, H. (2014). SCMA codebook design. In 2014 IEEE 80th vehicular technology conference (VTC2014-Fall), pp. 1–5. IEEE.

  6. Li, Z., Chen, W., Wei, F., Wang, F., Xu, X., & Chen, Y. (2016). Joint codebook assignment and power allocation for SCMA based on capacity with Gaussian input. In 2016 IEEE/CIC international conference on communications in China (ICCC), pp. 1–6 (2016). IEEE.

  7. Du, Y., Dong, B., Chen, Z., Wang, X., & Gao, P. (2017). Improved serial scheduling-based detection for sparse code multiple access systems. IEEE Wireless Communications Letters, 6(5), 570–573.

    Article  Google Scholar 

  8. Rebhi, M., Hassan, K., Raoof, K., & Chargé, P. (2021). Sparse code multiple access: Potentials and challenges. IEEE Open Journal of the Communications Society, 2, 1205–1238.

    Article  Google Scholar 

  9. Arora, K., Singh, J., & Randhawa, Y. S. (2020). A survey on channel coding techniques for 5G wireless networks. Telecommunication Systems, 73(4), 637–663.

    Article  Google Scholar 

  10. Zhai, D. (2017). Adaptive codebook design and assignment for energy saving in SCMA networks. IEEE Access, 5, 23550–23562.

    Article  Google Scholar 

  11. Ghaffari, A., Leonardon, M., Cassagne, A., Leroux, C., & Savaria, Y. (2019). Toward high-performance implementation of 5G SCMA algorithms. IEEE Access, 7, 10402–10414.

    Article  Google Scholar 

  12. Zhang, C., Patras, P., & Haddadi, H. (2019). Deep learning in mobile and wireless networking: A survey. IEEE Communications Surveys & Tutorials, 21(3), 2224–2287.

    Article  Google Scholar 

  13. Rekkas, V. P., Sotiroudis, S., Sarigiannidis, P., Wan, S., Karagiannidis, G. K., & Goudos, S. K. (2021). Machine learning in beyond 5G/6G networks-state-of-the-art and future trends. Electronics, 10(22), 2786.

    Article  Google Scholar 

  14. Ali, R., Qadri, Y. A., Bin Zikria, Y., Umer, T., Kim, B.-S., & Kim, S. W. (2019). Q-learning-enabled channel access in next-generation dense wireless networks for IoT-based eHealth systems. EURASIP Journal on Wireless Communications and Networking, 2019(1), 1–12.

    Article  Google Scholar 

  15. Ding, H., Zhao, F., Tian, J., Li, D., & Zhang, H. (2020). A deep reinforcement learning for user association and power control in heterogeneous networks. Ad Hoc Networks, 102, 102069.

    Article  Google Scholar 

  16. Nie, J., & Haykin, S. (1999). A Q-learning-based dynamic channel assignment technique for mobile communication systems. IEEE Transactions on Vehicular Technology, 48(5), 1676–1687.

    Article  Google Scholar 

  17. Jang, S.-J., Han, C.-H., Lee, K.-E., & Yoo, S.-J. (2019). Reinforcement learning-based dynamic band and channel selection in cognitive radio ad-hoc networks. EURASIP Journal on Wireless Communications and Networking, 2019(1), 1–25.

    Article  Google Scholar 

  18. Pham, T. M. T., Nguyen, T.-T., & Kim, D.-S. (2017). Geographical awareness hybrid routing protocol in mobile ad hoc networks. Wireless Networks, 23(1), 1–13.

    Article  Google Scholar 

  19. Kiran, N., Pan, C., Wang, S., & Yin, C. (2019). Joint resource allocation and computation offloading in mobile edge computing for SDN based wireless networks. Journal of Communications and Networks, 22(1), 1–11.

    Article  Google Scholar 

  20. Yang, N., Zhang, H., Long, K., Hsieh, H.-Y., & Liu, J. (2019). Deep neural network for resource management in NOMA networks. IEEE Transactions on Vehicular Technology, 69(1), 876–886.

    Article  Google Scholar 

  21. He, C., Hu, Y., Chen, Y., & Zeng, B. (2019). Joint power allocation and channel assignment for NOMA with deep reinforcement learning. IEEE Journal on Selected Areas in Communications, 37(10), 2200–2210.

    Article  Google Scholar 

  22. Lin, J., Feng, S., Zhang, Y., Yang, Z., & Zhang, Y. (2020). A novel deep neural network based approach for sparse code multiple access. Neurocomputing, 382, 52–63.

    Article  Google Scholar 

  23. Kim, M., Kim, N.-I., Lee, W., & Cho, D.-H. (2018). Deep learning-aided SCMA. IEEE Communications Letters, 22(4), 720–723.

    Article  Google Scholar 

  24. Jiang, Z., Chen, S., Molisch, A. F., Vannithamby, R., Zhou, S., & Niu, Z. (2019). Exploiting wireless channel state information structures beyond linear correlations: A deep learning approach. IEEE Communications Magazine, 57(3), 28–34.

    Article  Google Scholar 

  25. Lin, C., Chang, Q., & Li, X. (2019). A deep learning approach for MIMO-NOMA downlink signal detection. Sensors, 19(11), 2526.

    Article  Google Scholar 

  26. Hu, Y., Gursoy, M. C., & Schmeink, A. (2018). Relaying-enabled ultra-reliable low-latency communications in 5G. IEEE Network, 32(2), 62–68.

    Article  Google Scholar 

  27. Bhardwaj, S., & Kim, D.-S. (2021). Dragonfly-based swarm system model for node identification in ultra-reliable low-latency communication. Neural Computing and Applications, 33(6), 1837–1880.

    Article  Google Scholar 

  28. Bhardwaj, S., Ginanjar, R. R., & Kim, D.-S. (2020). Deep Q-learning based resource allocation in industrial wireless networks for URLLC. IET Communications, 14(6), 1022–1027.

    Article  Google Scholar 

  29. Mahmood, N. H., Abreu, R., Böhnke, R., Schubert, M., Berardinelli, G., & Jacobsen, T. H. (2019). Uplink grant-free access solutions for URLLC services in 5G new radio. In 2019 16th International symposium on wireless communication systems (ISWCS), pp. 607–612 (2019). IEEE.

  30. Tominaga, E. N., Alves, H., Souza, R. D., Rebelatto, J. L., & Latva-Aho, M. (2021). Non-orthogonal multiple access and network slicing: Scalable coexistence of eMBB and URLLC. In 2021 IEEE 93rd vehicular technology conference (VTC2021-Spring), pp. 1–6 (2021). IEEE.

  31. Wei, F., & Chen, W.: A low complexity SCMA decoder based on list sphere decoding. In 2016 IEEE global communications conference (GLOBECOM), pp. 1–6 (2016). IEEE.

  32. Nikopour, H., & Baligh, H. (2013). Sparse code multiple access. In 2013 IEEE 24th annual international symposium on personal, indoor, and mobile radio communications (PIMRC), pp. 332–336 (2013). IEEE.

  33. Chu, D. (1972). Polyphase codes with good periodic correlation properties (corresp.). IEEE Transactions on Information Theory, 18(4), 531–532.

  34. Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine learning, 8(3), 279–292.

  35. Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., De Maria, A., Panneershelvam, V., Suleyman, M., Beattie, C., & Petersen, S., et al. (2015). Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv:1507.04296.

  36. Conformance specification Radio, U.E.U. (2011). 3rd generation partnership project; technical specification group radio access network; evolved universal terrestrial radio access (E-UTRA); user equipment (UE) conformance specification radio transmission and reception (2011).

  37. Zhang, Q., Zhang, Q., Zhang, W., Shen, F., Loh, T. H., & Qin, F. (2018). Understanding the temporal fading in wireless industrial networks: Measurements and analyses. In 2018 10th International Conference on wireless communications and signal processing (WCSP), pp. 1–6 (2018). IEEE.

  38. Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379–423.

    Article  Google Scholar 

  39. Rinne, M., Kuusela, M., Tuomaala, E., Kinnunen, P., Kovacs, I., Pajukoski, K., & Ojala, J. (2009). A performance summary of the evolved 3G (E-UTRA) for voice over internet and best effort traffic. IEEE Transactions on Vehicular Technology, 58(7), 3661–3673.

    Article  Google Scholar 

  40. Bertsekas, D., & Gallager, R. (2021). Data networks. Greece: Athena Scientific.

  41. Lagrange, X. (2011). Performance analysis of HARQ protocols with link adaptation on fading channels. Annals of Telecommunications-Annales des Télécommunications, 66(11), 695–705.

  42. Asia, A. I. (2022). Presentation 1st 5g algorithm innovation competition-env1. 0-scma. http://www.innovateasia.com/5g/images/pdf/1st%205G%20Algorithm%20Innovation%20Competition-ENV1. 0.

  43. Brent, R. P., & Zimmermann, P. (2010). Modern computer arithmetic. Cambridge: Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press.

  44. Chen, J., Zhang, Z., Fu, S., & Hu, J. (2017). A joint update parallel MCMC-method-based sparse code multiple access decoder. IEEE Transactions on Vehicular Technology, 67(2), 1280–1291.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Priority Research Centers Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (2018R1A6A1A03024003) and by the MSIT (Ministry of Science and ICT), Korea, under the Innovative Human Resource Development for Local Intellectualization support program (IITP-2023-2020-0-01612) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation).

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of IT Convergence Engineering, ICT Convergence Research Center, Kumoh National Institute of Technology, Gumi, South Korea

    Sanjay Bhardwaj & Dong-Seong Kim

Authors
  1. Sanjay Bhardwaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dong-Seong Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

CONCEPTION: SB and DS, INTERPRETATION: SB and DS, LITERATURE REVIEW: SB and DS, FINAL APPROVAL: SB and DS.

Corresponding author

Correspondence to Dong-Seong Kim.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhardwaj, S., Kim, DS. Deep Q-learning based sparse code multiple access for ultra reliable low latency communication in industrial wireless networks. Telecommun Syst 83, 409–421 (2023). https://doi.org/10.1007/s11235-023-01034-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-023-01034-0

Keywords

Search

Navigation