Skip to main content

Advertisement

SpringerLink
Log in

Joint resource allocation for emotional 5G IoT systems using deep reinforcement learning

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In emotional computing related IoT system, emotional sensors, as the IoT devices, are usually deployed to collect the emotional data from humans. The IoT devices need wireless connections to send the collected data to the server, that conducts the prediction to give user instructions. Mobile edge computing (MEC) is a promising technology to fit into this scenario. However, the IoT devices are usually short of energy supply and the local computation gives less accurate emotional computing results. To solve the problem, this paper intends to maximize the total energy efficiency of communication and computation within the MEC servers and sensors by jointly optimizing the allocation of channels and computing resources. The formulated problem is non-convex and usually solved through the successive convex approximation (SCA) method. Compared to SCA, deep Q network (DQN) method is used in this paper, which involves less computation cost to be more practically deployed. The simulation results show that the DQN solution outperforms the other benchmarking solutions, and the total energy consumption of the system is effectively reduced with a guaranteed emotional computing accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Yamazaki Y, Yamamoto M, Nagata N (2017) Estimation of emotional state in personal fabrication: analysis of emotional motion based on Laban movement analysis. In: 2017 International Conference on Culture and Computing (Culture and Computing), Kyoto, 2017, pp 71–74, https://doi.org/10.1109/Culture.and.Computing.2017.33

  2. Yuehong F (2020) Computational model analysis of affective cognitive reasoning in artificial intelligence. J Shanghai Normal Univ (Philos Soc Sci Edn) 49(02):94–103

    Google Scholar 

  3. Lou G, Shi H (2020) Face image recognition based on convolutional neural network. China Commun 17(2):117–124. https://doi.org/10.23919/JCC.2020.02.010

    Article  Google Scholar 

  4. Tao F, Liu G, Zhao Q (2018) An ensemble framework of voice-based emotion recognition system. In: 2018 First Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia), pp 1-6

  5. You X, Zhang C, Tan X et al. (2019) AI for 5G: research directions and paradigms. Sci China Inf Sci 62:21301. https://doi-org-443-buaa.naihes.cn/10.1007/s11432-018-9596-5

  6. Yang Z et al (2019) Energy-efficient joint resource allocation algorithms for MEC-enabled emotional computing in urban communities. IEEE Access 7:137410–137419. https://doi.org/10.1109/ACCESS.2019.2942391

    Article  Google Scholar 

  7. Wang K, Yang K, Chen H, Zhang L (2017) Computation diversity in emerging networking paradigms. IEEE Wirel Commun 24(1):88–94. https://doi.org/10.1109/MWC.2017.1600161WC

    Article  Google Scholar 

  8. Mao Y, You C, Zhang J, Huang K, Letaief KB (2017) Mobile edge computing: survey and research outlook. IEEE Commun Surv Tutor

  9. Liu H, Eldarrat F, Alqahtani H, Reznik A, de Foy X, Zhang Y (2017) Mobile edge cloud system: architectures challenges and approaches. IEEE Syst J 12:2495–2508

    Article  Google Scholar 

  10. Wei X et al (2017) MVR: an architecture for computation offloading in mobile edge computing. In: 2017 IEEE International Conference on Edge Computing (EDGE), Honolulu, HI, 2017, pp 232–235. https://doi.org/10.1109/IEEE.EDGE.2017.42

  11. Patel M, Naughton B, Chan C, Sprecher N, Abeta S, Neal A et al (2014) Mobile-edge computing introductory technical white paper, White Paper Mobile-edge Computing (MEC) industry initiative

  12. Hu YC, Patel M, Sabella D, Sprecher N, Young V (2015) Mobile edge computing a key technology towards 5g, ETSI White Paper, vol 11

  13. Ahmed A, Ahmed E (2016) A survey on mobile edge computing. IEEE International Conference on Intelligent Systems and Control

  14. He C, Li W, Chik D (2017) Waveform compensation of ECG data using segment fitting functions for individual identification. In: 2017 13th International Conference on Computational Intelligence and Security (CIS), Hong Kong, pp 475–479. https://doi.org/10.1109/CIS.2017.00110

  15. Zucco C, Calabrese B, Cannataro M (2017) Sentiment analysis and affective computing for depression monitoring. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Kansas City, MO, 2017, pp 1988–1995, https://doi.org/10.1109/BIBM.2017.8217966

  16. Picard RW (1997) Affective computing. MIT Press, London

    Google Scholar 

  17. Rao T, Xu M, Liu H, Wang J, Burnett I (2016) Multi-scale blocks based image emotion classification using multiple instance learning. In: 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, 2016, pp 634–638, https://doi.org/10.1109/ICIP.2016.7532434

  18. Mansourifar H, Bastanfard A (2011) Fast conic spline data fitting of noise-free data points. In: 2011 Eighth International Conference Computer Graphics, Imaging and Visualization, Singapore, pp 59–64. https://doi.org/10.1109/CGIV.2011.33

  19. Haghighi V, Moayedian NS (2018) An offloading strategy in mobile cloud computing considering energy and delay constraints. IEEE Access 6:11849–11861

    Article  Google Scholar 

  20. Mobile edge computing (mec) (2014) [online] Available: http://www.etsi.org/technologies-clusters/technologies/mobile-edge-computing

  21. Chiang M, Zhang T (2016) Fog and IoT: an overview of research opportunities. IEEE Internet Things J. 3(6):854–864

    Article  Google Scholar 

  22. Wang K, Yang K, Magurawalage CS (2018) Joint energy minimization and resource allocation in C-RAN with mobile cloud. IEEE Trans Cloud Comput 6(3):760–770. https://doi.org/10.1109/TCC.2016.2522439

    Article  Google Scholar 

  23. Zhang Y, Zhou X, Teng Y, Fang J, Zheng W (2018) Resource allocation for multi-user MEC system: machine learning approaches. In: 2018 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA. pp 794–799. https://doi.org/10.1109/CSCI46756.2018.00159

  24. Liang K, Zhao L, Zhao X, Wang Y, Ou S (2016) Joint resource allocation and coordinated computation offloading for fog radio access networks. China Commun 13(Supplement 2):131–139. https://doi.org/10.1109/CC.2016.7833467

    Article  Google Scholar 

  25. Zeng Q, Wu Q, Chen G (2020) A novel joint resource allocation algorithm in 5G heterogeneous integrated networks. In: Communications, signal processing, and systems

  26. Praveen-Kumar D, Amgoth T, Annavarapu CSR (2019) Machine learning algorithms for wireless sensor networks: a survey. Inform Fusion 49:1–25

    Article  Google Scholar 

  27. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction, vol 1. MIT Press, Cambridge

    MATH  Google Scholar 

  28. Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) Deep reinforcement learning: a brief survey. IEEE Signal Process Mag 34(6):26–38. https://doi.org/10.1109/MSP.2017.2743240

    Article  Google Scholar 

  29. Yang T, Hu Y, Gursoy MC, Schmeink A, Mathar R (2018) Learning deep reinforcement, based resource allocation in low latency edge computing networks. In: 15th International Symposium on Wireless Communication Systems (ISWCS), Lisbon, pp 1–5. https://doi.org/10.1109/ISWCS.2018.8491089

  30. Yang K, Ou S, Chen H (2008) On effective offloading services for resource-constrained mobile devices running heavier mobile Internet applications. IEEE Commun Mag 46(1):56–63. https://doi.org/10.1109/MCOM.2008.4427231

    Article  Google Scholar 

  31. Hu X, Wong K, Yang K (2018) Wireless powered cooperation-assisted mobile edge computing. IEEE Trans Wirel Commun 17(4):2375–2388. https://doi.org/10.1109/TWC.2018.2794345

    Article  Google Scholar 

  32. Wu J, Shin S, Kim C, Kim S (2017) Effective lazy training method for deep q-network in obstacle avoidance and path planning. In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AB. pp 1799–1804, https://doi.org/10.1109/SMC.2017.8122877

  33. Zhang Q, Lin M, Yang LT, Chen Z, Khan SU, Li P (2019) A double deep Q-learning model for energy-efficient edge scheduling. IEEE Trans Serv Comput 12(5):739–749. https://doi.org/10.1109/TSC.2018.2867482

    Article  Google Scholar 

  34. Zongtao X, Jin Q (2018) Deep Q network based on advantage learning. Comput Eng Appl. http://kns.cnki.net/kcms/detail/ll.2127.TR20181224.1726.026.html

  35. Zhang T, Li Q, Zhang C-S, Liang H-W, Li P, Wang T-M et al (2017) Current trends in the development of intelligent unmanned autonomous systems. Front Inform Technol Electron Eng 18(1):68–85

    Article  Google Scholar 

  36. Xia W, Li H, Li B (2016) A control strategy of autonomous vehicles based on deep reinforcement learning. In: 9th International Symposium on Computational Intelligence and Design, pp 198–201

  37. Zhang X, Li J (2021) Power control for cognitive users of perception layer in complex industrial CPS based on DQN. IEEE Access 9:25371–25382. https://doi.org/10.1109/ACCESS.2021.3057911

    Article  Google Scholar 

  38. Carta S, Ferreira A, Podda AS et al. Multi-DQN: an ensemble of deep Q-learning agents for stock market forecasting. Expert Syst Appl 164

  39. Yang L, Cao J, Tang S, Li T, Chan ATS (2012) A framework for partitioning and execution of data stream applications in mobile cloud computing. In: 2012 IEEE Fifth International Conference on Cloud Computing, pp 794–802

  40. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Antonoglou DHI, Wierstra D, Riedmiller MA (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Article  Google Scholar 

Download references

Acknowledgements

The work in this paper was partly supported by Natural Science Foundation of China (Grant No. 61620106011).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Northeast Normal University, Changchun, Jilin, China

    Ziyan Yang, Wenyong Wang & Dongdai Zhou

  2. School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China

    Haibo Mei

  3. School of Computer Science and Electronic Engineering, University of Essex, CO43SQ, Colchester, UK

    Kun Yang

Authors
  1. Ziyan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haibo Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenyong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dongdai Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haibo Mei.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Z., Mei, H., Wang, W. et al. Joint resource allocation for emotional 5G IoT systems using deep reinforcement learning. Int. J. Mach. Learn. & Cyber. 12, 3517–3528 (2021). https://doi.org/10.1007/s13042-021-01398-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01398-2

Keywords

Search

Navigation