Skip to main content
SpringerLink
Log in

Adaptive Monitoring Optimization Based on Deep-Q-Network for Energy Harvesting Wireless Sensor Networks

  • Conference paper
  • First Online:
Mobile Networks and Management (MONAMI 2022)

Included in the following conference series:

  • 196 Accesses

Abstract

In order to improve the energy efficiency of environmental monitoring for energy harvesting wireless sensor networks (EH-WSNs) in remote areas and achieve energy-neutral operation, an adaptive monitoring and energy management optimization method of EH-WSNs based on deep Q network (DQN) algorithm is proposed. In this paper, aiming at EH-WSNs with single-hop cluster structure, we first present a more realistic energy model established by combining different climate characteristics. Then, the optimization model of maximizing long-term monitoring utility is formulated based on harvested energy constraints. We use deep Q network (DQN) to learn random and dynamic solar energy harvesting process on solar-powered sensor nodes and optimize the monitored performance of EH-WSNs through the replay memory mechanism and freezing parameter mechanism. Finally, we present an adaptive monitoring optimization method based DQN to achieve the long-term utility. Through simulation verification and comparative analysis, in different rainy weather environments, the proposed optimization algorithm has greatly improved in terms of average monitoring reward, monitoring interruption rate and energy overflow rate. Moreover, it also indicates that the proposed algorithm has effective adaptation to the random and dynamic solar energy arrival.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Lombardo, L., Corbellini, S., Parvis, M., Elsayed, A., Angelini, E., Grassini, S.: Wireless sensor network for distributed environmental monitoring. IEEE Trans. Instrum. Meas. 67(5), 1214–1222 (2017)

    Article  Google Scholar 

  2. Muduli, L., Mishra, D.P., Jana, P.K.: Application of wireless sensor network for environmental monitoring in underground coal mines: a systematic review. J. Netw. Comput. Appl. 106, 48–67 (2018)

    Article  Google Scholar 

  3. Cao, Y., Ji, R., Ji, L., Lei, G., Wang, H., Shao, X.: l2-MPTCP: a learning-driven latency-aware multipath transport scheme for industrial internet applications. IEEE Transactions on Industrial Informatics (2022)

    Google Scholar 

  4. Cao, Y., Ji, R., Huang, X., Lei, G., Shao, X., You, I.: Empirical Mode Decomposition-empowered Network Traffic Anomaly Detection for Secure Multipath TCP Communications, Mobile Networks and Applications (2022)

    Google Scholar 

  5. Antony, S.M., Indu, S., Pandey, R.: An efficient solar energy harvesting system for wireless sensor network nodes. J. Inf. Optim. Sci. 41(1), 39–50 (2020)

    Google Scholar 

  6. Sun, W., Ding, Z., Qin, Z., Chu, F., Han, Q.: Wind energy harvesting based on fluttering double-flag type triboelectric nanogenerators. Nano Energy 70, 104526 (2020)

    Article  Google Scholar 

  7. Sharma, H., Haque, A., Jaffery, Z.A.: Modeling and optimisation of a solar energy harvesting system for wireless sensor network nodes. J. Sens. Actuator Netw. 7(3), 40 (2018)

    Google Scholar 

  8. Sharma, H., Haque, A., Jaffery, Z.A.: Maximization of wireless sensor network lifetime using solar energy harvesting for smart agriculture monitoring. Ad Hoc Netw. 94, 101966 (2019)

    Article  Google Scholar 

  9. Sarang, S., Drieberg, M., Awang, A., Ahmad, R.: A QoS MAC protocol for prioritized data in energy harvesting wireless sensor networks. Comput. Netw. 144, 141–153 (2018)

    Article  Google Scholar 

  10. Lakshmi, P.S., Jibukumar, M.G., Neenu, V.S.: Network lifetime enhancement of multi-hop wireless sensor network by RF energy harvesting. In: Proceedings of the 2018 International Conference on Information Networking, pp. 738–743 (2018)

    Google Scholar 

  11. Nguyen, H.S., Ly, T.T.H., Nguyen, T.S., Huynh, V.V., Nguyen, T.L., Voznak, M.: Outage performance analysis and SWIPT optimization in energy-harvesting wireless sensor network deploying NOMA. Sensors 19(3), 613 (2019)

    Google Scholar 

  12. Ren, Q., Yao, G.: An energy-efficient cluster head selection scheme for energy-harvesting wireless sensor networks. Sensors 20(1), 187 (2020)

    Google Scholar 

  13. Xiong, Y., Chen, G., Lu, M., Wan, X., Wu, M., She, J.: A two-phase lifetime-enhancing method for hybrid energy-harvesting wireless sensor network. IEEE Sens. J. 20(4), 1934–1946 (2019)

    Article  Google Scholar 

  14. Bengheni, A., Didi, F., Bambrik, I.: EEM-EHWSN: enhanced energy management scheme in energy harvesting wireless sensor networks. Wireless Netw. 25(6), 3029–3046 (2019)

    Article  Google Scholar 

  15. Qiu, C., Hu, Y., Chen, Y., Zeng, B.: Lyapunov optimization for energy harvesting wireless sensor communications. IEEE Internet Things J. 5(3), 1947–1956 (2018)

    Article  Google Scholar 

  16. Lee, P., Eu, Z.A., Han, M., Tan, H.: Empirical modeling of a solar-powered energy harvesting wireless sensor node for time-slotted operation. In: Proceedings of the 2011 IEEE Wireless Communications and Networking Conference, pp. 179–184 (2011)

    Google Scholar 

  17. Fraternali, F., Balaji, B., Agarwal, Y., Gupta, R.K.: Aces: automatic configuration of energy harvesting sensors with reinforcement learning. ACM Trans. Sens. Netw. 16(4), 1–31 (2020)

    Article  Google Scholar 

  18. Tekin, N., Gungor, V.C.: The impact of error control schemes on lifetime of energy harvesting wireless sensor networks in industrial environments. Comput. Stand. Interfaces 70, 103417 (2020)

    Article  Google Scholar 

  19. Han, C., Zhang, S., Zhang, B., Zhou, J., Sun, L.: A distributed image compression scheme for energy harvesting wireless multimedia sensor networks. Sensors 20(3), 667 (2020)

    Google Scholar 

  20. Raja, J., Mookhambika, N.: A novel energy harvesting with middle-order weighted probability (EHMoWP) for performance improvement in wireless sensor network (WSN). J. Ambient Intell. Humaniz. Comput. 1–12 (2021)

    Google Scholar 

  21. Zairi, S., Zouari, B., Niel, E., Dumitrescu, E.: Nodes self-scheduling approach for maximising wireless sensor network lifetime based on remaining energy. IET Wirel. Sens. Syst. 2(1), 52–62 (2012)

    Article  Google Scholar 

  22. Sahoo, J., Sahoo, B.: Solving target coverage problem in wireless sensor networks using greedy approach. In: Proceedings of the 2020 International Conference on Computer Science, Engineering and Applications, pp. 1–4 (2020)

    Google Scholar 

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (Grant No. 61961026, 61962036), Natural Science Foundation of Jiangxi Province, China (Grant No. 20202BABL202003), China Postdoctoral Science Foundation (Grant No. 2020M671556), Major science and technology projects in Jiangxi province (20213AAG01012).

Author information

Authors and Affiliations

  1. Jiangxi Province Key Laboratory of Water Information Cooperative Sensing and Intelligent Processing, Nanchang Institute of Technology, Nanchang, Jiangxi, 330099, China

    Xuecai Bao, Peilun Bian, Wenqun Tan & Jugen Nie

  2. Jiangxi Provincial Technology Innovation Center for Ecological Water Engineering in Poyang Lake Basin, Jiangxi, 330029, China

    Xuecai Bao, Peilun Bian, Wenqun Tan, Xiaohua Xu & Jugen Nie

Authors
  1. Xuecai Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peilun Bian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenqun Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaohua Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jugen Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuecai Bao .

Editor information

Editors and Affiliations

  1. Jiangxi Normal University, Nanchang, China

    Yuanlong Cao

  2. Toyohashi University of Technology, Toyohashi, Japan

    Xun Shao

Rights and permissions

Reprints and permissions

Copyright information

© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bao, X., Bian, P., Tan, W., Xu, X., Nie, J. (2023). Adaptive Monitoring Optimization Based on Deep-Q-Network for Energy Harvesting Wireless Sensor Networks. In: Cao, Y., Shao, X. (eds) Mobile Networks and Management. MONAMI 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 474. Springer, Cham. https://doi.org/10.1007/978-3-031-32443-7_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-32443-7_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-32442-0

  • Online ISBN: 978-3-031-32443-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation