Abstract
In this paper, we consider a point-to-point wireless communication in embedded system. This system is supposed to be battery powered and equipped with an energy harvester. According to the battery level and the harvested energy, the transmitter has to make decision following an optimal policy in order to maximize its reward over the operating period. We first consider a prior stochastic knowledge of the transition matrix probabilities to find out the optimal policy using algorithm originated from DP methods. With no such stochastic knowledge, we will adopt algorithms from RL methods to find out the optimal policies. The resulting performances are then compared.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Atzori, L., Iera, A., Morabito, G.: The internet of things: a survey. Comput. Netw. 54, 2787–2805 (2010)
Goldsmith, A.: Wireless Communications, 1st edn. Cambridge University Press, Cambridge (2005)
Estrin, D., Girod, L., Pottie, G., Srivastava, M.: Instrumenting the world with wireless sensor networks. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2001), Salt Lake City, Utah, USA, vol. 4, pp. 2033–2036, May 2001
Yrjola, J.: Summary of energy efficiency communication protocol for wireless micro sensor networks, 13 March 2005
Nguyen, L.T., Defago, X., Beuran, R., Shinoda, Y.: An energy efficient routing scheme for mobile wireless sensor networks. In: IEEE/ISWCS, pp. 568–572 (2008)
Vullers, R., Schaijk, R., Visser, H.J., Penders, J., Hoof, C.V.: Energy harvesting for autonomous wireless sensor networks. IEEE Solid-State Circuits Mag. 2, 29–38 (2010)
Devillers, B., Gunduz, D.: A general framework for the optimization of energy harvesting communication systems with battery imperfections. J. Commun. Netw. 14(2), 130–139 (2012)
Tutuncuoglu, K., Yener, A.: Sum-rate optimal power policies for energy harvesting transmitters in an interference channel. J. Commun. Netw. 14(2), 151–161 (2012)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, A. B. Book edn. MIT Press, Cambridge (1998)
Prabhu, B., Antony, A.J., Balakumar, N.: A research on smart transportation using sensors and embedded systems. Int. J. Innov. Res. Comput. Sci. Technol. (IJIRCST), 5(1) (2017). ISSN: 2347-5552
McQueen, B., McQueen, J.: Intelligent Transportation Systems Architecture. Artech House (Intelligent Transportation Systems Library), Norwood (1999)
https://www.iis.fraunhofer.de/en/ff/lok/proj/daedalus.html. Accessed 4 Apr 2017
Blasco, P., Gunduz, D., Dohler, M.: A learning theoretic approach to energy harvesting communication system optimization. IEEE Trans. Wirel. Commun. 12(4), 1872–1882 (2013)
Blasco, P., Gunduz, D.: Multi-access communications with energy harvesting: a multi-armed bandit model and the optimality of the myopic policy. IEEE J. Sel. Areas Commun. 33(3), 585–597 (2015)
Papoulis, A.: Probability, Random Variables, and Stochastic Processes. McGraw- Hill, New York (1965)
Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)
Putterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley-Interscience, Hoboken (2005)
Mansour, Y., Singh, S.: On the complexity of policy iteration. In: Proceedings of the 15th International Conference on Uncertainty in AI, Stockholm, SE, pp. 401–408 (1999)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)
Szepesvari, C.: Reinforcement Learning Algorithm for MDPs. Morganand Claypool Publishers, San Rafael (2010)
Singh, S., Jaakkola, T., Littman, M.L., Szepesvari, C.: Convergence results for single-step on-policy reinforcement-learning algorithms. Mach. Learn. 38(3), 287–308 (2000)
Corazza, M., Sangalli, A.: Q-learning and SARSA: a comparison between two intelligent stochastic control approaches for financial trading. University Ca’ Foscari of Venice, Department of Economics Research Paper Series No. 15/WP/2015, 10 June 2015
IEEE 802.15.4e Draft Standard: Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications for Low-Rate Wireless Personal Area Networks (WPANs), IEEE Std., March 2010
Chalasani, S., Conrad, J.: A survey of energy harvesting sources for embedded systems. In: Southeastcon, pp. 442–447. IEEE, April 2008
Galindo-Serrano, A., Giupponi, L., Dohler, M.: Cognition and docition in OFDMA-based femtocell networks. In: IEEE Globecomm, Miami, Florida, USA, pp. 6–10, December 2010
Acknowledgement
There is no word that can express our deep and sincere regret about the death of Pr. Driss Aboutajdine, the LRIT laboratory leader, that occurred on Saturday March 4th 2017. He was an extraordinary man and we are many in the community who will profoundly mourn his death as he was the model of generosity, perseverance and excellence throughout his life. May he rest in peace.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Assaouy, M., Zytoune, O., Aboutajdine, D. (2017). DP and RL Approach Optimization for Embedded System Communications with Energy Harvesting. In: Eichler, G., Erfurth, C., Fahrnberger, G. (eds) Innovations for Community Services. I4CS 2017. Communications in Computer and Information Science, vol 717. Springer, Cham. https://doi.org/10.1007/978-3-319-60447-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-60447-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60446-6
Online ISBN: 978-3-319-60447-3
eBook Packages: Computer ScienceComputer Science (R0)