Abstract
Pure peer-to-peer networks serve to secure information in a decentralized, distributed topology. The multi-armed bandit (MAB) problem formulation proves to be a useful tool for analyzing the problem of optimizing new peer connections. In this paper, we outline the new peer scenario described as a reinforcement learning problem with MABs in order to identify the fastest peer to download from during the connection process. The MAB problem involves k slot machines which are also called one-armed bandits and pay out reward values according to an internal distribution, of which the agent is not aware. The aim is to choose a strategy to learn which arms pay out the most in order to maximize total reward over a set number of rounds. Results indicate that UCB and \(\varepsilon \)-first performed the best at selecting the optimal peer in each of our test scenarios. Contrariwise, SoftMax and \(\varepsilon \)-greedy unperformed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Schollmeier, R.: A definition of peer-to-peer networking for the classification of peer-to-peer architectures and applications. In: Proceedings First International Conference on Peer-to-Peer Computing, pp. 101–102. IEEE (2001)
Karras, C., Karras, A., Sioutas, S.: Pattern recognition and event detection on IoT data-streams. arXiv preprint arXiv:2203.01114 (2022)
Karras, C., Karras, A.: DBSOP: an efficient heuristic for speedy MCMC sampling on polytopes. arXiv preprint arXiv:2203.10916 (2022)
Avner, O., Mannor, S.: Multi-user communication networks: a coordinated multi-armed bandit approach. IEEE/ACM Trans. Netw. 27(6), 2192–2207 (2019)
Karras, C., Karras, A., Avlonitis, M., Giannoukou, I., Sioutas, S.: Maximum likelihood estimators on MCMC sampling algorithms for decision making. In: Maglogiannis, I., Iliadis, L., Macintyre, J., Cortez, P. (eds.) AIAI 2022. IFIP Advances in Information and Communication Technology, vol. 652, pp. 345–356. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-08341-9_28
Karras, C., Karras, A., Avlonitis, M., Sioutas, S.: An overview of MCMC methods: from theory to applications. In: Maglogiannis, I., Iliadis, L., Macintyre, J., Cortez, P. (eds.) AIAI 2022. IFIP Advances in Information and Communication Technology, vol. 652, pp. 319–332. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-08341-9_26
Szorenyi, B., Busa-Fekete, R., Hegedus, I., Ormándi, R., Jelasity, M., Kégl, B.: Gossip-based distributed stochastic bandit algorithms. In: International Conference on Machine Learning, pp. 19–27. PMLR (2013)
Yang, M., Zhu, H., Wang, H., Koucheryavy, Y., Samouylov, K., Qian, H.: Peer to peer offloading with delayed feedback: an adversary bandit approach. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5035–5039. IEEE (2020)
Modi, N., Mary, P., Moy, C.: QoS driven channel selection algorithm for cognitive radio network: multi-user multi-armed bandit approach. IEEE Trans. Cogn. Commun. Netw. 3(1), 49–66 (2017)
Boldrini, S., De Nardis, L., Caso, G., Le, M.T., Fiorina, J., Di Benedetto, M.G.: muMAB: a multi-armed bandit model for wireless network selection. Algorithms 11(2), 13 (2018)
Vial, D., Shakkottai, S., Srikant, R.: Robust multi-agent multi-armed bandits. In: Proceedings of the Twenty-second International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, pp. 161–170 (2021)
Li, F., Yu, D., Yang, H., Yu, J., Karl, H., Cheng, X.: Multi-armed-bandit-based spectrum scheduling algorithms in wireless networks: a survey. IEEE Wirel. Commun. 27(1), 24–30 (2020)
Hillel, E., Karnin, Z.S., Koren, T., Lempel, R., Somekh, O.: Distributed exploration in multi-armed bandits. In: Advances in Neural Information Processing Systems, vol. 26 (2013)
Korda, N., Szorenyi, B., Li, S.: Distributed clustering of linear bandits in peer to peer networks. In: International Conference on Machine Learning, pp. 1301–1309. PMLR (2016)
Si, P., Yu, F.R., Ji, H., Leung, V.C.: Distributed sender scheduling for multimedia transmission in wireless mobile peer-to-peer networks. IEEE Trans. Wirel. Commun. 8(9), 4594–4603 (2009)
Tasoulas, Z.G., Anagnostopoulos, I.: Improving GPU performance with a power-aware streaming multiprocessor allocation methodology. Electronics 8(12), 1451 (2019)
Silva, N., Werneck, H., Silva, T., Pereira, A.C., Rocha, L.: Multi-armed bandits in recommendation systems: a survey of the state-of-the-art and future directions. Expert Syst. Appl. 197, 116669 (2022)
Vermorel, J., Mohri, M.: Multi-armed Bandit Algorithms and Empirical Evaluation. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 437–448. Springer, Heidelberg (2005). https://doi.org/10.1007/11564096_42
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Karras, A., Karras, C., Giotopoulos, K.C., Giannoukou, I., Tsolis, D., Sioutas, S. (2022). Download Speed Optimization in P2P Networks Using Decision Making and Adaptive Learning. In: Daimi, K., Al Sadoon, A. (eds) Proceedings of the ICR’22 International Conference on Innovations in Computing Research. ICR 2022. Advances in Intelligent Systems and Computing, vol 1431. Springer, Cham. https://doi.org/10.1007/978-3-031-14054-9_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-14054-9_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-14053-2
Online ISBN: 978-3-031-14054-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)