Abstract
The coordinated learning is importance of technique for cooperative multi-objects system in large-scale Internet of Things . The coordinated learning has attracted a lot of attention for its applications in Internet of Things. However, the self-adaptive makes the coordinated learning difficult to be used in IoT. This paper proposes multi-objects scalable coordinated learning algorithm based on the maximum potential loss of coordination. The algorithm defines an interaction measure that allows objects to dynamically estimate the potential utility loss of coordination with any cluster of objects. The interaction mechanism makes each object compute their beneficial coordination set in different situations and makes the best use of their limited communication resource in Internet of Things. As a result of experiments, our algorithm adapts policy learning of object and their coordination network for different context. Finally, the experiments with the smart agriculture data set demonstrate that the proposed scheme is effective and robust.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Atzori L, Lera A, Morabito G (2010) The internet of things: a survey. Comput Netw 54(15):2787–2805
Miorandi D, Sicari S, De Pellegrini F, Chlamtac I (2012) The internet of things:vision, applications and research challenges. Ad Hoc Netw 10(7):1497–1516
Cheng SJ, Raja A, Lesser V (2013) Multiagent meta-level control for radar coordination. Int J Web Intell Agent Syst 11(1):81–105
Carlin A, Zilberstein S (2011) Decentralized monitoring of distributed anytime algorithms. In: Proceedings of the tenth international conference on autonomous agents and multiagent systems (AAMAS 2011), Taiwan, pp 157–164
Zhang C, Lesser VR (2011) Coordinated multi-agent reinforcement learning in networked distributed pomdps. In: Proceedings of the twenty-fifth national conference on argificial intelligence (AAAI), California, USA
Banerjee B, Peng J (2007) Generalized multiagent learning with performance bound. Auton Agent Multi-Agent Syst 15(3):281–312
Cheng S, Raja A, Lesser V (2010) Multiagent meta-level control for a network of weather radars. In: Proceedings of IEEE/WIC/ACM international conference on intelligent agent technology (IAT-2010),Toronto, Canada, pp 157–164
Kim Y, Krainin M, Lesse VR (2011) Effective variants of the maxsum algorithm for radar coordination and scheduling. In the maxsum algorithm for radar coordination and scheduling. In: Proceedings of 2011 IEEE/WIC/ACM international conference on intelligent agent technology (IAT-2011), Lyon, France, pp 357–364
Cheng S (2012) Coordinating decentralized learning and conflict resolution across agent boundaries. In PhD thesis, University of North Carolina at Charlotte
Vinyals M, Rodriguez-Aguilar JA, Cerquides J (2011) A survey on sensor networks from a multiagent perspective. Comput J 54(3):455–470
Raja A, Lesser V (2007) A Framework for meta-level control in multi-agent systems. Auton Agents and Multi-Agent Syst 15(2):147196
Kennedy C (2010) Decentralised metacognition in context-aware autonomic systems: some key challenges. In: Proceedings of workshops at the twenty-fourth AAAI conference on artificial intelligence
Todd H, Peter S (2009) Generalized model learning for reinforcement learning in factored domains. In: Proceedings of international conference on autonomous agents and multiagent systems, pp 490–496
Zhang C, Lesser V, Abdallah S (2010) Self-organization for coordinating decentralized reinforcement learning. In AAMAS10
Zhang CJ, Lesser V (2013) Coordinating multi-agent reinforcement learning with limited communication. In: Proceedings of the 12th international conference on autonomous agents and multiagent systems, pp 1101–1108
Conitzer V, Sandholm T (2007) Awesome: a general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents. Mach Learn 67(2):23–43
Azizi E (2013) Joint learning of modular structures from multiple data types. In NIPS workshop of frontiers of network analysis: methods, models, and applications
Soufiani HA, Airoldi EM (2012) Graphlet decomposition of a weighted network. J Mach Learn Res JMLR WCP 22:5463
Chiang M, Low SH, Doyle JC (2007) Layering as optimization decomposition: a mathematical theory of network architectures. Proc IEEE 95(1):255–312
Wang J, Li L, Low SH, Doyle JC (2005) Cross-layer optimization in TCP/IP networks. Trans Netw IEEE/ACM 13(3):582–595
Yen N, Tung T, Tomi S, Yun T (2012) Transferring expectations in model-based reinforcement learning. In: Proceedings of the advances in neural information processing systems, NIPS12
OSGi HomePage http://www.osgi.org/
Aiello F, Fortino G, Gravina R, Guerrieri A (2011) A Java-based agent platform for programming wireless sensor networks. Comput J 54(3):439–454
Busoniu L, Babuska R, Schutte BD (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern Part C 38(2):156–172
Stranders R, Farinelli A, Rogers A, Jennings NR (2009) Decentralised coordination of mobile sensors using the max-sum algorithm. In IJCAI, pp 299–304
Varakantham P, Tambe M, Yokoo M (2005) Networked distributed pomdps: A synthesis of distributed constraint optimization and pomdps. In AAAI, pp 133–139
Acknowledgments
The author also would like to thank anonymous editor and reviewers who gave valuable suggestion that has helped to improve the quality of the manuscript. This research has been supported by the Project for 2015 National Key Technologies RD Program No. 2015BAH04F 01.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, J., Duan, S. & Shi, Y. Multi-objects scalable coordinated learning in internet of things. Pers Ubiquit Comput 19, 1133–1144 (2015). https://doi.org/10.1007/s00779-015-0888-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00779-015-0888-2