Skip to main content
SpringerLink
Log in

Learning-based coordination model for spontaneous self-composition of reliable services in a distributed system

  • Foundation for Mastering Change
  • Special Section REoCAS
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Context-aware, pervasive systems, mobile devices, intelligent virtual assistants activating services or controlling connected devices are pervading our everyday life. These systems rely on centralized services provided by servers in a cloud gathering all requests, performing pre-defined computations and involving pre-defined devices. Large-scale scenarios, involving unanticipated devices, adaptation to dynamically changing conditions, call for alternative solutions favoring edge computing and decentralized behavior. For several years, we have worked on a new type of applications, built and spontaneously composed on-demand. Applications arise from the interactions of multiple sensors and devices, working together as a decentralized collective adaptive system. Our solution relies on a learning-based coordination model providing decentralized communication platforms among agents working on behalf of heterogeneous devices. Each device provides few simple services and data regarding itself (properties and capabilities). In this article, we discuss first the design of complex services, arising from the spontaneous self-composition of simpler services. Second, we present our learning-based coordination model combining coordination and reinforcement learning, and how this approach ensures reliable self-composition of services in terms of functionality and expected quality of services. On the basis of a humanitarian scenario, we show the feasibility of the approach and discuss our current implementation. Preliminary results show convergence toward learning and correct functionality. Spontaneous self-composition and learning provide a self-adaptive solution for creating on-demand complex services evolving in highly dynamic scenarios comprising large numbers of connected devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Avizienis, Algirdas, Laprie, J.-C., Randell, B., Landwehr, C.: Basic concepts and taxonomy of dependable and secure computing. IEEE Comput. Soc. 1, 11 (2004)

    Google Scholar 

  2. Baresi, L., Ghezzi, C., Guinea, S.: Smart monitors for composed services. In: Proceedings of the 2nd International Conference on Service Oriented Computing, ICSOC ’04, pp. 193–202 (2004). ACM, New York

  3. Beal, J., Bachrach, J.: Infrastructure for engineered emergence on sensor/actuator networks. IEEE Intell. Syst. 21, 10 (2006)

    Article  Google Scholar 

  4. Beckert, B., Damiani, F., de Boer, F.S., Bonsangue, M.M. (eds.) Formal Methods for Components and Objects, 10th International Symposium, FMCO 2011, Turin, Italy, October 3–5, 2011, Revised Selected Papers. Lecture Notes in Computer Science, vol. 7542. Springer (2013)

  5. Boes, J., Nigon, J., Verstaevel, N., Gleizes, M.-P., Migeon, F.: The self-adaptive context learning pattern: overview and proposal. In: International and Interdisciplinary Conference on Modeling and Using Context (CONTEXT 2015). Larnaca, Cyprus, 02/11/2015-06/11/2015, LNAI 9405, pp. 91–104. Springer, New York (2015)

  6. Buşoniu, L., Babuška, R., De Schutter, B.: Multi-agent reinforcement learning: an overview. In: Innovations in Multi-agent Systems and Applications-1, pp. 183–221. Springer, New York (2010)

  7. Canfora, G., Di Penta, M., Esposito, R., Villani, M.L.: An approach for qos-aware service composition based on genetic algorithms. In: Proceedings of the 7th Annual Conference on Genetic and Evolutionary Computation, GECCO ’05, pp. 1069–1075. ACM, New York (2005)

  8. Ciatto, G., Mariani, S., Louvel, M., Omicini, A., Zambonelli, F.: Twenty years of coordination technologies: State-of-the-art and perspectives. In: Di Marzo Serugendo, G., Loreti, M. (eds.) 20th IFIP WG 6.1 International Conference, COORDINATION 2018, DisCoTec 2018, Madrid, June 18–21, 2018, pp. 51–80 (2018). Springer International Publishing, Cham

  9. De Angelis, F.L., Fernandez-Marquez, J.L., Di Marzo Serugendo, G.: Self-composition of services in pervasive systems: a chemical-inspired approach, multi-agent systems: technologies and applications. In: Advances in Intelligent Systems and Computing, vol. 296. Springer, New York (2014)

  10. De Angelis, F.L., Fernandez-Marquez, J.L., Di Marzo Serugendo, G.: Self-composition of services with chemical reactions. In: 29th Annual ACM Symposium on Applied Computing (SAC), Gyeongju, Republic of Korea (2014)

  11. Di Marzo Serugendo, G., Abdennadher, N., Mahfoudh, H.B., De Angelis, F.L., Tomaylla, R.: Spatial edge services. Global IoT Summit (2017)

  12. De Nicola, R., Latella, D., Lluch-Lafuente, A., Loreti, M., Margheri, A., Massink, M., Morichetta, A., Pugliese, R., Tiezzi, F., Vandin, A.: The SCEL language: design, implementation, verification. In: Wirsing et al. [39], pp. 3–71

  13. De Schutter, B., Busoniu, L., Babuska, R.: A comprehensive survey of multiagent reinforcement learning. IEEE 11(3), 156 (2008)

    Google Scholar 

  14. Fernandez-Marquez, J.L., Di Marzo Serugendo, G., Montagna, S., Viroli, M., Arcos, J.L.: Description and composition of bio-inspired design patterns: a complete overview. Nat. Comput. 1–25 (2012)

  15. Gelernter, D.: Generative communication in linda. ACM Transactions on Programming Languages and Systems (TOPLAS) (1985)

  16. Gu, C., Zheng, Y.: A context-aware method for service composition in pervasive computing environments. In: Advances in Computer Science, Environment, Ecoinformatics, and Education - International Conference, CSEE 2011 (2011)

  17. HatziDimitris, O., Bassiliades, V., Anagnostopoulos, D., Vlahavas, I.: Semantic awareness in automated web service composition through planning. In: Artificial Intelligence: Theories, Models and Applications (2010)

  18. Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)

    Article  Google Scholar 

  19. Khadka, S., Tumer, K.: Evolution-guided policy gradient in reinforcement learning. In: 32nd Conference on Neural Information Processing Systems (2018)

  20. Mahfoudh, H.B., Di Marzo Serugendo, G., Boulmier, A., Abdennadher, N.: Coordination model with reinforcement learning for ensuring reliable on-demand services in collective adaptive systems. ISOLA (2018)

  21. Mamei, M., Zambonelli, F.: Programming pervasive and mobile computing applications: the tota approach. ACM Trans. Softw. Eng. Methodol. 18(4), 15:1–15:56 (2009)

    Article  Google Scholar 

  22. Mazac, S., Armetta, F., Hassas, S.: Bootstrapping sensori-motor patterns for a constructivist learning system in continuous environments. In: 14th International Conference on the Synthesis and Simulation of Living Systems (Alife’14), New York, NY, USA (2014)

  23. Matthes, F., Langen, M., Braun, D., Mendez, A.H.: Evaluating natural language understanding services for conversational question answering systems. In: Proceedings of the SIGDIAL (2017)

  24. Montagna, S., Viroli, M., Pianini, D., Fernandez-Marquez, J.L.: Towards a comprehensive approach to spontaneous self-composition in pervasive ecosystems. In: De Paoli, F., Vizzari, G. (eds.) Proceedings of the 13th Workshop on Objects and Agents. CEUR-WS (2012)

  25. Moustafa, A., Ito, T.: A deep reinforcement learning approach for large-scale service composition. In: International Conference on Principles and Practice of Multi-Agent Systems (2018)

  26. Omicini, A., Zambonelli, F.: Tucson: a coordination model for mobile information agents. Internet Research: Electronic Networking Applications and Policy, pp. 59 – 79 (1999)

  27. Peltz, C.: Web services orchestration and choreography. IEEE Comput. 36, 46–52 (2003)

    Article  Google Scholar 

  28. Rabanal, P., Mateo, J.A., Rodríguez, I., Díaz, G.: Data-aware automatic derivation of choreography-conforming systems of services. Comput. Stand. Interfaces 53, 59–79 (2017)

    Article  Google Scholar 

  29. Rao, J., Xiaomeng, S.: A survey of automated web service composition methods. Web Services and Web Process Composition, SWSWPC (2004)

    Google Scholar 

  30. Ren, L., Wang, W., Hang, X.: A reinforcement learning method for constraint-satisfied services composition. In: IEEE Transactions on Services Computing (2017)

  31. Shi, W., Cao, J., Youhuizi, L., Quan, Z., Xu, L.: Edge computing: vision and challenges. IEEE (2016)

  32. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  33. ter Beek, M., Bucchiarone, A., Gnesi, S.: Web service composition approaches: from industrial standards to formal methods. In: Proceedings of the Second International Conference on Internet and Web Applications and Services, ICIW ’07, p. 15 (2007). IEEE Computer Society, Washington

  34. Thi Nguyen, S.N.T., Nguyen, N.D.: Deep reinforcement learning for multi-agent systems: a review of challenges, solutions and applications. eprint arXiv:1812.11794 (2018)

  35. Wang, H., Chen, X., Wu, Q., Yu, Q., Zheng, Z., Bouguettaya, A.: Integrating on-policy reinforcement learning with multi-agent techniques for adaptive service composition. LNCS (2014)

  36. Wang, H., Mingzhu, G., Qi, Y., Tao, Y., Li, J., Fei, H., Yan, J., Zhao, W., Hong, T.: Adaptive and large-scale service composition based on deep reinforcement learning. Knowl. Syst. 180, 75–90 (2019)

    Article  Google Scholar 

  37. Wang, H., Wang, X., Xingguo, H., Zhang, X., Mingzhu, G.: A multi-agent reinforcement learning approach to dynamic service composition. Inf. Sci. 363, 96–119 (2016)

    Article  Google Scholar 

  38. Watkins, C.J.C.H.: Learning from Delayed Rewards. PhD thesis, King’s College, Cambridge (1989)

  39. Wirsing, M., Hölzl, M.M., Koch, N., Mayer, P. (eds.): Software Engineering for Collective Autonomic Systems - The ASCENS Approach. Lecture Notes in Computer Science, vol. 8998. Springer, New York (2015)

  40. Wirsing, M., Hölzl, M.M., Tribastone, M., Zambonelli, F.: ASCENS: engineering autonomic service-component ensembles. In: Beckert et al. [4], pp. 1–24

  41. Wu, Z., Ranabahu, A., Gomadam, K., Sheth, A.P., Miller, J.A.: Automatic composition of semantic web services using process and data mediation. In: Proceedings of the 9th International Conference on Enterprise Information Systems, pp. 453–461. Academic Press, London (2007)

  42. Zambonelli, F., Omicini, A., Anzengruber, B., Castelli, G., De Angelis, F.L., Di Marzo Serugendo, G., Dobson, S., Fernandez-Marquez, J.L., Ferscha, A., Mamei, M., Mariani, S., Molesini, A., Montagna, S., Nieminen, J., Pianini, D., Risoldi, M., Rosi, A., Stevenson, G., Viroli, M., Ye, J.: Developing pervasive multi-agent systems with nature-inspired coordination. Pervasive and Mobile Computing, 17, Part B: 236–252 (2015). 10 years of Pervasive Computing’ In Honor of Chatschik Bisdikian

Download references

Author information

Authors and Affiliations

  1. University of Geneva, Route Drize 7, 1227, Geneva, Switzerland

    Houssem Ben Mahfoudh, Giovanna Di Marzo Serugendo & Nabil Naja

  2. University of Applied Science of Western Switzerland, Rue de la Prairie 4, 1202, Geneva, Switzerland

    Nabil Abdennadher

Authors
  1. Houssem Ben Mahfoudh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giovanna Di Marzo Serugendo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nabil Naja
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nabil Abdennadher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Houssem Ben Mahfoudh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ben Mahfoudh, H., Di Marzo Serugendo, G., Naja, N. et al. Learning-based coordination model for spontaneous self-composition of reliable services in a distributed system. Int J Softw Tools Technol Transfer 22, 417–436 (2020). https://doi.org/10.1007/s10009-020-00557-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-020-00557-0

Keywords

Search

Navigation