Abstract
As Internet applications become larger and more complex, the task of managing them becomes overwhelming. “Abnormal” events such as software updates, failures, attacks, and hotspots become frequent. The SELFMAN project is tackling this problem by combining two technologies, namely structured overlay networks and advanced component models, to make the system self managing. Structured overlay networks (SONs) developed out of peer-to-peer systems and provide robustness, scalability, communication guarantees, and efficiency. Component models provide the framework to extend the self-managing properties of SONs over the whole system. SELFMAN is building a self-managing transactional storage and using it for two application demonstrators: a distributed Wiki and an on-demand media streaming service. This paper provides an introduction and motivation for the ideas underlying SELFMAN and a snapshot of its contributions midway through the project. We explain our methodology for building self-managing systems as networks of interacting feedback loops. We then summarize the work we have done to make SONs a practical basis for our architecture: using an advanced component model, handling network partitions, handling failure suspicions, and doing range queries with load balancing. Finally, we show the design of a self-managing transactional storage on a SON.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arad, C., Roverso, R., Haridi, S., Jaradin, Y., Mejias, B., Van Roy, P., Coupaye, T., Dillenseger, B., Diaconescu, A., Harbaoui, A., Jayaprakash, N., Kessis, M., Lefebvre, A., Leger, M.: Report on architectural framework specification. SELFMAN Deliverable D2.2a (June 2007), www.ist-selfman.org
Armstrong, J.: Making Reliable Distributed Systems in the Presence of Software Errors. Ph.D. dissertation, Royal Institute of Technology (KTH), Stockholm, Sweden (November 2003)
Arthur, W.B.: Complexity in economic theory: Inductive reasoning and bounded rationality. The American Economic Review 84(2), 406–411 (1994)
Ross, W.A.: An Introduction to Cybernetics. Chapman & Hall Ltd., London, (1956), pcp.vub.ac.be/books/IntroCyb.pdf
von Bertalanffy, L.: General System Theory: Foundations, Development, Applications, George Braziller (1969)
Capra, F.: It’s a Wonderful Life. Liberty Films (1946)
Carroll, L.: Through the Looking-Glass and What Alice Found There (1872) (Dover Publications reprint 1999)
Carton, B., Mesaros, V.: Improving the Scalability of Logarithmic-Degree DHT-Based Peer-to-Peer Networks. In: 10th International Euro-Par Conference, pp. 1060–1067 (August 2004)
Collet, R., Lienhardt, M., Schmitt, A., Stefani, J.-B., Van Roy, P.: Report on formal operational semantics (components and reflection). SELFMAN Deliverable D2.3a (November 2007), www.ist-selfman.org
Encyclopaedia Britannica. Article Human Endocrine System (2005)
Fairley, P.: The Unruly Power Grid, IEEE Spectrum (October 2005)
France Télécom, Zuse Institut Berlin, and Peerialism AB. User requirements, SELFMAN Deliverable D5.1 (November 2007), http://www.ist-selfman.org
Ghodsi, A.: Distributed K-ary System: Algorithms for Distributed Hash Tables, Ph.D. dissertation, Royal Institute of Technology (KTH), Stockholm, Sweden (October 2006)
Ghodsi, A., Alima, L.O., Haridi, S.: Symmetric replication for structured peer-to-peer systems. In: Moro, G., Bergamaschi, S., Joseph, S., Morin, J.-H., Ouksel, A.M. (eds.) DBISP2P 2005. LNCS, vol. 4125, pp. 74–85. Springer, Heidelberg (2007)
Gray, J., Lamport., L.: Consensus on transaction commit. In: ACM Trans. Database Syst., pp. 133–160. ACM Press, New York (2006)
Guerraoui, R., Rodrigues, L.: Introduction to Reliable Distributed Programming. Springer, Berlin (2006)
Halim, F., Wu, Y., Yap, R.: Security Issues in Small World Network Routing. In: Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO 2008) (October 2008)
Hardin, G.: The Tragedy of the Commons. Science 162(3859), 1243–1248 (1968)
Hellerstein, J.L., Diao, Y., Parekh, S., Tilbury, D.M.: Feedback Control of Computing Systems. Wiley-IEEE Press (2004)
Hoglund, G., McGraw, G.: Exploiting Online Games: Cheating Massively Distributed Systems. Addison-Wesley Software Security Series (2008)
IBM. Autonomic computing: IBM’s perspective on the state of information technology (2001) researchweb.watson.ibm.com/autonomic
Jelasity, M., Guerraoui, R., Kermarrec, A.-M., van Steen, M.: The Peer Sampling Service: Experimental Evaluation of Unstructured Gossip-Based Implementations. In: Jacobsen, H.-A. (ed.) Middleware 2004. LNCS, vol. 3231, pp. 79–98. Springer, Heidelberg (2004)
Kim, J-R, Yoon, Y., Cho, K.-H.: Coupled Feedback Loops Form Dynamic Motifs of Cellular Networks. Biophysical Journal 94, 359–365 (2008)
Kobayashi, T., Chen, L., Aihara, K.: Modeling Genetic Switches with Positive Feedback Loops. J. theor. Biol. 221, 379–399 (2003)
Lienhard, M., Schmitt, A., Stefani, J.-B.: Oz/K: A Kernel Language for Component-Based Open Programming. In: Sixth International Conference on Generative Programming and Component Engineering (GPCE 2007) (October 2007)
Mejias, B., Van Roy, P.: A Relaxed Ring for Self-Organising and Fault-Tolerant Peer-to-Peer Networks. In: XXVI International Conference of the Chilean Computer Science Society (SCCC 2007) (November 2007)
Moser, M., Haridi, S.: Atomic Commitment in Transactional DHTs. In: Proc. of the CoreGRID Symposium. Rennes, France (August 2007)
Moser, M., Haridi, S., Schütt, T., Plantikow, S., Reinefeld, A., Schintke, F.: First report on formal models for transactions over structured overlay networks. SELFMAN Deliverable D3.1a (June 2007), www.ist-selfman.org
Pigolotti, S., Krishna, S., Jensen, M.H.: Oscillation patterns in negative feedback loops. Proc. National Academy of Sciences 104(16) (April 2007)
Plantikow, S., Reinefeld, A., Schintke, F.: Transactions for distributed wikis on structured overlays. In: Clemm, A., Granville, L.Z., Stadler, R. (eds.) DSOM 2007. LNCS, vol. 4785, pp. 256–267. Springer, Heidelberg (2007)
Salen, K., Zimmerman, E.: Rules of Play: Game Design Fundamentals. MIT Press, Cambridge (2003)
Schütt, T., Schintke, F., Reinefeld, A.: Range Queries on Structured Overlay Networks. Computer Communications 31, 280–291 (2008)
SELFMAN: Self Management for Large-Scale Distributed Systems based on Structured Overlay Networks and Components, European Commission 6th Framework Programme (June 2006), http://www.ist-selfman.org
Shafaat, T.M., Ghodsi, A., Haridi, S.: Dealing with Network Partitions in Structured Overlay Networks. Journal of Peer-to-Peer Networking and Applications (to appear, 2008)
Shafaat, T.M., Moser, M., Ghodsi, A., Schütt, T., Haridi, S., Reinefeld, A.: On Consistency of Data in Structured Overlay Networks. In: CoreGRID Integration Workshop. Heraklion, Greece. LNCS. Springer, Heidelberg (2008)
Stoica, I., Morris, R., Karger, D.R., Frans Kaashoek, M., Balakrishnan, H.: Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications. In: SIGCOMM 2001. pp. 149–160 (2001)
Van Roy, P.: Self Management and the Future of Software Design. In: Third International Workshop on Formal Aspects of Component Software (FACS 2006), ENTCS. vol. 182, pp. 201–217 (June 2007)
Weinberg, G. M.: An Introduction to General Systems Thinking: Silver Anniversary Edition. Dorset House (2001) (original edition 1975)
Whitehead, A.N.: Quote: Civilization advances by extending the number of important operations which we can perform without thinking of them
Wiener, N.: Cybernetics, or Control and Communication in the Animal and the Machine. MIT Press, Cambridge (1948)
Wiger, U.: Four-Fold Increase in Productivity and Quality – Industrial-Strength Functional Programming in Telecom-Class Products. In: Proceedings of the 2001 Workshop on Formal Design of Safety Critical Embedded Systems (2001)
Wikipedia, the free encyclopedia. Article Belief Propagation (March 2008), http://en.wikipedia.org/wiki/Belief_propagation
Wolpert, D.H., Wheeler, K.R., Tumer, K.: General principles of learning-based multi-agent systems. In: Proc. Third Annual Conference on Autonomous Agents (AGENTS 1999). pp. 77–93 (May 1999)
Wolpert, D. H., Kevin, R., Wheeler, Tumer, K.: Collective intelligence for control of distributed dynamical systems. Europhys. Lett. (2000)
Yap, R., Halim, F., Wu, Y.: First report on security in structured overlay networks. SELFMAN Deliverable D1.3a (November 2007), www.ist-selfman.org
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Van Roy, P., Haridi, S., Reinefeld, A., Stefani, JB., Yap, R., Coupaye, T. (2008). Self Management for Large-Scale Distributed Systems: An Overview of the SELFMAN Project. In: de Boer, F.S., Bonsangue, M.M., Graf, S., de Roever, WP. (eds) Formal Methods for Components and Objects. FMCO 2007. Lecture Notes in Computer Science, vol 5382. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-92188-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-540-92188-2_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-92187-5
Online ISBN: 978-3-540-92188-2
eBook Packages: Computer ScienceComputer Science (R0)