Abstract
The context of this work is performance evaluation of IT systems based on load testing. It typically consists in generating a flow of requests on a system under test, and to measure response times, request throughput, or computing resource usage. A quick overview of available load testing platforms shows that there exist hundreds of such platforms, including in the open source domain. However, many testers still tend to develop their own ad hoc load testing tooling. Why? This paper starts by looking for possible answers to this question, in order to introduce the CLIF load injection framework, which intends not to be yet another load testing platform. Based on the Fractal component model, the CLIF open source project aims at addressing key issues such as flexibility, adaptation, and scalability. We give here details about CLIF’s architecture and associated tools as well as some feedback from a bunch of practical utilizations.











Similar content being viewed by others
Notes
We consider simultaneity not in a real-time sense but in a weak sense. Up to now, we have not been confronted to a requirement for a synchronized start within a well-specified time slot.
Application Programmatic Interface
The feature of creating component types from ADL definitions was added by the Fractal core team according to a feature request from the CLIF team.
The FractalRMI Registry has been jointly developed by the CLIF team and the Fractal team according to a feature request from the CLIF project.
Remember the collect operation is incompletely specified on purpose.
Simple Object Access Protocol, W3C recommendation.
Intel bi-Xeon 2.8 GHz processor, 2GB RAM, 1 Gb/s Ethernet network interface.
The threshold is commonly based on the response time mean value and standard deviation. Another common rejection algorithm consists in discarding a given percentage of the full set of measures, choosing among the greatest ones.
One of the motivations being sometimes the lack of budget for a top-level professional platform, or the lack of any existing load testing platform for a particular SUT.
This work is described in details in a pending submission.
CLIF won a Lutèce d’Or award during the “Paris, Capitale du Libre” event in June 2007, as the best open source project made by a large company (in the name of Orange Labs).
References
Barros T et al (2006) Extension of the Fractal ADL for the Specification of Behaviours of Distributed Components, poster presentation at the 5th Fractal Workshop (part of ECOOP‘06), July 3rd, 2006, Nantes, France, Jul
Batista T, Joolia A, Coulson G (2005) Managing Dynamic Reconfiguration in Component-based Systems. In Proc. EWSA 2005, LNCS 3527, pp. 1–17, Springer
Beauvois M (2003) Brenda: towards a composition framework for non-orthogonal non-functional properties. Proc of DAIS 2003, LNCS 2893:29–40
Bouchenak S et al (2006) Autonomic management of clustered applications. IEEE International Conference on Cluster Computing, Barcelona, Spain, September
Bruneton E et al (2004) An open component model and its support in Java. Seventh International Symposium on Component-Based Software Engineering (CBSE-7). LNCS 3054:7–22 May
Cecchet E et al (2005) Implementing Probes for J2EE Cluster Monitoring. Studia Informatica, 4(1)
Clif project, http://clif.ow2.org
Collet P et al (2005) A Contracting System for Hierarchical Components. In Proceedings of the Component-Based Software Engineering, 8th International Symposium (CBSE‘2005), vol. 3489, pages 187–202, Springer, St-Louis (Missouri), USA, 14–15 May
David P-C, Ledoux T (2006) Safe Dynamic Reconfigurations of Fractal Architectures with FScript. Proceeding of Fractal CBSE Workshop, ECOOP‘06, Nantes, France, July
Dillenseger B, Cecchet E (2003) JMOB latest news, introducing CLIF. ObjectWeb Conference, Roquencourt (France), November
Dillenseger B (2007) The CLIF load testing platform and its utilization at Orange Labs. Training And Research On Testing (TAROT) Summer School, Grenoble, July
Eclipse Test & Performance Tools Platform project, http://www.eclipse.org/tptp
Garlan D, Monroe R, Wile D (2000) ACME: architectural description of component-based systems. Foundations of Component-based Systems, Cambridge University Press, pp 47–68
Halili EH (2008) Apache Jmeter, A practical beginner’s guide to automated testing and performance measurement for your websites. PACKT Publishing, ISBN 1847192955, June
Harbaoui A et al (2008) Performance characterization of black boxes with self-controlled load injection for simulation-based sizing. Conférence Française sur les Systèmes d’Exploitation (CFSE‘6), Fribourg (Switzerland), February
Coupaye T et al (2007) Principes architecturaux pour la construction de systèmes autonomiques ouverts. In: Krief F. and Salaün M. (eds) L’autonomie dans les réseaux (Traité IC2 série Réseaux et télécommunications), chapter 11, Hermès-Lavoisier
Jain R (1991) The art of computer systems performance analysis. Wiley Professional Computing
Jasmine project, http://jasmine.objectweb.org
Jmeter project, http://jakarta.apache.org/jmeter
Leclercq M, Quema V, Stefani J-B (2005) DREAM: a Component Framework for the Construction of Resource-Aware, Configurable MOMs. IEEE Distributed Systems Online, 6(9) September
Lee JH, Lee D (2006) Description technique for component composition focusing on black-box view. Proc of ICCSA 2006, LNCS 3983:994–1002 Springer
Plasil F, Visnovsky S (2002) Behavior Protocols for Software Components. IEEE Transactions on Software Engineering 28(11), Nov
Rivierre N, Coupaye T (2003) Observing component behaviors with temporal logic. In 8th ECOOP Workshop on Correctness of Model-Based Software Composition (CMC‘03)
Salomé-TMF project, http://salome-tmf.ow2.org
Smith CU, Williams LG (2000) Software Performance AntiPatterns. Proceedings 2nd International Workshop on Software and Performance, ACM, pp 127–136, September
Szyperski C (2002) Component Software, 2nd edn. Addison-Wesley
The Grinder, http://grinder.sourceforge.net
Varoquaux E (2005) Ecriture en Java d’un moteur d’exécution optimisé de scénarios d’injection de charge pour une plate-forme de mesure de performances. Internship report France Telecom/ENSIMAG, Grenoble
Verlaine L, Hardange F, Biard F, Elias D (2003) Tests de performance des applications web-retours d’expérience: l’usine e-testing de France Télécom, Eyrolles
Acknowledgements
A great number of coworkers at France Telecom deserve a lot of thanks: managers supporting Fractal and CLIF as open source projects, as well as users and developers using, improving and extending the framework. Thanks also to the ObjectWeb community who supported the CLIF project. Thanks to all the contributors of CLIF, Fractal, and LeWYS projects. Thanks also to the many other open source projects CLIF relies on, not to be just a framework but a real load testing platform.
This work is supported by the French ANR through the RNTL Selfware project, and by the European IST Selfman project.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dillenseger, B. CLIF, a framework based on Fractal for flexible, distributed load testing. Ann. Telecommun. 64, 101–120 (2009). https://doi.org/10.1007/s12243-008-0067-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12243-008-0067-9