Skip to main content
SpringerLink
Log in

CLIF, a framework based on Fractal for flexible, distributed load testing

  • Published:
annals of telecommunications - annales des télécommunications Aims and scope Submit manuscript

Abstract

The context of this work is performance evaluation of IT systems based on load testing. It typically consists in generating a flow of requests on a system under test, and to measure response times, request throughput, or computing resource usage. A quick overview of available load testing platforms shows that there exist hundreds of such platforms, including in the open source domain. However, many testers still tend to develop their own ad hoc load testing tooling. Why? This paper starts by looking for possible answers to this question, in order to introduce the CLIF load injection framework, which intends not to be yet another load testing platform. Based on the Fractal component model, the CLIF open source project aims at addressing key issues such as flexibility, adaptation, and scalability. We give here details about CLIF’s architecture and associated tools as well as some feedback from a bunch of practical utilizations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. We consider simultaneity not in a real-time sense but in a weak sense. Up to now, we have not been confronted to a requirement for a synchronized start within a well-specified time slot.

  2. Application Programmatic Interface

  3. The feature of creating component types from ADL definitions was added by the Fractal core team according to a feature request from the CLIF team.

  4. The FractalRMI Registry has been jointly developed by the CLIF team and the Fractal team according to a feature request from the CLIF project.

  5. Remember the collect operation is incompletely specified on purpose.

  6. Simple Object Access Protocol, W3C recommendation.

  7. Intel bi-Xeon 2.8 GHz processor, 2GB RAM, 1 Gb/s Ethernet network interface.

  8. The threshold is commonly based on the response time mean value and standard deviation. Another common rejection algorithm consists in discarding a given percentage of the full set of measures, choosing among the greatest ones.

  9. One of the motivations being sometimes the lack of budget for a top-level professional platform, or the lack of any existing load testing platform for a particular SUT.

  10. This work is described in details in a pending submission.

  11. CLIF won a Lutèce d’Or award during the “Paris, Capitale du Libre” event in June 2007, as the best open source project made by a large company (in the name of Orange Labs).

References

  1. Barros T et al (2006) Extension of the Fractal ADL for the Specification of Behaviours of Distributed Components, poster presentation at the 5th Fractal Workshop (part of ECOOP‘06), July 3rd, 2006, Nantes, France, Jul

  2. Batista T, Joolia A, Coulson G (2005) Managing Dynamic Reconfiguration in Component-based Systems. In Proc. EWSA 2005, LNCS 3527, pp. 1–17, Springer

  3. Beauvois M (2003) Brenda: towards a composition framework for non-orthogonal non-functional properties. Proc of DAIS 2003, LNCS 2893:29–40

    Google Scholar 

  4. Bouchenak S et al (2006) Autonomic management of clustered applications. IEEE International Conference on Cluster Computing, Barcelona, Spain, September

  5. Bruneton E et al (2004) An open component model and its support in Java. Seventh International Symposium on Component-Based Software Engineering (CBSE-7). LNCS 3054:7–22 May

    Google Scholar 

  6. Cecchet E et al (2005) Implementing Probes for J2EE Cluster Monitoring. Studia Informatica, 4(1)

  7. Clif project, http://clif.ow2.org

  8. Collet P et al (2005) A Contracting System for Hierarchical Components. In Proceedings of the Component-Based Software Engineering, 8th International Symposium (CBSE‘2005), vol. 3489, pages 187–202, Springer, St-Louis (Missouri), USA, 14–15 May

  9. David P-C, Ledoux T (2006) Safe Dynamic Reconfigurations of Fractal Architectures with FScript. Proceeding of Fractal CBSE Workshop, ECOOP‘06, Nantes, France, July

  10. Dillenseger B, Cecchet E (2003) JMOB latest news, introducing CLIF. ObjectWeb Conference, Roquencourt (France), November

  11. Dillenseger B (2007) The CLIF load testing platform and its utilization at Orange Labs. Training And Research On Testing (TAROT) Summer School, Grenoble, July

  12. Eclipse Test & Performance Tools Platform project, http://www.eclipse.org/tptp

  13. Garlan D, Monroe R, Wile D (2000) ACME: architectural description of component-based systems. Foundations of Component-based Systems, Cambridge University Press, pp 47–68

  14. Halili EH (2008) Apache Jmeter, A practical beginner’s guide to automated testing and performance measurement for your websites. PACKT Publishing, ISBN 1847192955, June

  15. Harbaoui A et al (2008) Performance characterization of black boxes with self-controlled load injection for simulation-based sizing. Conférence Française sur les Systèmes d’Exploitation (CFSE‘6), Fribourg (Switzerland), February

  16. Coupaye T et al (2007) Principes architecturaux pour la construction de systèmes autonomiques ouverts. In: Krief F. and Salaün M. (eds) L’autonomie dans les réseaux (Traité IC2 série Réseaux et télécommunications), chapter 11, Hermès-Lavoisier

  17. Jain R (1991) The art of computer systems performance analysis. Wiley Professional Computing

  18. Jasmine project, http://jasmine.objectweb.org

  19. Jmeter project, http://jakarta.apache.org/jmeter

  20. Leclercq M, Quema V, Stefani J-B (2005) DREAM: a Component Framework for the Construction of Resource-Aware, Configurable MOMs. IEEE Distributed Systems Online, 6(9) September

  21. Lee JH, Lee D (2006) Description technique for component composition focusing on black-box view. Proc of ICCSA 2006, LNCS 3983:994–1002 Springer

    Google Scholar 

  22. Plasil F, Visnovsky S (2002) Behavior Protocols for Software Components. IEEE Transactions on Software Engineering 28(11), Nov

  23. Rivierre N, Coupaye T (2003) Observing component behaviors with temporal logic. In 8th ECOOP Workshop on Correctness of Model-Based Software Composition (CMC‘03)

  24. Salomé-TMF project, http://salome-tmf.ow2.org

  25. Smith CU, Williams LG (2000) Software Performance AntiPatterns. Proceedings 2nd International Workshop on Software and Performance, ACM, pp 127–136, September

  26. Szyperski C (2002) Component Software, 2nd edn. Addison-Wesley

  27. The Grinder, http://grinder.sourceforge.net

  28. Varoquaux E (2005) Ecriture en Java d’un moteur d’exécution optimisé de scénarios d’injection de charge pour une plate-forme de mesure de performances. Internship report France Telecom/ENSIMAG, Grenoble

    Google Scholar 

  29. Verlaine L, Hardange F, Biard F, Elias D (2003) Tests de performance des applications web-retours d’expérience: l’usine e-testing de France Télécom, Eyrolles

Download references

Acknowledgements

A great number of coworkers at France Telecom deserve a lot of thanks: managers supporting Fractal and CLIF as open source projects, as well as users and developers using, improving and extending the framework. Thanks also to the ObjectWeb community who supported the CLIF project. Thanks to all the contributors of CLIF, Fractal, and LeWYS projects. Thanks also to the many other open source projects CLIF relies on, not to be just a framework but a real load testing platform.

This work is supported by the French ANR through the RNTL Selfware project, and by the European IST Selfman project.

Author information

Authors and Affiliations

  1. Orange Labs, Grenoble, France

    Bruno Dillenseger

Authors
  1. Bruno Dillenseger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bruno Dillenseger.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dillenseger, B. CLIF, a framework based on Fractal for flexible, distributed load testing. Ann. Telecommun. 64, 101–120 (2009). https://doi.org/10.1007/s12243-008-0067-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12243-008-0067-9

Keywords

Search

Navigation