Skip to main content
SpringerLink
Log in

A successful application of a Test-Driven Development strategy in the industrial environment

  • Experience Report
  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Unit Test-Driven Development (UTDD) and Acceptance Test-Driven Development (ATDD) are software development techniques to incrementally develop software where the test cases, unit or acceptance tests respectively, are specified before the functional code. There are little empirical evidences supporting or refuting the utility of these techniques in an industrial context. Just a few case studies can be found in literature within the industrial environment and they show conflicting results (positive, negative and neutral). In this report, we present a successful application of UTDD in combination with ATDD in a commercial project. By successful we mean that the project goals are reached without an extra economic cost. All the UTDD and ATDD implementations are based on the same basic concepts, but they may differ in specific adaptations to each project or team. In the implementation presented here, the business requirements are specified by means of executable acceptance tests, which then are the input of a development process where the functional code is written in response to specific unit tests. Our goal is to share our successful experience in a specific project from an empirical point of view. We highlight the advantages and disadvantages of adopting UTDD and ATDD and identify some conditions that facilitate success. The main conclusions we draw from this project are that ATDD contributes to clearly capture and validate the business requirements, but it requires an extensive cooperation from the customer; and that UTDD has not a significant impact neither on productivity nor on software quality. These results cannot be generalized, but they point out that under some circumstances a test-driven development strategy can be a possible option to take into account by software professionals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://struts.apache.org

  2. http://www.springsource.org

  3. http://www.prototypejs.org

  4. http://jquery.com

  5. http://www.postgresql.org

  6. http://www.mybatis.org/

  7. http://fitnesse.org/

  8. http://www.concordion.org

  9. http://cukes.info/

  10. In Concordion the term example denotes sentences, tables or whatever that demonstrate the specified behavior for a feature. Later these examples will be linked with the functional code through JUnit test cases known as fixtures.

References

  • Adzic G (2011) Specification by example: how successful teams deliver the right software. Manning Publications Co

  • Andersson J, Bache G, Sutton P (2003) Xp with acceptance-test driven development: a rewrite project for a resource optimization system. In: Proceedings of the 4th international conference on extreme programming and agile processes in software engineering, XP’03. Springer, Berlin, Heidelberg. ISBN 3-540-40215-2, pp 180–188. http://dl.acm.org/citation.cfm?id=1763875.1763904

    Chapter  Google Scholar 

  • Astels D (2003) Test driven development: a practical guide. Prentice Hall Professional Technical Reference. ISBN 0131016490

  • Basili VR, Shull F, Lanubile F (1999) Building knowledge through families of experiments. IEEE Trans Softw Eng 25(4):456–473. ISSN 0098-5589

    Article  Google Scholar 

  • Beck K (2001) Aim, fire. IEEE Softw 18(5):87–89. ISSN 0740-7459

    Article  Google Scholar 

  • Beck K (2003) Test driven development: by example. Addison-Wesley Professional (2003)

  • Beck K, Andres C (2004) Extreme programming explained: embrace change, 2nd edn. Addison-Wesley Professional. ISBN 0321278658

  • Bhat T, Nagappan N (2006) Evaluating the efficacy of test-driven development: industrial case studies. In: Proceedings of the 2006 ACM/IEEE international symposium on empirical software engineering, ISESE ’06. ACM, New York. ISBN 1-59593-218-6, pp 356–363

    Chapter  Google Scholar 

  • Buschmann F, Meunier R, Rohnert H, Sommerlad P, Stal M (1996) Pattern-oriented software architecture: a system of patterns. Wiley. ISBN 0471958697

  • Cao L, Ramesh B (2008) Agile requirements engineering practices: an empirical study. IEEE Softw 25(1):60–67

    Article  Google Scholar 

  • Causevic A, Sundmark D, Punnekkat S (2011) Factors limiting industrial adoption of test driven development: a systematic review. In: 2011 IEEE fourth international conference on software testing, verification and validation (ICST). IEEE, pp 337–346

  • Cockburn A (2002) Agile software development. Addison-Wesley Longman Publishing Co., Inc., Boston. ISBN 0-201-69969-9

    Google Scholar 

  • Crispin L (2006) Driving software quality: how test-driven development impacts software quality. IEEE Softw 23(6):70–71

    Article  Google Scholar 

  • Damm L-O, Lundberg L (2006) Results from introducing component-level test automation and test-driven development. J Syst Softw 79(7):1001–1014. ISSN 0164-1212

    Article  Google Scholar 

  • Dustin E, Rashka J, Paul J (1999) Automated software testing: introduction, management, and performance. Addison-Wesley Longman Publishing Co., Inc., Boston. ISBN 0-201-43287-0

    Google Scholar 

  • Erdogmus H, Morisio M, Torchiano M (2005) On the effectiveness of the test-first approach to programming. IEEE Trans Softw Eng 31(3):226–237. ISSN 0098-5589

    Article  Google Scholar 

  • Erdogmus H, Melnik G, Jeffries R (2010) Test-driven development. In: Encyclopedia of software engineering, pp 1211–1229

  • Ficco M, Pietrantuono R, Russo S (2011) Bug localization in test-driven development. Adv Soft Eng 2011:2:1–2:18. ISSN 1687-8655. doi:10.1155/2011/492757

    Google Scholar 

  • Flohr T, Schneider T (2006) Lessons learned from an xp experiment with students: test-first needs more teachings. In: Proceedings of the 7th international conference on product-focused software process improvement, PROFES’06. Springer, Berlin, Heidelberg. ISBN 3-540-34682-1, 978-3-540-34682-1, pp 305–318

    Chapter  Google Scholar 

  • Gärtner M (2012) ATDD by example: a practical guide to acceptance test-driven development. Addison-Wesley Professional

  • George B, Williams L (2003) An initial investigation of test driven development in industry. In: Proceedings of the 2003 ACM symposium on applied computing, SAC ’03. ACM, New York. ISBN 1-58113-624-2, pp 1135–1139

    Chapter  Google Scholar 

  • Gupta A, Jalote P (2007) An experimental evaluation of the effectiveness and efficiency of the test driven development. In: Proceedings of the first international symposium on empirical software engineering and measurement, ESEM ’07. IEEE Computer Society, Washington. ISBN 0-7695-2886-4, pp 285–294

    Chapter  Google Scholar 

  • Haugset B, Stalhane T (2012) Automated acceptance testing as an agile requirements engineering practice. In: 2012 45th Hawaii international conference on system science (HICSS). IEEE, pp 5289–5298

  • Hendricksons E (2008) Acceptance test driven development (atdd): an overview. In: Seventh software testing Australia/New Zealand (STANZ). Wellington

  • Höfer A, Philipp M (2009) An empirical study on the tdd conformance of novice and expert pair programmers. In: XP, pp 33–42

  • Janzen D, Saiedian H (2005) Test-driven development: concepts, taxonomy, and future direction. Computer 38(9):43–50. ISSN 0018-9162

    Article  Google Scholar 

  • Janzen DS, Saiedian H (2006) On the influence of test-driven development on software design. In: Proceedings of the 19th conference on software engineering education & training, CSEET ’06. IEEE Computer Society, Washington. ISBN 0-7695-2557-1, pp 141–148

    Chapter  Google Scholar 

  • Janzen D, Saiedian H (2008) Does test-driven development really improve software design quality? IEEE Softw 25(2):77–84. ISSN 0740-7459

    Article  Google Scholar 

  • Jeffries R, Melnik G (2007) Guest editors’ introduction: Tdd–the art of fearless programming. IEEE Softw 24(3):24–30. ISSN 0740-7459

    Article  Google Scholar 

  • Kaufmann R, Janzen D (2003) Implications of test-driven development: a pilot study. In: Companion of the 18th annual ACM SIGPLAN conference on object-oriented programming, systems, languages, and applications, OOPSLA ’03. ACM, New York. ISBN 1-58113-751-6, pp 298–299

    Chapter  Google Scholar 

  • Koskela L (2007) Test driven: practical tdd and acceptance tdd for java developers. Manning Publications Co., Greenwich. ISBN 9781932394856

    Google Scholar 

  • Larman C, Basili VR (2003) Iterative and incremental development: a brief history. Computer 36(6):47–56. ISSN 0018-9162

    Article  Google Scholar 

  • Latorre R, López F, Martínez, AE (2005) Sharing of precompiled database statements in j2ee applications. Softw Pract Exper 35(3):301–311. ISSN 0038-0644

    Article  Google Scholar 

  • Leffingwell D (2010) Agile software requirements: lean requirements practices for teams, programs, and the enterprise. Addison-Wesley Professional

  • Madeyski L (2005) Preliminary analysis of the effects of pair programming and test-driven development on the external code quality. In: Proceedings of the 2005 conference on software engineering: evolution and emerging technologies. IOS Press, Amsterdam. ISBN 1-58603-559-2, pp 113–123

    Google Scholar 

  • Martin RC (2003) Agile software development: principles, patterns, and practices. Prentice Hall PTR, Upper Saddle River. ISBN 0135974445

    Google Scholar 

  • Martin RC (2007) Professionalism and test-driven development. IEEE Softw 24(3):32–36. ISSN 0740-7459

    Article  Google Scholar 

  • Martin RC (2008) Clean code: a handbook of Agile software craftsmanship. Prentice Hall, Upper Saddle River. ISBN 978-0-13235-088-4

    Google Scholar 

  • Maximilien EM, Williams LA (2003) Assessing test-driven development at IBM. In: Clarke LA, Dillon L, Tichy WF (eds) ICSE. IEEE Computer Society, pp 564–569

  • Müller MM, Hagner O (2002) Experiment about test-first programming. IEE Proc-Softw 149(5):131–136

    Article  Google Scholar 

  • Nagappan N, Maximilien EM, Bhat T, Williams L (2008) Realizing quality improvement through test driven development: results and experiences of four industrial teams. Empir Software Eng 13(3):289–302. ISSN 1382-3256

    Article  Google Scholar 

  • Newkirk JW, Vorontsov AA (2004) Test-driven development in microsoft .net. Microsoft Press, Redmond. ISBN 0735619484

    Google Scholar 

  • Park S, Maurer F (2009) Communicating domain knowledge in executable acceptance test driven development. In: Abrahamsson P, Marchesi M, Maurer F (eds) Agile processes in software engineering and extreme programming. Lecture notes in business information processing, vol 31. Springer Berlin Heidelberg. ISBN 978-3-642-01852-7, pp 23–32

    Chapter  Google Scholar 

  • Pugh K (2010) Lean-agile acceptance test-driven development: better software through collaboration. Addison-Wesley Professional

  • Rafique Y, Misic VB (2013) The effects of test-driven development on external quality and productivity: a meta-analysis. Softw Eng IEEE Trans 39(6):835–856. ISSN 0098-5589. doi:10.1109/TSE.2012.28

    Article  Google Scholar 

  • Ricca F, Torchiano M, Di Penta M, Ceccato M, Tonella P (2009) Using acceptance tests as a support for clarifying requirements: a series of experiments. Inf Softw Technol 51(2):270–283. ISSN 0950-5849. doi:10.1016/j.infsof.2008.01.007

    Article  Google Scholar 

  • Sanchez JC, Williams L, Maximilien EM (2007) On the sustained use of a test-driven development practice at ibm. In: AGILE Conference (AGILE), pp 5–14

  • Shadish WR, Cook TD, Campbell DT (2001) Experimental and quasi-experimental designs for generalized causal inference, 2nd edn. Houghton Mifflin. ISBN 0395615569

  • Williams L, Maximilien EM, Vouk M (2003) Test-driven development as a defect-reduction practice. In: Proceedings of the 14th international symposium on software reliability engineering, ISSRE ’03. IEEE Computer Society, Washington. ISBN 0-7695-2007-3, pp 34–45

    Google Scholar 

Download references

Acknowledgements

The author would like to thank Ramón Huerta for his comments and suggestions on the initial version of the manuscript. I am also very grateful to the insightful and helpful feedback of the anonymous reviewers which helped substantially to improve the paper.

Author information

Authors and Affiliations

  1. Dpto. Ingenier’ia Inform’atica, Escuela Polit’ecnica Superior, Universidad Aut’onoma de Madrid, Madrid, 28049, Spain

    Roberto Latorre

  2. NeuroLogic Soluciones Informáticas, 28037, Madrid, Spain

    Roberto Latorre

Authors
  1. Roberto Latorre
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roberto Latorre.

Additional information

Communicated by: Brian Robinson

Rights and permissions

Reprints and permissions

About this article

Cite this article

Latorre, R. A successful application of a Test-Driven Development strategy in the industrial environment. Empir Software Eng 19, 753–773 (2014). https://doi.org/10.1007/s10664-013-9281-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-013-9281-9

Keywords

Search

Navigation