Skip to main content
SpringerLink
Log in

Test Automation with the Gauge Framework: Experience and Best Practices

  • Conference paper
  • First Online:
Computational Science and Its Applications – ICCSA 2020 (ICCSA 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12250))

Included in the following conference series:

  • 1384 Accesses

Abstract

While Behavior-driven development (BDD) tools such as Cucumber are powerful tools for automated testing, they have certain limitations. For example, they often enforce strict syntax for test cases, like the “Given-When-Then” format, which may not always be easy to write for a given test case. A new test automation framework named Gauge (gauge.org) addresses that limitation since it does not prescribe the BDD testing process with a strict syntax. In Gauge, writing a test case is as easy as writing down the flow of test cases in several itemized sentences in a natural language, like English.

In the context of Testinium (testinium.com), a large software testing company which provides software testing services, tools and solutions to a large number of clients, we have actively used the Gauge framework since 2018 to develop large automated front-end test suites for several large web applications.

In this paper/talk, the speakers will share several examples and best practices of developing automated tests in natural-language requirements using the Gauge framework. By learning from the ideas presented in the talk, readers (attendees) will be able to consider applying the Gauge framework in their own test automation projects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    www.mosaicinc.com/mosaicinc/successful_test.htm.

  2. 2.

    www.softwaretestinggenius.com/successful-automation-only-tool-is-not-enough/.

  3. 3.

    www.agilemodeling.com/essays/barelyGoodEnough.html.

References

  1. Garousi, V., Mäntylä, M.V.: When and what to automate in software testing? A multivocal literature review. Inf. Softw. Technol. 76, 92–117 (2016)

    Article  Google Scholar 

  2. Garousi, V., Tasli, S., Sertel, O., Tokgöz, M., Herkiloğlu, K., Arkin, H.F.E., Bilir, O.: Automated testing of simulation software in the aviation industry: an experience report. IEEE Softw. 36(4), 63–75 (2019)

    Article  Google Scholar 

  3. Smart, J.F.: BDD in Action: Behavior-Driven Development for the Whole Software Lifecycle. Manning Publications, New York (2014)

    Google Scholar 

  4. North, D.: How to sell BDD to the business. https://skillsmatter.com/skillscasts/923-how-to-sell-bdd-to-the-business. Accessed May 2020

  5. Maliackal, Z.: Why we built Gauge. https://blog.getgauge.io/why-we-built-gauge-6e31bb4848cd. Accessed May 2019

  6. Lakshmi, T.: “Why I chose Gauge over Cucumber. https://medium.com/@divi6.1990/why-i-chose-gauge-over-cucumber-5b3de478d889. Accessed May 2020

  7. Matts, C.: The tragedy of Given-When-Then. https://theitriskmanager.com/2019/04/06/the-tragedy-of-given-when-then/. Accessed May 2019

  8. Akbari, A.: Cucumber or Gauge? Things to consider when choosing the best Test Automation Framework for your team. https://www.linkedin.com/pulse/things-consider-when-choosing-best-test-automation-framework-akbari/. Accessed May 2020

  9. Garousi, V., Shepherd, D.C., Herkiloğlu, K.: Successful engagement of practitioners and software engineering researchers: evidence from 26 international industry-academia collaborative projects. IEEE Software (2019, in press)

    Google Scholar 

  10. Gorschek, T., Garre, P., Larsson, S., Wohlin, C.: A model for technology transfer in practice. IEEE Softw. 23(6), 88–95 (2006)

    Article  Google Scholar 

  11. Janicki, M., Katara, M., Pääkkönen, T.: Obstacles and opportunities in deploying model-based GUI testing of mobile software: a survey. Softw. Test. Verif. Reliab. 22(5), 313–341 (2012)

    Article  Google Scholar 

  12. Raulamo, P., Mäntylä, M.V., Garousi, V.: Choosing the right test automation tool: a grey literature review. In: International Conference on Evaluation and Assessment in Software Engineering, pp. 21–30 (2017)

    Google Scholar 

  13. Li, N., Escalona, A., Kamal, T.: Skyfire: model-based testing with cucumber. In: IEEE International Conference on Software Testing, Verification and Validation, pp. 393–400 (2016)

    Google Scholar 

  14. Sivanandan, S.: Agile development cycle: approach to design an effective Model Based Testing with Behaviour driven automation framework. In: Annual International Conference on Advanced Computing and Communications, pp. 22–25 (2014)

    Google Scholar 

  15. Graham, D., Fewster, M.: Experiences of Test Automation: Case Studies of Software Test Automation. Addison-Wesley Professional, Boston (2012)

    MATH  Google Scholar 

  16. Garousi, V., Keleş, A.B., Güler, Z.Ö., Balaman, Y.: Executable natural-language test specifications: a test-automation experience report. In: Proceedings of the Turkish National Software Engineering Symposium (UYMS) (2019)

    Google Scholar 

  17. Shewchuk, Y., Garousi, V.: Experience with maintenance of a functional GUI test suite using IBM rational functional tester. In: Proceedings of the International Conference on Software Engineering and Knowledge Engineering, pp. 489–494 (2010)

    Google Scholar 

  18. Garousi, V., Eskandar, M.M., Herkiloğlu, K.: Industry-academia collaborations in software testing: experience and success stories from Canada and Turkey. Software Qual. J. 25(4), 1091–1143 (2017)

    Article  Google Scholar 

  19. Şentürk, Ş., Akın, A., Karagöz, A.B., Garousi, V.: Model-based testing in practice: an experience report from the banking domain. In: Proceedings of the Turkish National Software Engineering Symposium (UYMS) (2019)

    Google Scholar 

  20. Urul, G., Garousi, V., Urul, G.: Test automation for embedded real-time software: an approach and experience report in the Turkish industry. In: Proceedings of the Turkish National Software Engineering Symposium (UYMS) (2014)

    Google Scholar 

  21. Akin, A., Sentürk, S., Garousi, V.: Transitioning from manual to automated software regression testing: experience from the banking domain. In: Asia-Pacific Software Engineering Conference (APSEC), pp. 591–597 (2018)

    Google Scholar 

  22. Garousi, V., Felderer, M.: Developing, verifying and maintaining high-quality automated test scripts. IEEE Softw. 33(3), 68–75 (2016)

    Article  Google Scholar 

  23. Lewis, W.E.: Software Testing and Continuous Quality Improvement. CRC Press, Boca Raton (2017)

    Book  Google Scholar 

  24. Eldh, S.: On test design. Ph.D. thesis, Mälardalen University (2011)

    Google Scholar 

  25. Garousi, V., Coşkunçay, A., Can, A.B., Demirörs, O.: A survey of software engineering practices in Turkey. J. Syst. Softw. 108, 148–177 (2015)

    Article  Google Scholar 

  26. Garousi, V., Zhi, J.: A survey of software testing practices in Canada. J. Syst. Softw. 86(5), 1354–1376 (2013)

    Article  Google Scholar 

  27. Kramer, A., Legeard, B.: Model-Based Testing Essentials-Guide to the ISTQB Certified Model-Based Tester. Wiley, Hoboken (2016)

    Book  Google Scholar 

  28. Koopman, P., Plasmeijer, R., Achten, P.: Model-based testing of thin-client web applications. In: Havelund, K., Núñez, M., Roşu, G., Wolff, B. (eds.) FATES/RV -2006. LNCS, vol. 4262, pp. 115–132. Springer, Heidelberg (2006). https://doi.org/10.1007/11940197_8

    Chapter  Google Scholar 

  29. Monsma, J.R.: Model-based testing of web applications. MSc thesis, Radboud University, The Netherlands (2015)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the European ITEA3 program via the “TESTOMAT (The Next Level of Test Automation)” project with grant number 16032, and by the Scientific and Technological Research Council of Turkey (TÜBİTAK) with grant number 9180076.

Author information

Authors and Affiliations

  1. Queen’s University Belfast, Belfast, Northern Ireland, UK

    Vahid Garousi

  2. Testinium A.Ş., Istanbul, Turkey

    Alper Buğra Keleş, Yunus Balaman & Zeynep Özdemir Güler

  3. Bahar Software Engineering Consulting Limited, Belfast, UK

    Vahid Garousi

Authors
  1. Vahid Garousi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alper Buğra Keleş
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yunus Balaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zeynep Özdemir Güler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vahid Garousi .

Editor information

Editors and Affiliations

  1. University of Perugia, Perugia, Italy

    Osvaldo Gervasi

  2. University of Basilicata, Potenza, Potenza, Italy

    Beniamino Murgante

  3. Chair- Center of ICT/ICE, Covenant University, Ota, Nigeria

    Sanjay Misra

  4. University of Cagliari, Cagliari, Italy

    Chiara Garau

  5. University of Cagliari, Cagliari, Italy

    Ivan Blečić

  6. Clayton School of Information Technology, Monash University, Clayton, VIC, Australia

    David Taniar

  7. Department of Information Science, Kyushu Sangyo University, Fukuoka, Japan

    Bernady O. Apduhan

  8. University of Minho, Braga, Portugal

    Ana Maria A.C. Rocha

  9. Polytechnic University of Bari, Bari, Italy

    Eufemia Tarantino

  10. Polytechnic University of Bari, Bari, Italy

    Carmelo Maria Torre

  11. Department of Neurology, University of Massachusetts Medical School, Worcester, MA, USA

    Yeliz Karaca

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Garousi, V., Keleş, A.B., Balaman, Y., Güler, Z.Ö. (2020). Test Automation with the Gauge Framework: Experience and Best Practices. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2020. ICCSA 2020. Lecture Notes in Computer Science(), vol 12250. Springer, Cham. https://doi.org/10.1007/978-3-030-58802-1_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58802-1_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58801-4

  • Online ISBN: 978-3-030-58802-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation