Skip to main content
SpringerLink
Log in

Using Surveys and Web-Scraping to Select Tools for Software Testing Consultancy

  • Conference paper
  • First Online:
Product-Focused Software Process Improvement (PROFES 2016)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10027))

Abstract

We analyzed findings from data collected utilizing surveys and Web-scraping, to support Knowit Oy, a software testing consultation company, in the process of selecting the right tools for software testing & test automation. We conducted two surveys (2013 & 2016) among (mostly Finnish) software professionals to acquire criteria and a list of tools used for software testing in industry. Considering all our data sources Selenium was the most popular pure tool, while Robot Framework was the most referenced tool (latter survey). According to the surveys Jenkins and Sikuli have the highest increase in popularity (or familiarity). Top referred criteria for selection were usability, functionality, maintainability and available support for a tool. While Knowit considers it best to utilize traditional surveys, Web-scraping is seen as cost effective support for such instruments. To get comprehensive picture and to gain knowledge of the tools in markets multiple sources should be used.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://wikitech.wikimedia.org/wiki/Analytics/PageviewAPI.

  2. 2.

    https://www.google.com/intl/en/policies/terms/?fg=1.

  3. 3.

    http://data.stackexchange.com/.

  4. 4.

    https://github.com/Jefferson-Henrique/GetOldTweets-java/.

  5. 5.

    http://iso25000.com/index.php/en/iso-25000-standards/iso-25010.

References

  1. Capgemini Consulting: World Quality Report 2015–2016. (2015). https://www.capgemini.com/thought-leadership/world-quality-report-2015-16

  2. Fewster, M., Graham, D.: Software Test Automation: Effective Use of Test Execution Tools. ACM Press/Addison-Wesley Publishing Co (1999)

    Google Scholar 

  3. Garousi, V., Zhi, J.: A survey of software testing practices in Canada. J. Syst. Softw. 86(5), 1354–1376 (2013)

    Article  Google Scholar 

  4. Hussain, S., Wang, Z., Toure, I. K., Diop, A.: Web service testing tools: A comparative study. arXiv preprint arXiv:1306.4063 (2013)

  5. ISTQB (International Software Testing Qualifications Board): ISTQB® Worldwide Software Testing Practices Report 2015–2016 (2016). http://www.istqb.org/references/surveys/istqb-worldwide-software-testing-practices-report.html

  6. Kaur, H., Gupta, G.: Comparative study of automated testing tools: selenium, quick test professional and testcomplete. Int. J. Eng. Res. Appl. 3(5), 1739–1743 (2013)

    Google Scholar 

  7. Kaur, M., Kumari, R.: Comparative study of automated testing tools: testcomplete and quicktest pro. Int. J. Comput. Appl. 24(1), 1–7 (2011)

    Google Scholar 

  8. Michael, J.B., Bossuyt, B.J., Snyder, B.B.: Metrics for measuring the effectiveness of software-testing tools. In: 13th International Symposium on Software Reliability Engineering, pp. 117–128. IEEE (2003)

    Google Scholar 

  9. Ng, S., Murnane, T., Reed, K., Grant, D., Chen, T.: A preliminary survey on software testing practices in Australia. In: 2004 Australian Software Engineering Conference, pp. 116–125. IEEE (2004)

    Google Scholar 

  10. Pan, A., Raposo, J., Álvarez, M., Hidalgo, J., Viña, Á.: Semi-automatic wrapper generation for commercial web sources. In: Rolland, C., Brinkkemper, S., Saeki, M. (eds.). IFIP, vol. 103, pp. 265–283. Springer, Heidelberg (2002). doi:10.1007/978-0-387-35614-3_16

    Chapter  Google Scholar 

  11. Petersen, K., Wohlin, C.: Context in industrial software engineering research. In: 3rd International Symposium on Empirical Software Engineering and Measurement, pp. 401–404. IEEE (2009)

    Google Scholar 

  12. Portillo-Rodríguez, J., Vizcaíno, A., Piattini, M., Beecham, S.: Tools used in global software engineering: a systematic mapping review. Inf. Softw. Technol. 54(7), 663–685 (2012)

    Article  Google Scholar 

  13. Poston, R.M., Sexton, M.P.: Evaluating and selecting testing tools. Software 9(3), 33–42, IEEE (1992)

    Google Scholar 

  14. Surowiecki, J.: The Wisdom of Crowds. Anchor, New York (2005)

    Google Scholar 

  15. Tassey, G.: The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and Technology, RTI Project, 7007.011 (2002)

    Google Scholar 

  16. Tiitinen, M., Kakkonen, K.: Software Testing Tools Research in Finland, Knowit Oy (2013). http://www.knowit.fi/Documents/Testausty%C3%B6kalututkimus_Posteri_A3_Tiitinen_V11_ENG.pdf

  17. Wikipedia Page view. https://en.wikipedia.org/wiki/Page_view

  18. Wikipedia article traffic. https://en.wikipedia.org/wiki/Wikipedia_article_traffic

  19. Yehezkel, S.: Test Automation Survey (2016). http://blog.testproject.io/2016/03/16/test-automation-survey-2016/

Download references

Acknowledgments

Our thanks to all respondents for their contribution to this survey.

Author information

Authors and Affiliations

  1. M3S (M-Group), ITEE University of Oulu, Oulu, Finland

    Päivi Raulamo-Jurvanen & Mika Mäntylä

  2. Knowit Oy, Helsinki, Finland

    Kari Kakkonen

Authors
  1. Päivi Raulamo-Jurvanen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kari Kakkonen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mika Mäntylä
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Päivi Raulamo-Jurvanen .

Editor information

Editors and Affiliations

  1. Norwegian University of Science and Technology, Trondheim, Norway

    Pekka Abrahamsson

  2. Fraunhofer Institute for Experimental Software Engineering, Kaiserslautern, Germany

    Andreas Jedlitschka

  3. and Technology, Norwegian University of Science and Technology, Trondheim, Norway

    Anh Nguyen Duc

  4. University of Innsbruck, Innsbruck, Austria

    Michael Felderer

  5. Okayama Prefectural University, Soja, Japan

    Sousuke Amasaki

  6. Tampere University of Technology, Tampere, Finland

    Tommi Mikkonen

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Raulamo-Jurvanen, P., Kakkonen, K., Mäntylä, M. (2016). Using Surveys and Web-Scraping to Select Tools for Software Testing Consultancy. In: Abrahamsson, P., Jedlitschka, A., Nguyen Duc, A., Felderer, M., Amasaki, S., Mikkonen, T. (eds) Product-Focused Software Process Improvement. PROFES 2016. Lecture Notes in Computer Science(), vol 10027. Springer, Cham. https://doi.org/10.1007/978-3-319-49094-6_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49094-6_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49093-9

  • Online ISBN: 978-3-319-49094-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation