Skip to main content
SpringerLink
Log in

Crawlability metrics for automated web testing

  • WSE 2009
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Web applications are exposed to frequent changes both in requirements and involved technologies. At the same time, there is a continuously growing demand for quality and trust and such a fast evolution and quality constraints claim for mechanisms and techniques for automated testing. Web application automated testing often involves random crawlers to navigate the application under test and automatically explore its structure. However, owing to the specific challenges of the modern Web systems, automatic crawlers may leave large portions of the application unexplored. In this paper, we propose the use of structural metrics to predict whether an automatic crawler with given crawling capabilities will be sufficient or not to achieve high coverage of the application under test. In this work, we define a taxonomy of such capabilities and we determine which combination of them is expected to give the highest reward in terms of coverage increase. Our proposal is supported by an experiment in which 19 web applications have been analyzed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alshahwan, N., Harman, M.: Automated session data repair for web application regression testing. In: ICST ’08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation, pp. 298–307. IEEE Computer Society, Washington (2008)

  2. Alshahwan, N., Harman, M., Marchetto, A., Tonella, P.: Improving web application testing using testability measure. In: IEEE International Symposium on Web Systems Evolution (WSE), IEEE Computer Society, Edmonton (September 2009)

  3. Anneliese J.O., Andrews A., Alexander R.T.: Testing web applications by modeling with fsms. Softw. Syst. Model. 4, 326–345 (2005)

    Article  Google Scholar 

  4. Artzi, S., Kiezun, A., Dolby, J., Tip, F., Dig, D., Paradkar, A., Ernst, M.D.: Finding bugs in dynamic web applications. In: ISSTA ’08: Proceedings of the 2008 International Symposium on Software Testing and Analysis, pp. 261–272. ACM, New York (2008)

  5. Bellettini, C., Marchetto, A., Trentini, A.: TestUml: user-metrics driven web applications testing. In: SAC ’05: Proceedings of the 2005 ACM Symposium on Applied Computing, pp. 1694–1698. ACM, New York (2005)

  6. Bruntink M., van Deursen A.: An empirical study into class testability. J. Syst. Softw. 79(9), 1219–1232 (2006)

    Article  Google Scholar 

  7. Conallen, J.: Modeling web applications with uml. In: White Paper (1999)

  8. Conallen J.: Building Web Applications with UML. Addison-Wesley, Reading (2000)

    Google Scholar 

  9. Dhyani D., Ng W.K., Bhowmick S.S.: A survey of web metrics. ACM Comput. Surv. 34(4), 469–503 (2002)

    Article  Google Scholar 

  10. Douglis, F., Feldmann, A., Krishnamurthy, B., Mogul, J.: Rate of change and other metrics: a live study of the world wide web. In: USITS’97: Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems. USENIX Association, Berkeley (1997)

  11. Elbaum, S., Karre, S., Rothermel, G.: Improving Web application testing with user session data. In: Proceedings of the 25th International Conference on Software Engineering (ICSE), pp. 49–59. IEEE Computer Society, Portland (May 2003)

  12. Elbaum S., Rothermel G., Karre S., Fisher M. II: Leveraging user-session data to support web application testing. IEEE Trans. Softw. Eng. 31(3), 187–202 (2005)

    Article  Google Scholar 

  13. Fisher, M. II, Elbaum, S., Rothermel, G.: Dynamic characterization of web application interfaces. In: FASE’07: Proceedings of the Fundamental Approaches to Software Engineering, pp. 260–275. Springer, Berlin (2007)

  14. Ghosheh, E., Qaddour, J., Kuofie, M., Black, S.: A comparative analysis of maintainability approaches for web applications. In: AICCSA ’06: Proceedings of the IEEE International Conference on Computer Systems and Applications, pp. 1155–1158, IEEE Computer Society, Washington (2006)

  15. Girardi C., Ricca F., Tonella P.: Web crawlers compared. Int. J. Web Inform. Syst. 2, 85–94 (2006)

    Article  Google Scholar 

  16. Halfond, W.G., Anand, S., Orso, A.: Precise interface identification to improve testing and analysis of web applications. In: ISSTA ’09: Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, pp. 285–296. ACM, New York (2009)

  17. Halfond, W.G.J., Orso, A.: Improving test case generation for web applications using automated interface discovery. In: ESEC-FSE ’07: Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering, pp. 145–154. ACM, New York (2007)

  18. Halfond, W.G.J., Orso, A.: Automated identification of parameter mismatches in web applications. In: SIGSOFT ’08/FSE-16: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 181–191. ACM, New York (2008)

  19. Harman, M.: Open problems in testability transformation (keynote). In: 1st International Workshop on Search Based Testing (SBT 2008), Lillehammer (2008)

  20. Harman M., Hu L., Hierons R.M., Wegener J., Sthamer H., Baresel A., Roper M.: Testability transformation. IEEE Trans. Softw. Eng. 30(1), 3–16 (2004)

    Article  Google Scholar 

  21. Heydon A., Najork M.: Mercator: a scalable, extensible web crawler. World Wide Web 2(4), 219–229 (1999)

    Article  Google Scholar 

  22. Hieatt E., Mee R., Faster G.: Testing the web application engineering internet. IEEE Softw. 19(2), 60–65 (2002)

    Article  Google Scholar 

  23. Jungmayr, S.: Testability measurement and software dependencies. In: Proceedings of the 12th International Workshop on Software Measurement, pp. 179–202. Magdeburg,Shaker Publication, Aachen (2002)

  24. Kan S.H.: Metrics and models in software quality engineering. Addison-Wesley, Reading (2003)

    Google Scholar 

  25. Kim K.-H., Kim Y.-G.: Process reverse engineering for bpr: a form-based approach. J. Inform. Manage. 23(4), 187–200 (1998)

    Article  Google Scholar 

  26. Lakhotia, K., McMinn, P., Harman, M.: Automated test data generation for coverage: Haven’t we solved this problem yet? In: 4th Testing Academia and Industry Conference—Practice and Research Techniques (TAIC PART’09), pp. 95–104. IEEE Computer Society, Washington (2009)

  27. Marchetto, A., Tonella, P., Ricca, F.: State-based testing of ajax web applications. In: International Conference on Software Testing Verification and Validation (ICST), IEEE Computer Society Lillehammer (April 2008)

  28. McCabe T.: A complexity measure. IEEE Trans. Softw. Eng. 2(4), 308–320 (1976)

    Article  MathSciNet  Google Scholar 

  29. Mesbah, A., van Deursen, A.: Invariant-based automatic testing of ajax user interfaces. In: 31st International Conference on Software Engineering (ICSE). IEEE Computer Society (May 2009)

  30. Miller R.C., Bharat K.: Sphinx: a framework for creating personal, site-specific web crawlers. Comput. Netw. ISDN Syst. 30(1–7), 119–130 (1998)

    Article  Google Scholar 

  31. Palmer J.W.: Web site usability, design, and performance metrics. Info. Syst. Res. 13(2), 151–167 (2002)

    Article  Google Scholar 

  32. Ricca, F., Tonella, P.: Analysis and testing of web applications. In: ICSE ’01: Proceedings of the 23rd International Conference on Software Engineering, pp. 25–34. IEEE Computer Society, Washington (2001)

  33. Sampath, S., Bryce, R.C., Viswanath, G., Kandimalla, V., Koru, A.G.: Prioritizing user-session-based test cases for web applications testing. In: ICST ’08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation, pp. 141–150. IEEE Computer Society, Washington (2008)

  34. Sampath, S., Sprenkle, S., Gibson, E., Pollock, L., Souter, A.: Analyzing clusters of web application user sessions. In: WODA ’05: Proceedings of the Third International Workshop on Dynamic Analysis, pp. 1–7. ACM, New York (2005)

  35. Sampath S., Sprenkle S., Gibson E., Pollock L., Greenwald A. Souter: Applying concept analysis to user-session-based testing of web applications. IEEE Trans. Softw. Eng. 33(10), 643–658 (2007)

    Article  Google Scholar 

  36. Shepperd M.J.: Foundations of software measurement. Prentice Hall, New Jersey (1995)

    MATH  Google Scholar 

  37. Sprenkle, S., Sampath, S., Gibson, E., Pollock, L., Souter, A.: An empirical comparison of test suite reduction techniques for user-session-based testing of web applications. In: ICSM ’05: Proceedings of the 21st IEEE International Conference on Software Maintenance, pp. 587–596, IEEE Computer Society Washington (2005)

  38. Tip F.: A survey of program slicing techniques. J. Program. Lang. 3(3), 121–189 (1995)

    Google Scholar 

  39. Warren, P., Boldyreff, C., Munro, M.: The evolution of websites. In: IWPC ’99: Proceedings of the 7th International Workshop on Program Comprehension, p. 178. IEEE Computer Society, Washington (1999)

  40. Wassermann, G., Yu, D., Chander, A., Dhurjati, D., Inamura, H., Su, Z.: Dynamic test input generation for web applications. In: ISSTA ’08: Proceedings of the 2008 International Symposium on Software Testing and Analysis, pp. 249–260. ACM, New York (2008)

  41. Weiser M.: Program slicing. IEEE Trans. Softw. Eng. 10(4), 352–357 (1984)

    Article  Google Scholar 

  42. Wu, Y., Offutt, J.: Modeling and testing web-based applications. Technical report, George Mason University (2002)

Download references

Author information

Authors and Affiliations

  1. Fondazione Bruno Kessler, IRST, Trento, Italy

    Alessandro Marchetto, Roberto Tiella & Paolo Tonella

  2. Kings College London, CREST Centre, Strand, London, UK

    Nadia Alshahwan & Mark Harman

Authors
  1. Alessandro Marchetto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roberto Tiella
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paolo Tonella
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nadia Alshahwan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mark Harman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandro Marchetto.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Marchetto, A., Tiella, R., Tonella, P. et al. Crawlability metrics for automated web testing. Int J Softw Tools Technol Transfer 13, 131–149 (2011). https://doi.org/10.1007/s10009-010-0177-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-010-0177-3

Keywords

Search

Navigation