Skip to main content
SpringerLink
Log in

A case study on bypass testing of web applications

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Society’s increasing reliance on services provided by web applications places a high demand on their reliability. The flow of control through web applications heavily depends on user inputs and interactions, so user inputs should be thoroughly validated before being passed to the back-end software. Although several techniques are used to validate inputs on the client, users can easily bypass this validation and submit arbitrary data to the server. This can cause unexpected behavior, and even allow unauthorized access. A test technique called bypass testing intentionally sends invalid data to the server by bypassing client-side validation. This paper reports results from a comprehensive case study on 16 deployed, widely used, commercial web applications. As part of this project, the theory behind bypass testing was extended and an automated tool, AutoBypass, was built. The case study found failures in 14 of the 16 web applications tested, some significant. This study gives evidence that bypass testing is effective, has positive return on investment, and scales to real applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. To emphasize that this research tests programs, not static web sites, we use the term “screens” when the web page is used as part of the web application.

  2. The screenshot has a typo. “resume” should have been “assumes.”

References

  • Ammann P, Offutt J (1994) Using formal methods to derive test frames in category-partition testing. In: Proceedings of the ninth annual conference on computer assurance (COMPASS 94). IEEE Computer Society, Gaithersburg, pp 69–80

    Google Scholar 

  • Andrews A, Offutt J, Alexander R (2005) Testing Web applications by modeling with FSMs. Softw Syst Model, Springer 4(3):326–345

    Article  Google Scholar 

  • Andrews A, Offutt J, Dyreson C, Mallery CJ, Jerath K, Alexander R (2010) Scalability issues with using FSMWeb to test web applications. Inf Softw Technol (Elsevier) 52(1):52–66. doi:10.1016/j.infsof.2009.06.002

    Article  Google Scholar 

  • Cooper A, Reimann R (2003) Designing for the Web, about face 2.0: the essentials of interaction design. Wiley, New York

    Google Scholar 

  • DeMillo RA, Offutt J (1991) Constraint-based automatic test data generation. IEEE Trans Softw Eng 17(9):900–910

    Article  Google Scholar 

  • Eaton C, Memon AM (2007) An empirical approach to testing web applications across diverse client platform configurations. International Journal on Web Engineering and Technology (IJWET), Special Issue on Empirical Studies in Web Engineering 3(3):227–253

    Google Scholar 

  • ChristeyKarre S, Rothermel G S (ed) (2011) CWE/SANS top 25 most dangerous software errors. Online, September 2011. http://cwe.mitre.org/top25, last access April 2012

  • Elbaum S, Karre S, Rothermel G (2003) Improving Web application testing with user session data. In: Proceedings of the 25th international conference on software engineering. IEEE Computer Society, Portland, pp 49–59

    Google Scholar 

  • Elbaum S, Rothermel G, Karre S, Fisher M (2005) Leveraging user-session data to support web application testing. IEEE Trans Softw Eng 31(3):187–202

    Article  Google Scholar 

  • Gold R (2003) Httpunit home. online: SourceForge. http://httpunit.sourceforge.net/, last access June 2005

  • Halfond WGJ, Orso A (2007) Improving test case generation for web applications using automated interface discovery. In: Proceedings of the foundations of software engineering. Dubrovnik, Croatia, pp 145–154

    Google Scholar 

  • Harrold MJ (2000) Testing: A roadmap. In: International conference on software engineering, workshop on the future of software engineering. Limerick, Ireland, pp 61–72

    Google Scholar 

  • Hilburn T, Towhidnejad M (2002) Software quality across the curriculum. In: Proceedings of the 15th conference on software engineering education and training

  • Hower R (2002) Web site test tools and site management tools. www.softwareqatest.com/qatweb1.html

  • Joomla (2007) The Joomla project. Online: http://www.joomla.org/. Last accessed June 2012

  • Kung D, Liu CH, Hsia P (2000) An object-oriented Web test model for testing Web applications. In: 24th annual international computer software and applications conference (COMPSAC2000). IEEE Computer Society, Taipei, pp 537–542

    Chapter  Google Scholar 

  • Liu CH, Kung D, Hsia P, Hsu CT (2000) Structural testing of Web applications. In: Proceedings of the 11th international symposium on software reliability engineering. IEEE Computer Society, San Jose, pp 84–96

    Google Scholar 

  • Lucca GAD, Penta MD (2003) Considering browser interaction in web application testing. In: 5th international workshop on web site evolution (WSE 2003). IEEE Computer Society, Amsterdam, pp 74–84

    Google Scholar 

  • Miller J, Zhang L, Ofuonye E, Smith M (2010) Towards automated bypass testing of web applications. In: Web engineering advancements and trends. IGI, pp 212–229

  • Mouelhi T, Traon YL, Abgrall E, Baudry B, Gombault S (2011) Tailored shielding and bypass testing of web applications. In: 4th international conference on software testing, verification and validation (ICST). IEEE, Berlin, pp 210–219

    Google Scholar 

  • Nguyen-Tuong A, Guarnieri S, Greene D, Shirley J, Evans D (2005) Automatically hardening Web applications using precise tainting. In: Proceedings of the 20th IFIP international information security conference. Makuhari-Messe, Chiba, Japan

  • Offutt J (2002) Quality attributes of Web software applications. IEEE Softw (Special Issue on Software Engineering of Internet Software) 19(2):25–32

    Google Scholar 

  • Offutt J, Jin Z Pan J (1999) The dynamic domain reduction approach to test data generation. Softw Pract Exp 29(2):167–193

    Article  Google Scholar 

  • Offutt J, Wu Y, Du X, Huang H (2004a) Bypass testing of web applications. In: 15th international symposium on software reliability engineering. IEEE Computer Society, Saint-Malo, pp 187–197

    Chapter  Google Scholar 

  • Offutt J, Wu Y, Du X, Huang H (2004b) Web application bypass testing. In: Workshop on quality assurance and testing of web-based applications; associated with COMPSAC 2004. PRC, Hong Kong, pp 106–109

    Google Scholar 

  • Offutt J, Wu Y (2010) Modeling presentation layers of web applications for testing. Softw Syst Model 9(2):257–280. doi:10.1007/s10270-009-0125-4

    Article  Google Scholar 

  • Papadimitriou V (2006) Automating bypass testing for web applications. Master’s thesis, Department of Information and Software Engineering, George Mason University, Fairfax VA. Available on the web at: http://www.cs.gmu.edu/~offutt/

  • Raggett D, Hors AL, Jacobs I (1999) HTML 4.01 specification—W3C recommendation 24. World Wide Web Consortium (W3C). Online: http://www.w3.org/TR/html401/, last access September 2007

  • Ricca F, Tonella P (2001) Analysis and testing of web applications. In: IEEE 23rd international conference on software engineering (ICSE ‘01). Toronto, CA, pp 25–34

  • Ricca F, Tonella P (2002) Testing processes of web applications. Ann Softw Eng 14(1–4):93–114

    Article  MATH  Google Scholar 

  • Sampath S, Mihaylov V, Souter A, Pollock L (2004) A scalable approach to user-session based testing of web applications through concept analysis. In: 19th IEEE international conference on automated software engineering, pp 132–141

  • Sampath S, Sprenkle S, Gibson E, Pollock L (2006) Web application testing with customized test requirements—an experimental comparison study. In: 17th international symposium on software reliability engineering (ISSRE’06). IEEE Computer Society Press, pp 266–278

  • Sampath S, Sprenkle S, Gibson E, Pollock L, Greenwald AS (2007) Applying concept analysis to user-session-based testing of web applications. IEEE Trans Softw Eng 33(10):643–658

    Article  Google Scholar 

  • Sprenkle S, Gibson E, Sampath S, Pollock L (2005a) Automated replay and failure detection for web applications. In: Proceedings of the 20th international conference of automated software engineering. ACM, Long Beach, pp 253–262

    Google Scholar 

  • Sprenkle S, Sampath S, Gibson E, Souter A, Pollock L (2005b) An empirical comparison of test suite reduction techniques for user-session-based testing of web applications. In: 21st IEEE international conference on software maintenance (ICSM), pp 587–596

  • Sprenkle S, Pollock L, Esquivel H, Hazelwood B, Ecott S (2007) Automated oracle comparators for testing web applications. In: Proceedings of the 18th international symposium on software reliability engineering. IEEE Computer Society, Trollhatten, pp 253–262

    Google Scholar 

  • Sprenkle S, Cobb C, Pollock L (2012) Leveraging user-privilege classification to customize usage-based statistical models of web applications. In: 5th international conference on software testing, verification and validation (ICST). IEEE, Montreal

    Google Scholar 

  • Tappenden A, Beatty P, Miller J, Geras A, Smith M (2005) Agile security testing of web-based systems via HTTPUnit. In: Proceedings of the Agile development conference (ADC ’05). IEEE Computer Society, Denver, pp 29–38

    Chapter  Google Scholar 

  • van Kestern A (2008) The XMLHttpRequest object. World Wide Web Consortium (W3C). Online: http://www.w3.org/TR/XMLHttpRequest/, last access July 2009

  • Wang WL, Tang MH (2003) User-oriented reliability modeling for a web system. In: Proceedings of the 14th international symposium on software reliability engineering. IEEE Computer Society, Denver, pp 293–304

    Google Scholar 

  • Wu Y, Offutt J (2002) Modeling and testing web-based applications. Technical report ISE-TR-02-08, Department of Information and Software Engineering, George Mason University, Fairfax, VA. http://www.cs.gmu.edu/~tr_admin/2002.html

  • Xu W, Bhatkar S, Sekar R (2005) A unified approach for preventing attacks exploiting a range of software vulnerabilities. Technical Report SECLAB-05-05, Department of Computer Science, Stony Brook University. http://seclab.cs.sunysb.edu/seclab1/pubs/papers/seclab-05-05.pdf

Download references

Author information

Authors and Affiliations

  1. Software Engineering, George Mason University, Fairfax, VA, USA

    Jeff Offutt, Vasileios Papadimitriou & Upsorn Praphamontripong

Authors
  1. Jeff Offutt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vasileios Papadimitriou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Upsorn Praphamontripong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeff Offutt.

Additional information

Editors: J. C. Maldonado

Rights and permissions

Reprints and permissions

About this article

Cite this article

Offutt, J., Papadimitriou, V. & Praphamontripong, U. A case study on bypass testing of web applications. Empir Software Eng 19, 69–104 (2014). https://doi.org/10.1007/s10664-012-9216-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-012-9216-x

Keywords

Search

Navigation