Skip to main content
Log in

Model-based automated testing of JavaScript Web applications via longer test sequences

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

JavaScript has become one of the most widely used languages for Web development. Its dynamic and event-driven features make it challenging to ensure the correctness of Web applications written in JavaScript. A variety of dynamic analysis techniques have been proposed which are, however, limited in either coverage or scalability. In this paper, we propose a simple, yet effective, model-based automated testing approach to achieve a high code-coverage within the time budget via testing with longer event sequences. We implement our approach as an open-source tool LJS, and perform extensive experiments on 21 publicly available benchmarks. On average, LJS is able to achieve 86.5% line coverage in 10 minutes. Compared with JSDEP, a state-of-the-art breadth-first search based automated testing tool enriched with partial order reduction, the coverage of LJS is 11%–19% higher than that of JSDEP on real-world large Web applications. Our empirical findings support that proper longer test sequences can achieve a higher code coverage in JavaScript Web application testing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Saxena P, Akhawe D, Hanna S, Mao F, McCamant S, Song D. A symbolic execution framework for JavaScript. In: Proceedings of IEEE Symposium on Security and Privacy. 2010, 513–528

  2. Artzi S, Dolby J, Jensen S H, Møller A, Tip F. A framework for automated testing of JavaScript web applications. In: Proceedings of International Conference on Software Engineering. 2011, 571–580

  3. Mesbah A, Van Deursen A, Lenselink S. Crawling ajaxbased web applications through dynamic analysis of user interface state changes. ACM Transactions on the Web, 2012, 6(1): 1–30

    Article  Google Scholar 

  4. Sen K, Kalasapur S, Brutch T G, Gibbs S. Jalangi: a selective record-replay and dynamic analysis framework for JavaScript. In: Proceedings of Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 2013, 488–498

  5. Mirshokraie S, Mesbah A, Pattabiraman K. Efficient JavaScript mutation testing. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2013, 74–83

  6. Li G, Andreasen E, Ghosh I. SymJS: automatic symbolic testing of JavaScript Web applications. In: Proceedings of ACM SIGSOFT International Symposium on Foundations of Software Engineering. 2014, 449–459

  7. Pradel M, Schuh P, Necula G, Sen K. EventBreak: analyzing the responsiveness of user interfaces through performance-guided test generation. In: Proceedings of ACM International Conference on Object Oriented Programming Systems Languages & Applications. 2014, 33–47

  8. Mirshokraie S, Mesbah A, Pattabiraman K. Guided mutation testing for JavaScript Web Applications. IEEE Transactions on Software Engineering, 2015, 41(5): 429–444

    Article  Google Scholar 

  9. Sen K, Necula G C, Gong L, Choi W. MultiSE: multipath symbolic execution using value summaries. In: Proceedings of Joint Meeting on Foundations of Software Engineering. 2015, 842–853

  10. Andreasen E, Gong L, Møller A, Pradel M, Selakovic M, Sen K, Staicu C. A survey of dynamic analysis and test generation for JavaScript. ACM Computing Surveys, 2017, 50(5): 66:1–66:36

    Google Scholar 

  11. Sung C, Kusano M, Sinha N, Wang C. Static DOM event dependency analysis for testing Web applications. In: Proceedings of ACM SIGSOFT International Symposium on Foundations of Software Engineering. 2016, 447–459

  12. Arcuri A. Longer is better: On the role of test sequence length in software testing. In: Proofceedings of International Conference on Software Testing, Verification and Validation. 2010, 469–478

  13. Andrews J H, Groce A, Weston M, Xu R G. Random test run length and effectiveness. In: Proofceedings of IEEE/ACM International Conference on Automated Software Engineering. 2008, 19–28

  14. Fraser G, Gargantini A. Experiments on the test case length in specification based test case generation. In: Proceedings of International Workshop on Automation of Software Test. 2009, 18–26

  15. Carino S, Andrews J H. Evaluating the effect of test case length on GUI test suite performance. In: Proceedings of IEEE/ACM International Workshop on Automation of Software Test. 2015, 13–17

  16. Li Y F, Das P K, Dowe D L. Two decades of Web application testing — A survey of recent advances. Information Systems, 2014, 43: 20–54

    Article  Google Scholar 

  17. Cheng L, Yang Z, Wang C. Systematic reduction of GUI test sequences. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering. 2017, 849–860

  18. Godefroid P, van Leeuwen J, Hartmanis J, Goos G, Wolper P. Partial-Order Methods for the Verification of Concurrent Systems — An Approach to the StateExplosion Problem. Heidelberg: Springer, 1996

    Google Scholar 

  19. Arcuri A, Briand L C. A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability, 2014, 24(3): 219–250

    Article  Google Scholar 

  20. Dias-Neto A C, Travassos G H. A picture from the model-based testing area: Concepts, techniques, and challenges. Advances in Computers, 2010, 80: 45–120

    Article  Google Scholar 

  21. Utting M, Pretschner A, Legeard B. A taxonomy of model-based testing approaches. Software Testing, Verification & Reliability, 2012, 22(5): 297–312

    Article  Google Scholar 

  22. Li W, Le Gall F, Spaseski N. A survey on model-based testing tools for test case generation. In: Proceedings of International Conference on Tools and Methods of Program Analysis. 2018, 77–89

  23. Jensen C S, Prasad M R, Møller A. Automated testing with targeted event sequence generation. In: Proceedings of International Symposium on Software Testing and Analysis. 2013, 67–77

  24. Takala T, Katara M, Harty J. Experiences of systemlevel model-based GUI testing of an android application. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2011, 377–386

  25. Rau A, Hotzkow J, Zeller A. Efficient GUI test generation by learning from tests of other apps. In: Proceedings of International Conference on Software Engineering: Companion Proceedings. 2018, 370–371

  26. Andrews A A, Offutt J, Alexander R T. Testing Web applications by modeling with fsms. Software and System Modeling, 2005, 4(3): 326–345

    Article  Google Scholar 

  27. Ricca F, Tonella P. Analysis and testing of Web applications. In: Proceedings of International Conference on Software Engineering. 2001, 25–34

  28. Dallmeier V, Burger M, Orth T, Zeller A. Webmate: a tool for testing Web 2.0 applications. In: Proceedings of Workshop on JavaScript Tools. 2012, 11–15

  29. Alshahwan N, Harman M. Automated Web application testing using search based software engineering. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering. 2011, 3–12

  30. Nguyen C, Yoshida H, Prasad M R, Ghosh I, Sen K. Generating succinct test cases using don’t care analysis. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2015, 1–10

  31. Rau A, Hotzkow J, Zeller A. Transferring tests across Web applications. In: Proceedings of International Conference on Web Engineering. 2018, 50–64

Download references

Acknowledgements

The authors would like to thank Dr. Ting Su for his valuable comments and suggestions. P. Gao, Y. Xu and F. Song were partially supported by the National Natural Science Foundation of China (NSFC) (Grant Nos. 62072309, 61532019, 61761136011). T. Chen is partially supported by the National Natural Science Foundation of China (Grant No. 61872340), Guangdong Science and Technology Department (2018B01010 7004) and Natural Science Foundation of Guangdong Province (2019A 1515011689).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Fu Song.

Additional information

Pengfei Gao received the BS degree in Computer Science from China University of Mining and Technology, China in 2017. He is currently a PhD student in ShanghaiTech University, China supervised by Prof. Fu Song. His research interests include program analysis and software security.

Yongjie Xu received the BS degree in Computer Science and Technology from Shanghaitech University, China in 2019. He is currently a MS student in ShanghaiTech University, China supervised by Prof. Fu Song. His research interests include SAT solving and software security.

Fu Song received the BS degree from Ningbo University, China in 2006, the MS degree from East China Normal University, China in 2009, and the PhD degree in computer science from University Paris-Diderot, France in 2013.From 2013 to 2016, he was a Lecturer and Associate Research Professor at East China Normal University. China. Since August 2016, he is an Assistant Professor with ShanghaiTech University, China. His research interests include formal methods and computer security, especially about automata, logic, model checking, and program analysis. Dr. Song was a recipient of EASST best paper award at ETAPS 2012.

Taolue Chen received the BS and MS degrees from the Nanjing University, China both in Computer Science. He was a junior researcher at the Centrum Wiskunde & Informatica (CWI) and acquired the PhD degree from the Vrije Universiteit Amsterdam, The Netherlands. He is currently a lecturer at the Department of Computer Science and Information Systems, Birkbeck, University of London, UK. His research interests include formal verification and synthesis, program analysis, logic in computer science, and software engineering.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, P., Xu, Y., Song, F. et al. Model-based automated testing of JavaScript Web applications via longer test sequences. Front. Comput. Sci. 16, 163204 (2022). https://doi.org/10.1007/s11704-020-0356-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-020-0356-7

Keywords