Abstract
JavaScript has become one of the most widely used languages for Web development. Its dynamic and event-driven features make it challenging to ensure the correctness of Web applications written in JavaScript. A variety of dynamic analysis techniques have been proposed which are, however, limited in either coverage or scalability. In this paper, we propose a simple, yet effective, model-based automated testing approach to achieve a high code-coverage within the time budget via testing with longer event sequences. We implement our approach as an open-source tool LJS, and perform extensive experiments on 21 publicly available benchmarks. On average, LJS is able to achieve 86.5% line coverage in 10 minutes. Compared with JSDEP, a state-of-the-art breadth-first search based automated testing tool enriched with partial order reduction, the coverage of LJS is 11%–19% higher than that of JSDEP on real-world large Web applications. Our empirical findings support that proper longer test sequences can achieve a higher code coverage in JavaScript Web application testing.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Saxena P, Akhawe D, Hanna S, Mao F, McCamant S, Song D. A symbolic execution framework for JavaScript. In: Proceedings of IEEE Symposium on Security and Privacy. 2010, 513–528
Artzi S, Dolby J, Jensen S H, Møller A, Tip F. A framework for automated testing of JavaScript web applications. In: Proceedings of International Conference on Software Engineering. 2011, 571–580
Mesbah A, Van Deursen A, Lenselink S. Crawling ajaxbased web applications through dynamic analysis of user interface state changes. ACM Transactions on the Web, 2012, 6(1): 1–30
Sen K, Kalasapur S, Brutch T G, Gibbs S. Jalangi: a selective record-replay and dynamic analysis framework for JavaScript. In: Proceedings of Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 2013, 488–498
Mirshokraie S, Mesbah A, Pattabiraman K. Efficient JavaScript mutation testing. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2013, 74–83
Li G, Andreasen E, Ghosh I. SymJS: automatic symbolic testing of JavaScript Web applications. In: Proceedings of ACM SIGSOFT International Symposium on Foundations of Software Engineering. 2014, 449–459
Pradel M, Schuh P, Necula G, Sen K. EventBreak: analyzing the responsiveness of user interfaces through performance-guided test generation. In: Proceedings of ACM International Conference on Object Oriented Programming Systems Languages & Applications. 2014, 33–47
Mirshokraie S, Mesbah A, Pattabiraman K. Guided mutation testing for JavaScript Web Applications. IEEE Transactions on Software Engineering, 2015, 41(5): 429–444
Sen K, Necula G C, Gong L, Choi W. MultiSE: multipath symbolic execution using value summaries. In: Proceedings of Joint Meeting on Foundations of Software Engineering. 2015, 842–853
Andreasen E, Gong L, Møller A, Pradel M, Selakovic M, Sen K, Staicu C. A survey of dynamic analysis and test generation for JavaScript. ACM Computing Surveys, 2017, 50(5): 66:1–66:36
Sung C, Kusano M, Sinha N, Wang C. Static DOM event dependency analysis for testing Web applications. In: Proceedings of ACM SIGSOFT International Symposium on Foundations of Software Engineering. 2016, 447–459
Arcuri A. Longer is better: On the role of test sequence length in software testing. In: Proofceedings of International Conference on Software Testing, Verification and Validation. 2010, 469–478
Andrews J H, Groce A, Weston M, Xu R G. Random test run length and effectiveness. In: Proofceedings of IEEE/ACM International Conference on Automated Software Engineering. 2008, 19–28
Fraser G, Gargantini A. Experiments on the test case length in specification based test case generation. In: Proceedings of International Workshop on Automation of Software Test. 2009, 18–26
Carino S, Andrews J H. Evaluating the effect of test case length on GUI test suite performance. In: Proceedings of IEEE/ACM International Workshop on Automation of Software Test. 2015, 13–17
Li Y F, Das P K, Dowe D L. Two decades of Web application testing — A survey of recent advances. Information Systems, 2014, 43: 20–54
Cheng L, Yang Z, Wang C. Systematic reduction of GUI test sequences. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering. 2017, 849–860
Godefroid P, van Leeuwen J, Hartmanis J, Goos G, Wolper P. Partial-Order Methods for the Verification of Concurrent Systems — An Approach to the StateExplosion Problem. Heidelberg: Springer, 1996
Arcuri A, Briand L C. A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability, 2014, 24(3): 219–250
Dias-Neto A C, Travassos G H. A picture from the model-based testing area: Concepts, techniques, and challenges. Advances in Computers, 2010, 80: 45–120
Utting M, Pretschner A, Legeard B. A taxonomy of model-based testing approaches. Software Testing, Verification & Reliability, 2012, 22(5): 297–312
Li W, Le Gall F, Spaseski N. A survey on model-based testing tools for test case generation. In: Proceedings of International Conference on Tools and Methods of Program Analysis. 2018, 77–89
Jensen C S, Prasad M R, Møller A. Automated testing with targeted event sequence generation. In: Proceedings of International Symposium on Software Testing and Analysis. 2013, 67–77
Takala T, Katara M, Harty J. Experiences of systemlevel model-based GUI testing of an android application. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2011, 377–386
Rau A, Hotzkow J, Zeller A. Efficient GUI test generation by learning from tests of other apps. In: Proceedings of International Conference on Software Engineering: Companion Proceedings. 2018, 370–371
Andrews A A, Offutt J, Alexander R T. Testing Web applications by modeling with fsms. Software and System Modeling, 2005, 4(3): 326–345
Ricca F, Tonella P. Analysis and testing of Web applications. In: Proceedings of International Conference on Software Engineering. 2001, 25–34
Dallmeier V, Burger M, Orth T, Zeller A. Webmate: a tool for testing Web 2.0 applications. In: Proceedings of Workshop on JavaScript Tools. 2012, 11–15
Alshahwan N, Harman M. Automated Web application testing using search based software engineering. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering. 2011, 3–12
Nguyen C, Yoshida H, Prasad M R, Ghosh I, Sen K. Generating succinct test cases using don’t care analysis. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2015, 1–10
Rau A, Hotzkow J, Zeller A. Transferring tests across Web applications. In: Proceedings of International Conference on Web Engineering. 2018, 50–64
Acknowledgements
The authors would like to thank Dr. Ting Su for his valuable comments and suggestions. P. Gao, Y. Xu and F. Song were partially supported by the National Natural Science Foundation of China (NSFC) (Grant Nos. 62072309, 61532019, 61761136011). T. Chen is partially supported by the National Natural Science Foundation of China (Grant No. 61872340), Guangdong Science and Technology Department (2018B01010 7004) and Natural Science Foundation of Guangdong Province (2019A 1515011689).
Author information
Authors and Affiliations
Corresponding author
Additional information
Pengfei Gao received the BS degree in Computer Science from China University of Mining and Technology, China in 2017. He is currently a PhD student in ShanghaiTech University, China supervised by Prof. Fu Song. His research interests include program analysis and software security.
Yongjie Xu received the BS degree in Computer Science and Technology from Shanghaitech University, China in 2019. He is currently a MS student in ShanghaiTech University, China supervised by Prof. Fu Song. His research interests include SAT solving and software security.
Fu Song received the BS degree from Ningbo University, China in 2006, the MS degree from East China Normal University, China in 2009, and the PhD degree in computer science from University Paris-Diderot, France in 2013.From 2013 to 2016, he was a Lecturer and Associate Research Professor at East China Normal University. China. Since August 2016, he is an Assistant Professor with ShanghaiTech University, China. His research interests include formal methods and computer security, especially about automata, logic, model checking, and program analysis. Dr. Song was a recipient of EASST best paper award at ETAPS 2012.
Taolue Chen received the BS and MS degrees from the Nanjing University, China both in Computer Science. He was a junior researcher at the Centrum Wiskunde & Informatica (CWI) and acquired the PhD degree from the Vrije Universiteit Amsterdam, The Netherlands. He is currently a lecturer at the Department of Computer Science and Information Systems, Birkbeck, University of London, UK. His research interests include formal verification and synthesis, program analysis, logic in computer science, and software engineering.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Gao, P., Xu, Y., Song, F. et al. Model-based automated testing of JavaScript Web applications via longer test sequences. Front. Comput. Sci. 16, 163204 (2022). https://doi.org/10.1007/s11704-020-0356-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11704-020-0356-7