skip to main content
10.1145/3368089.3417069acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
research-article

WebRR: self-replay enhanced robust record/replay for web application testing

Published: 08 November 2020 Publication History

Abstract

Record-and-replay tools are important for quality assurance of Web applications by capturing user case scenarios and executing them automatically when needed. However, the tests generated by existing techniques are brittle, and often lead to test breakages as the dynamic behavior and frequent updates of modern Web applications. In this paper, we propose WebRR, a self-replay enhanced robust record-and-replay technique for Web applications testing. The novelty of WebRR is that, it introduces a new self-replay mechanism in the recording phase, which checks the captured event from the record module online, and generates multiple locators (including DOM locators, visual locator and proximity locators) automatically, to improve the robustness of generated test cases. During the replay, it combines multiple locators and new local workflow repair technique to repair test breakages, and can improve the resilience of generated tests to frequent updates of the applications. We applied our approach to 3 enterprise Web applications, which are deployed in a large power grid company of China. The experimental results show that WebRR is effective, and substantially improve the robustness of end-to-end web tests that are generated using record-and-replay technique.

Supplementary Material

Auxiliary Teaser Video (fse20ind-p222-p-teaser.mp4)
This is a presentation video of my talk at ESEC/FSE 2020 on our paper accepted in the industry track. In this paper we propose WebRR, a self-replay enhanced robust record-and-replay technique for Web applications testing. The novelty of WebRR is that, it introduces a new self-replay mechanism in the recording phase, which checks the captured event from the record module online, and generates multiple locators automatically, to improve the robustness of generated test cases. During the replay, it combines multiple locators and new local workflow repair technique to repair test breakages, and can improve the resilience of generated tests to frequent updates of the applications. We applied our approach to 3 enterprise Web applications, which are deployed in a large power grid company of China. The experimental results show that WebRR is effective, and substantially improve the robustness of end-to-end web tests that are generated using record-and-replay technique.
Auxiliary Presentation Video (fse20ind-p222-p-video.mp4)
This is a presentation video of my talk at ESEC/FSE 2020 on our paper accepted in the industry track. In this paper we propose WebRR, a self-replay enhanced robust record-and-replay technique for Web applications testing. The novelty of WebRR is that, it introduces a new self-replay mechanism in the recording phase, which checks the captured event from the record module online, and generates multiple locators automatically, to improve the robustness of generated test cases. During the replay, it combines multiple locators and new local workflow repair technique to repair test breakages, and can improve the resilience of generated tests to frequent updates of the applications. We applied our approach to 3 enterprise Web applications, which are deployed in a large power grid company of China. The experimental results show that WebRR is effective, and substantially improve the robustness of end-to-end web tests that are generated using record-and-replay technique.

References

[1]
Silviu Andrica and George Candea. 2011. WaRR: A Tool for High-Fidelity Web Application Record and Replay. 2011 IEEE/IFIP 41th International Conference On Dependable Systems And Networks (DSN) ( 2011 ), 403-410. http://infoscience.epfl. ch/record/166078
[2]
Brian Burg, Richard Bailey, Andrew J. Ko, and Michael D. Ernst. 2013. Interactive Record/Replay for Web Application Debugging. In Proceedings of 26th Annual ACM Symposium on User Interface Software and Technology (St. Andrews, Scotland, United Kingdom) (UIST'13). Association for Computing Machinery, New York, NY, USA, 473-484. https://doi.org/10.1145/2501988.2502050
[3]
Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, and Alessandro Orso. 2011. Water: Web application test repair. In Proceedings of the First International Workshop on End-to-End Test Script Engineering. 24-29.
[4]
Crawljax. 2020. http://crawljax.com/.
[5]
Electron. 2020. https://www.electronjs.org/.
[6]
Robot Framework. 2020. https://robotframework.org/.
[7]
Jiaqi Guo, Shuyue Li, Jian-Guang Lou, Zijiang Yang, and Ting Liu. 2019. Sara: Self-Replay Augmented Record and Replay for Android in Industrial Cases. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (Beijing, China) ( ISSTA 2019 ). Association for Computing Machinery, New York, NY, USA, 90-100. https://doi.org/10.1145/3293882.3330557
[8]
Mouna Hammoudi, Gregg Rothermel, and Andrea Stocco. 2016. Waterfall: An incremental approach for repairing record-replay tests of web applications. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. 751-762.
[9]
M. Hammoudi, G. Rothermel, and P. Tonella. 2016. Why do Record/Replay Tests of Web Applications Break?. In 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST). 180-190.
[10]
Selenium IDE. 2020. https://selenium.dev/selenium-ide/.
[11]
Digital Automation Intelligence. 2020. https://www.eggplantsoftware.com/digitalautomation-intelligence.
[12]
H. Kirinuki, H. Tanno, and K. Natsukawa. 2019. COLOR: Correct Locator Recommender for Broken Test Scripts using Various Clues in Web Application. In 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER). 310-320.
[13]
Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2015. MetaHeuristic Generation of Robust XPath Locators for Web Testing. In Proceedings of the Eighth International Workshop on Search-Based Software Testing (Florence, Italy) (SBST'15). IEEE Press, 36-39.
[14]
Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2015. Using multi-locators to increase the robustness of web test cases. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST). IEEE, 1-10.
[15]
Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2016. ROBULA+: An algorithm for generating robust XPath locators for web testing. Journal of Software: Evolution and Process 28, 3 ( 2016 ), 177-204.
[16]
Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2018. Pesto: Automated migration of DOM-based Web tests towards the visual approach. Software Testing, Verification And Reliability 28, 4 ( 2018 ), 1665.
[17]
Gilly Leshed, Eben M. Haber, Tara Matthews, and Tessa Lau. 2008. CoScripter: Automating & Sharing How-to Knowledge in the Enterprise. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI'08). Association for Computing Machinery, New York, NY, USA, 1719-1728. https://doi.org/10.1145/1357054.1357323
[18]
Ali Mesbah, Arie van Deursen, and Stefan Lenselink. 2012. Crawling Ajax-based web applications through dynamic analysis of user interface state changes. ACM Transactions on the Web (TWEB) 6, 1 ( 2012 ). https://doi.org/10.1145/2109205. 2109208
[19]
James Mickens, Jeremy Elson, and Jon Howell. 2010. Mugshot: Deterministic Capture and Replay for Javascript Applications. In Proceedings of the 7th USENIX Conference on Networked Systems Design and Implementation (San Jose, California) ( NSDI'10). USENIX Association, USA, 11.
[20]
DOM Level 2 Model. 2020. http://www.w3.org/TR/2000/REC-DOM-Level-2-Events-20001113/events.html.
[21]
Mutation Observer. 2020. http://www.w3.org/TR/dom/mutation-observers.
[22]
HP QuickTest Professional. 2020. https://hp-quicktestprofessional.updatestar.com/.
[23]
Koushik Sen, Swaroop Kalasapur, Tasneem Brutch, and Simon Gibbs. 2013. Jalangi: A Selective Record-Replay and Dynamic Analysis Framework for JavaScript. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (Saint Petersburg, Russia) (ESEC/FSE 2013 ). Association for Computing Machinery, New York, NY, USA, 488-498. https://doi.org/10.1145/2491411. 2491447
[24]
Andrea Stocco, Rahulkrishna Yandrapally, and Ali Mesbah. 2018. Visual web test repair. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 503-514.
[25]
Suresh Thummalapenta, Nimit Singhania, Pranavadatta Devaki, Saurabh Sinha, Satish Chandra, Achin K. Das, and Srinivas Mangipudi. 2012. Eficiently Scripting Change-Resilient Tests. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (Cary, North Carolina) (FSE 2012 ). Association for Computing Machinery, New York, NY, USA, 2. https: //doi.org/10.1145/2393596.2393643
[26]
Suresh Thummalapenta, Saurabh Sinha, Nimit Singhania, and Satish Chandra. 2012. Automating Test Automation. In Proceedings of the 34th International Conference on Software Engineering (Zurich, Switzerland) ( ICSE 2012). IEEE Press, 881-891.
[27]
UIRecorder. 2020. https://uirecorder.com/.
[28]
Vue. 2020. https://cn.vuejs.org/.
[29]
WebDriverIO. 2020. https://webdriver.io/.
[30]
Rahulkrishna Yandrapally, Suresh Thummalapenta, Saurabh Sinha, and Satish Chandra. 2014. Robust Test Automation Using Contextual Clues. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (San Jose, CA, USA) ( ISSTA 2014 ). Association for Computing Machinery, New York, NY, USA, 304-314. https://doi.org/10.1145/2610384.2610390
[31]
Tom Yeh, Tsung-Hsiang Chang, and Robert C. Miller. 2009. Sikuli: Using GUI Screenshots for Search and Automation. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (Victoria, BC, Canada) ( UIST'09). Association for Computing Machinery, New York, NY, USA, 183-192. https://doi.org/10.1145/1622176.1622213

Cited By

View all
  • (2024)Enhancing the Resiliency of Automated Web Tests with Natural LanguageProceedings of the 2024 4th International Conference on Artificial Intelligence, Automation and Algorithms10.1145/3700523.3700536(63-69)Online publication date: 27-Sep-2024
  • (2024)Towards a Robust Waiting Strategy for Web GUI Testing for an Industrial Software SystemProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695269(2065-2076)Online publication date: 27-Oct-2024
  • (2024)Wasm-R3: Record-Reduce-Replay for Realistic and Standalone WebAssembly BenchmarksProceedings of the ACM on Programming Languages10.1145/36897878:OOPSLA2(2156-2182)Online publication date: 8-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
November 2020
1703 pages
ISBN:9781450370431
DOI:10.1145/3368089
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 November 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Web application
  2. end-to-end test
  3. record-and-replay

Qualifiers

  • Research-article

Conference

ESEC/FSE '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)79
  • Downloads (Last 6 weeks)3
Reflects downloads up to 08 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Enhancing the Resiliency of Automated Web Tests with Natural LanguageProceedings of the 2024 4th International Conference on Artificial Intelligence, Automation and Algorithms10.1145/3700523.3700536(63-69)Online publication date: 27-Sep-2024
  • (2024)Towards a Robust Waiting Strategy for Web GUI Testing for an Industrial Software SystemProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695269(2065-2076)Online publication date: 27-Oct-2024
  • (2024)Wasm-R3: Record-Reduce-Replay for Realistic and Standalone WebAssembly BenchmarksProceedings of the ACM on Programming Languages10.1145/36897878:OOPSLA2(2156-2182)Online publication date: 8-Oct-2024
  • (2024)ScoutDroid: A Tool For Mobile Augmented Testing with Live FeedbackProceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation10.1145/3678869.3685688(34-37)Online publication date: 13-Sep-2024
  • (2024)On Effectiveness and Efficiency of Gamified Exploratory GUI TestingIEEE Transactions on Software Engineering10.1109/TSE.2023.334803650:2(322-337)Online publication date: Feb-2024
  • (2024)Tree-Based Synthesis of Web Test Sequences from Manual ActionsTheoretical Aspects of Software Engineering10.1007/978-3-031-64626-3_14(242-260)Online publication date: 14-Jul-2024
  • (2023)Semantic Test Repair for Web ApplicationsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616324(1190-1202)Online publication date: 30-Nov-2023
  • (2023)Towards Efficient Record and Replay: A Case Study in WeChatProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3613880(1681-1692)Online publication date: 30-Nov-2023
  • (2023)Assistive-Technology Aided Manual Accessibility Testing in Mobile Apps, Powered by Record-and-ReplayProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580679(1-20)Online publication date: 19-Apr-2023
  • (2023)RIDA: Cross-App Record and Replay for Android2023 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST57152.2023.00031(246-257)Online publication date: Apr-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media