Abstract
Automated GUI tests typically comprise of several test steps that are executed on the GUI before reaching a point of assertion. Comparing a longer and complex execution of a GUI test to its test instructions for debugging is a laborious task: re-establish the test environment, slow down test execution for human perception, and locate the currently executed test step. Video documentation of GUI tests for debugging purposes is already present in several industry tools. However, it is not optimized for effective documentation of on-screen actions nor synched with the executed test instructions. We present a video-based documentation of automated GUI tests that links the executed test case instruction to the on-screen response of the application under test. Screen recording is optimized for speed and memory consumption while all relevant details are captured. Additional browsing capabilities for easier debugging are introduced. Concepts of aspect-oriented programming are adapted for tracing of pre-compiled test case scripts. Our concepts are evaluated by a working implementation, a series of performance measurements during a technical experiment, and industrial experience from 370 real-world test cases carried out in a large software company. The limits of our implementation regarding video capturing and code tracing are explored with a specialized test frame.
Similar content being viewed by others
Notes
In our semi-structured interviews (Sect. 5.4), GUI testers at Capgemini reported the increase in OS-version of Windows would regularly bring changes of IDs for standard GUI elements of system dialogs. This in turn renders several test cases wrong and results in the repeating task of correcting already passed test cases in order to maintain regression testing.
References
Beck, K. (2002). Test driven development. By example. Amsterdam: Addison-Wesley Longman.
Bodkin, R., Colyer, A., & Hugunin, J. (2003). Applying aop for middleware platform independence. Practitioner Reports, AOSD, 2003.
Böllert, K. (1999). On weaving aspects. In Proceedings of the ECOOP’99 workshop on aspect-oriented programming, 1999, Lisbon, Portugal.
DebugMode, Wink [Online]. Available: http://www.debugmode.com/wink. Accessed 2011.
Holzmann, H. (2011). Videounterstützte Ablaufverfolgung von Tests für Anwendungen mit grafischer Benutzeroberfläche. Bachelor Thesis, Leibniz Universität Hannover.
Kiczales, G., Lamping, J., Mendhekar, A., Maeda, C., Lopes, C., Loingtier, J., et al. (1997). Aspect-oriented programming, ECOOP’97—Object-oriented programming, pp. 220–242.
Laddad, R. (2003). Aspect-oriented programming will improve quality. IEEE Software, 20(6), 90–91.
Le Gall, D. (1991). Mpeg: A video compression standard for multimedia applications. Communications of the ACM, 34(4), 46–58.
Memon, A. (2002). GUI Testing: Pitfalls and Process. IEEE Computer, 35(8), 87–88.
Microsoft, Expression Encoder Pro [Online]. Available: http://www.microsoft.com/expression/products/encoderpro_overview.aspx. Accessed 2011.
Microsoft, IntelliTrace [Online]. Available: http://msdn.microsoft.com/en-us/magazine/ee336126.aspx. Accessed 2011.
Microsoft, Team Foundation Server [Online]. Available: http://www.microsoft.com/visualstudio/en-us/products/2010-editions/team-foundation-server/overview. Accessed 2011.
Myers, G. J. (1979). The art of software testing (1st ed.). New York: Wiley.
Patel K., Smith B., & Rowe, L. (1993). Performance of a software MPEG video decoder. In Proceedings of the first ACM international conference on Multimedia (pp. 75–82).
Ranorex [Online]. Available: http://www.ranorex.com. Accessed 2011.
RenderSoft, Camstudio [Online]. Available: http://camstudio.org. Accessed 2011.
Schneider, K. (2007). Abenteuer Softwarequalität: Grundlagen und Verfahren für Qualitätssicherung und Qualitätsmanagement. Heidelberg: Dpunkt.
Sikora, T. (1997). MPEG digital video-coding standards. IEEE Signal Processing Magazine, 14(5), 82–100.
TechSmith, Camtasia [Online]. Available: http://www.techsmith.com/camtasia. Accessed 2011.
Tightvnc [Online]. Available: http://www.tightvnc.com. Accessed 2011.
Ultravnc [Online]. Available: http://www.uvnc.com. Accessed 2011.
Ultravnc Screen Recorder [Online]. Available: http://www.uvnc.com/screenrecorder. Accessed 2011.
Wiegand, T., Sullivan, G., Bjontegaard, G., & Luthra, A. (2003). Overview of the H.264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology, 13(7), 560–576.
Zimmermann, T., Premraj, R., Bettenburg, N., Just, S., Schroter, A., & Weiss, C. (2010). What makes a good bug report? IEEE Transactions on Software Engineering, 35(5), 618–643.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pham, R., Holzmann, H., Schneider, K. et al. Tailoring video recording to support efficient GUI testing and debugging. Software Qual J 22, 273–292 (2014). https://doi.org/10.1007/s11219-013-9206-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11219-013-9206-2