On the Industrial Applicability of TextTest: An Empirical Case Study | IEEE Conference Publication | IEEE Xplore

On the Industrial Applicability of TextTest: An Empirical Case Study


Abstract:

Software systems are becoming more complex, not least in their Graphical User Interfaces (GUIs), which presents challenges for existing testing practices. Pressure to red...Show More

Abstract:

Software systems are becoming more complex, not least in their Graphical User Interfaces (GUIs), which presents challenges for existing testing practices. Pressure to reduce time to market leaves less time for manual testing and increases the importance of test automation. Previous research has identified several generations of automated GUI-based test approaches with different cost-benefit tradeoffs. Whilst test automation provides fast quality feedback it can be associated with high costs and inability to identify defects not explicitly anticipated by the test designer. TextTest is a capture-replay tool for GUI-based testing with a novel approach that overcomes several of the challenges experienced with previous approaches. Firstly the tool supports Approval Testing, an approach where ASCII-art representations of the GUI's visual state are used to verify correct application behavior at the system level. Secondly it records and replays test scripts in a user defined domain specific language (DSL) that is readable by all stakeholders. In this paper we present a three phase industrial case study that aims to identify TextTest's applicability in industrial practice. The paper reports that the tool is associated with (1) low script development costs due to recording functionality, (2) low maintenance costs, on average 7 minutes per test case, (3) better defect finding ability than manual system testing, (4) high test case execution performance (In this case 500 test cases in 20 minutes), (5) high script readability due to DSL defined scripts, and (6) test suites that are robust to change (In this case 93 percent per iteration). However, the tool requires a higher degree of technical skill for customization work, test maintainers need skills in designing regular expressions and the tool's applicability is currently restricted to Java and Python based applications.
Date of Conference: 13-17 April 2015
Date Added to IEEE Xplore: 07 May 2015
Electronic ISBN:978-1-4799-7125-1
Print ISSN: 2159-4848
Conference Location: Graz, Austria

References

References is not available for this document.