Integrating Traversal and Visual-based Techniques to Initiate UI Exploration Scripting
Pages 157 - 161
Abstract
Initiating UI Test Automation project usually requires the testers who have related business knowledge, testing skill, and technical skill. Helping the testers start the project quickly and able to focus on other tasks in charge, this paper presents an approach to generate the initial UI test scripts by combining three main concepts of GUI automation including Scripting, Visual, and Traversal. The designed framework uses the data from the Accessibility API to extract target graphical elements on application under test (AUT). The traversal technique is applied for generating the sequences of interactions on all interactable objects based on Python GUI automated scripting (SikuliX) format. In case the API is not available on some OS/Applications, the computer vision technology will then be deployed on the Open Source Computer Vision Library (OpenCV) to extract the GUI elements. The implementation of the proposed framework is demonstrated via a couple of open-source applications.
References
[1]
Al´egroth, E., Feldt, R., and Kolstr¨om, P. 2016. Maintenance of automated test suites in industry: An empirical study on Visual GUI Testing. Information and Software Technology. 73, 66--80.
[2]
Rafi, D. M., Reddy, K., Moses, K., and Petersen, K. 2012. Benefits and Limitations of Automated Software Testing: Systematic Literature Review and Practitioner Survey. Automation of Software Test (AST), 7th International Workshop.
[3]
Al´egroth, E., and Feldt, R. 2017. On the long-term use of visual GUI testing in industrial practice: a case study. Empirical Software Engineering. 22, 2937--2971.
[4]
Grechanik, M., Xie, Q., and Fu, C. 2009. Maintaining and evolving GUI-directed test scripts. International Conference on Software Engineering. 408--418.
[5]
Al´egroth, E., and Feldt, R. 2014. Industrial application of visual GUI testing: Lessons learned. Continuous software engineering, 127--140.
[6]
Al´egroth, E., Gao, Z., Oliveira, R., and Memon A. 2015. Conceptualization and evaluation of component-based testing unified with visual GUI testing: An empirical study. 2015 IEEE 8th International Conference on Software Testing, Verification and Validation ICST, 1--10.
[7]
Dobslaw, F., Feldt, R., Michaelsson, D., Haar, P., de Oliveira Neto, F. G., Torkar, R. 2019. Estimating Return on Investment for GUI Test Automation Tools. arXiv:1907.03475 [cs.SE].
[8]
Tsung-Hsiang Chang, Tom Yeh, and Robert Miller, C. 2010. GUI Testing Using Computer Vision. CHI 2010: Pixel and Perception, (Atlanta, GA, USA, April 10--15, 2010), 1535--1544.
[9]
Vos, T. E.J., Molina, U. R., and Prasetya, W. 2016. Automated Testing at the User Interface level. ICTopen2016, Dutch ICT-research.
[10]
A. M. Memon. 2002. Gui testing: Pitfalls and process. Computer. 35, 87--88.
[11]
B. N. Nguyen, B. Robbins, I. Banerjee, and A. M. Memon, 2014. Guitar:an innovative tool for automated testing of gui-driven software. Automated Software Engineering. 21, 65--105.
[12]
Capgemini, Micro Focus, Sogeti. 2019. World Quality Report 2018--2019 TENTH EDITION. https://www.sogeti.com/globalassets/global/wqr-201819/wqr-2018-19_secured.pdf, Accessed: 24 Sep 2019
[13]
Nistér, D., and Stewénius, H. 2008. Linear time maximally stable extremal regions. Computer Vision-ECCV 2008, 183--196.
Index Terms
- Integrating Traversal and Visual-based Techniques to Initiate UI Exploration Scripting
Comments
Information & Contributors
Information
Published In
January 2020
303 pages
ISBN:9781450376778
DOI:10.1145/3383845
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- Univ. of Manchester: University of Manchester
- The Hong Kong Polytechnic: The Hong Kong Polytechnic University
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 April 2020
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCMB 2020
ICCMB 2020: 2020 The 3rd International Conference on Computers in Management and Business
January 31 - February 2, 2020
Tokyo, Japan
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 96Total Downloads
- Downloads (Last 12 months)7
- Downloads (Last 6 weeks)1
Reflects downloads up to 16 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in