Skip to main content
SpringerLink
Log in

Employing user profiles to test a new version of a GUI component in its context of use

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Advancement in reusable component technology has had a significant impact on the development of complex graphical user interfaces (GUIs), which are front-ends to most of today’s software. Software developers can, with very little effort, integrate components into their software’s GUI. Problems, however, arise when new versions of GUI components replace their predecessors in an implementation. Often, the inclusion of a new version of a component breaks some part of the software, i.e., tasks that end-users were able to perform before modifications were made can no longer be performed. Software developers (who also happen to be component users) are unable to perform adequate regression testing in this context because of several factors, including lack of source code, environmental side-effects on GUI rendering, event-driven nature of GUIs, and large number of possible permutations of events. This paper presents a new technique that end-users’ quality assurance (QA) teams can employ to test the new version of a component in its application context by using the existing version as a baseline. The technique combines lightweight event-level dynamic profiling to collect user profiles in a transparent manner, GUI reverse engineering to extract the structure of the component’s GUI, test case execution to replay the collected profiles on the new version, and GUI oracles that collect properties from the existing version. Empirical studies demonstrate the practicality, usefulness, and limitations of the technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abbot Java GUI Test Framework, 2003. http://abbot.sourceforge.net.

  • Bertolino, A., Polini, A. 2003. A framework for component deployment testing. In Proceedings of the 25th International Conference on Software Engineering, pp. 221–231, IEEE Computer Society.

  • Beydeda, S., Gruhn, V. 2001. An integrated testing technique for component-based software.

  • Capture-Replay Tool, 2003. http://soft.com/.

  • Dawson, S., Jahanian, F., Mitton, T., Tung, T.-L. 1996. Testing of fault-tolerant and real-time distributed systems via protocol fault injektion. In Symposium on Fault-Tolerant Computing, pp. 404–414.

  • Dillon, L.K., Ramakrishna, Y.S. 1996. Generating oracles from your favorite temporal logic specifications. In Proceedings of the Fourth ACM SIGSOFT Symposium on the Foundations of Software Engineering (New York), acknowledgementsol. 21 of ACM Software Engineering Notes, pp. 106–117, ACM Press.

  • Dillon, L.K., Yu, Q. 1994. Oracles for checking temporal properties of concurrent systems. In Proceedings of the ACM SIGSOFT ’94 Symposium on the Foundations of Software Engineering, pp. 140–153.

  • Finsterwalder, M. 2001. Automating acceptance tests for gui applications in an extreme programming environment. In Proceedings of the 2nd International Conference on eXtreme Programming and Flexible Processes in Software Engineering, pp. 114–117.

  • Hamlet, D., Mason, D., Woit, D. 2001. Theory of software reliability based on components. In Proceedings of the 23rd International Conference on Software Engineering, pp. 361–370, IEEE Computer Society.

  • Haraty, R.A., Mansour, N., Daou, B. 2001. Regression testing of database applications. In Proceedings of the 2001 ACM Symposium on Applied Computing, pp. 285–289, ACM Press.

  • Harrold, M.J. 2000. Testing: a roadmap. In Proceedings of the Conference on The Future Software Engineering, pp. 61–72, ACM Press.

  • Hicinbothom, J.H., Zachary, W.W. 1993. A tool for automatically generating transcripts of human-computer interaction. In Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting, vol. 2 of Special Sessions: Demonstrations, p. 1042.

  • Huang, Y.-W., Huang, S.-K., Lin, T.-P., Tsai, C.-H. 2003. Web application security assessment by fault injection and behavior monitoring. In Proceedings of the Twelfth International Conference on World Wide Web, pp. 148–159, ACM Press.

  • Jeon, T., Seung, H.W., Lee, S. 2002. Embedding built-in tests in hot spots of an object-oriented framework. SIGPLAN Not. 37(8):25–34.

    Article  Google Scholar 

  • JUnit, Testing Resources for Extreme Programming. http://junit.org/news/extension/gui/index.htm.

  • Korel, B. 1999. Black-box understanding of COTS components. In Proceedings: Seventh International Workshop on Program Comprehension, pp. 92–99, IEEE Computer Society Press.

  • Kranitis, N., Paschalis, A., Gizopoulos, D., Zorian, Y. 2002. Effective software self-test methodology for processor cores. In Proceedings of the Conference on Design, Automation and Test in Europe, p. 592, IEEE Computer Society.

  • Krstic, A., Lai, W.C., Cheng, K.T., Chen, L., Dey, S. 2002. Embedded software-based self-testing for soc design. In Proceedings of the 39th Conference on Design Automation, pp. 355–360, ACM Press.

  • Kung, D.C., Gao, J., Hsia, P., Toyoshima, Y., Chen, C. 1996. On regression testing of object-oriented programs. The Journal of Systems and Software 32(1):21–31.

    Article  Google Scholar 

  • Lee White, H.A., Alzeidi, N. 2001. User-based testing of gui sequences and their interactions. In Proceedings of the 12th International Symposium Software Reliability Engineering, pp. 54–63.

  • Lu, G. 1991. Automatic testing for the applications with direct manipulation interfaces. In Proceedings of the Fourth International Conference on Human-Computer Interaction, vol. 1 of Congress II: Design and Implementation of Interactive Systems: SOFTWARE ENGINEERING—METHODS, TECHNIQUES AND TOOLS; Applications of Techniques and Tools, pp. 602–606.

  • Marick, B. 2002. Bypassing the GUI. Software Testing and Quality Engineering Magazine: 41–47.

  • Memon, A., Nagarajan, A., Xie, Q. 2005. Automating regression testing for evolving GUI software. Journal of Software Maintenance and Evolution: Research and Practice 17(1):27–64.

    Article  Google Scholar 

  • Memon, A.M. 2001. A Comprehensive Framework for Testing Graphical User Interfaces. Ph.D. thesis, Department of Computer Science, University of Pittsburgh.

  • Memon, A.M. 2002. GUI testing: Pitfalls and process. IEEE Computer 35(8)90–91.

    Google Scholar 

  • Memon, A.M., Banerjee, I., Nagarajan, A. 2003. GUI ripping: Reverse engineering of graphical user interfaces for testing. In Proceedings of the 10th Working Conference on Reverse Engineering.

  • Memon, A.M., Pollack, M.E., Soffa, M.L. 1999. Using a goal-driven approach to generate test cases for GUIs. In Proceedings of the 21st International Conference on Software Engineering, pp. 257–266, ACM Press.

  • Memon, A.M., Pollack, M.E., Soffa, M.L. 2000a. Automated test oracles for GUIs. In Proceedings of the ACM SIGSOFT 8th International Symposium on the Foundations of Software Engineering (FSE-8) (NY), pp. 30–39.

  • Memon, A.M., Pollack, M.E., Soffa, M.L. 2000b. Plan generation for GUI testing. In Proceedings of the Fifth International Conference on Artificial Intelligence Planning and Scheduling, pp. 226–235, AAAI Press.

  • Memon, A.M., Pollack, M.E., Soffa, M.L. 2001a. Hierarchical GUI test case generation using automated planning. IEEE Transactions on Software Engineering 27(2):144–155.

    Article  Google Scholar 

  • Memon, A.M., Soffa, M.L. 2003. Regression testing of guis. In Proceedings of the 9th European Software Engineering Conference held jointly with 10th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 118–127, ACM Press.

  • Memon, A.M., Soffa, M.L., Pollack, M.E. 2001b. Coverage criteria for GUI testing. In Proceedings of the 8th European Software Engineering Conference (ESEC) and 9th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE-9), pp. 256–267.

  • Memon, A.M., Xie, Q. 2004. Using transient/persistent errors to develop automated test oracles for event-driven software. In Proceedings of The International Conference on Automated Software Engineering 2004 (ASE’04), pp. 186–195.

  • Memon A.M., Xie, Q. 2005. Studying the fault-detection effectiveness of GUI test cases for rapidly evolving software. IEEE Transactions on Software Engineering, 31(10):884–896.

    Article  Google Scholar 

  • Mercury Interactive WinRunner, 2003. http://www.mercuryinteractive.com/products/winrunner.

  • Morris, J., Lam, P., Lee, G., Parker, K., Bundell, G.A. 2002. Determining component reliability using a testing index. In Proceedings of the Twenty-fifth Australasian Conference on Computer Science, pp. 167–176, Australian Computer Society, Inc.

  • Myers, B.A. 1993. Why are human-computer interfaces difficult to design and implement? Technical Report CS-93-183, Carnegie Mellon University, School of Computer Science.

  • Nagarajan, A., Memon, A.M. 2003. Refactoring using event-based profiling of GUI applications. In Proceedings of the First International Workshop on REFactoring: Achievements, Challenges, Effects 2003.

  • Onoma, A.K., Tsai, W.-T., Poonawala, M., Suganuma, H. 1998. Regression testing in an industrial environment. Commun. ACM 41(5)81–86.

    Article  Google Scholar 

  • Orso, A., Harrold, M.J., Rosenblum, D.S. 2001a. Component metadata for software engineering tasks. In Revised Papers from the Second International Workshop on Engineering Distributed Objects, pp. 129–144, Springer-Verlag.

  • Orso, A., Harrold, M.J., Rosenblum, D.S., Rothermel, G., Soffa, M.L., Do, H. 2001b. Using component metadata to support the regression testing of component -based software. In Proceedings of the IEEE International Conference on Software Maintenance.

  • Rational Robot, 2003. http://www.rational.com.ar/tools/robot.html.

  • Richardson, D.J., Leif-Aha, S., OMalley, T.O. 1992. Specification-based Test Oracles for Reactive Systems. In Proceedings of the 14th International Conference on Software Engineering, pp. 105–118.

  • Sparling, M. 2000. Lessons learned through six years of component-based development. Communication ACM 43(10):47–53.

    Article  Google Scholar 

  • Ulrich, A., Chrobok-Diening, G. 1999. International workshop on testing distributed component-based systems. SIGSOFT Software Engineering Notes 24(4):43–46.

    Google Scholar 

  • Vitharana, P. 2003. Risks and challenges of component-based software development. Communication ACM 46(8):67–72.

    Article  Google Scholar 

  • Voas, J., Charron, F., McGraw, G., Miller, K., Friedman, M. 1997. Predicting how badly “good” software can behave. IEEE Software 14(4):73–83.

    Article  Google Scholar 

  • Voas, J., Payne, J. 1998. Cots software failures: Can anything be done? In Proceedings of the First IEEE Workshop on Application Specific Software Engineering and Technology (ASSET’98), pp. 140–145, IEEE Press.

  • White, L. 1996. Regression testing of GUI event interactions. In Proceedings of the International Conference on Software Maintenance (Washington, Nov.4–8), pp. 350–358.

  • Wittenberg, C.H. 2000. Progress in testing component-based software (abstract only). In Proceedings of the International Symposium on Software Testing and Analysis, p. 178, ACM Press.

  • Xie, Q., Memon, A.M. 2005. Rapid crash testing for continuously evolving GUI-based software applications. In Proceedings of The International Conference on Software Maintenance 2005 (ICSM’05) (Budapest, Hungary).

  • Xie, Q., Memon, A.M. Designing and comparing automated test oracles for GUI-based software applications. ACM Transactions on Software Testing and Methodology (to appear).

  • Zhao, W., Papachristou, C. 1998. Testing dsp cores based on self-test programs. In Proceedings of the Conference on Design, Automation and Test in Europe, pp. 166–172, IEEE Computer Society.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Maryland, College Park, MD, 20742, USA

    Atif M. Memon

Authors
  1. Atif M. Memon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Atif M. Memon.

Additional information

Atif M. Memon is an Assistant Professor at the Department of Computer Science, University of Maryland. He received his BS, MS, and Ph.D. in Computer Science in 1991, 1995, and 2001 respectively. He was awarded a Gold Medal in BS. He was awarded Fellowships from the Andrew Mellon Foundation for his Ph.D. research. He received the NSF CAREER award in 2005. His research interests include program testing, software engineering, artificial intelligence, plan generation, reverse engineering, and program structures. He is a member of the ACM and the IEEE Computer Society.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Memon, A.M. Employing user profiles to test a new version of a GUI component in its context of use. Software Qual J 14, 359–377 (2006). https://doi.org/10.1007/s11219-006-0040-7

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-006-0040-7

Keyword

Search

Navigation