research-article

A feasibility study of usability and UX evaluation technologies in multi-touch context: A quantitative and qualitative analysis

Authors:

Guilherme Filho,

Guilherme Guerino,

Natasha ValentimAuthors Info & Claims

SBES '23: Proceedings of the XXXVII Brazilian Symposium on Software Engineering

Pages 475 - 484

https://doi.org/10.1145/3613372.3613393

Published: 25 September 2023 Publication History

Abstract

The use of multi-touch systems has been increasing in the last few years. These systems also need to pass through an evaluation to check their quality level. For this, usability and User Experience (UX) are two important aspects. Seeking to find out which technologies are being used for this purpose, we performed a Systematic Mapping Study (SMS). We discover that no technologies are being used to evaluate usability and UX on multi-touch interfaces that consider the specificities of this context, such as lack of touch precision and mechanical feedback. We proposed the User Experience and usability Multi-touch Evaluation Questionnaire (UXUMEQ) to fill this gap. This technology was evaluated through a feasibility study performed with 47 participants. Efficiency and effectiveness were compared to a combination of the System usability Scale (SUS) and Intuitive Questionnaire (INTUI). According to quantitative results, UXUMEQ presented a statistically significant difference over SUS+INTUI in terms of efficiency, while in effectiveness the comparison remained similar. In other words, UXUMEQ allows the users to find more problems, though it cannot be said that it saves time in this search compared to SUS+INTUI. UXUMEQ was considered easy to use, proposal-fulfilling, and highly assertive by the study participants. The qualitative data was analyzed and generated in a new version of UXUMEQ.

References

[1]

Victor R Basili and H Dieter Rombach. 1988. Towards a comprehensive framework for reuse: A reuse-enabling software evolution environment. In NASA, Goddard Space Flight Center, Proceedings of the Thirteenth Annual Software Engineering Workshop.

[2]

Joshua Blake. 2011. Natural user interfaces in. NET: WPF 4, Surface 2, and Kinect. Manning.

[3]

J. Brooke. 1996. SUS: A “quick and dirty” usability scale. In Usability evaluation in industry, P. W. JORDAN and et al. (Eds.). Taylor&Francis, London, 1–7.

[4]

Bill Buxton 2007. Multi-touch systems that I have known and loved. Microsoft Research 56 (2007), 1–11.

[5]

Jeffrey Carver, Letizia Jaccheri, Sandro Morasca, and Forrest Shull. 2004. Issues in using students in empirical studies in software engineering education. In Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No. 03EX717). IEEE, 239–249.

[6]

Tayana Conte, Jobson Massollar, Emilia Mendes, and Guilherme H Travassos. 2007. Usability evaluation based on web design perspectives. In First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007). IEEE, 146–155.

Digital Library

[7]

Juliet Corbin and Anselm Strauss. 2014. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications.

[8]

Fred D Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly (1989), 319–340.

Digital Library

[9]

Statista Research Department. 2022. Global smartphone sales to end users since 2007.

[10]

Connor Dodd, Rukshan Athauda, and Marc Adam. 2017. Designing user interfaces for the elderly: a systematic literature review. (2017).

[11]

Hadziq Fabroyir. 2019. Multitouch Interface is not Good for Spatial Navigation in Virtual Reality. In 2019 12th International Conference on Information & Communication Technology and System (ICTS). IEEE, 323–326.

[12]

Adrian Fernandez, Silvia Abrahão, Emilio Insfran, and Maristella Matera. 2012. Further analysis on the validation of a usability inspection method for model-driven web development. In Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement. 153–156.

Digital Library

[13]

Adrian Fernandez, Silvia Abrahão, Emilio Insfran, and Maristella Matera. 2013. Usability inspection in model-driven web development: Empirical validation in webml. In Model-Driven Engineering Languages and Systems: 16th International Conference, MODELS 2013, Miami, FL, USA, September 29–October 4, 2013. Proceedings 16. Springer, 740–756.

Digital Library

[14]

Saad Q Fleh, Oğuz Bayat, Saad Al-Azawi, and Osman Nuri Uçan. 2018. A systematic mapping study on touch classification. (2018).

[15]

Market Data Forecast. 2022. Smartphone market size and growth.

[16]

Clifton Forlines, Daniel Wigdor, Chia Shen, and Ravin Balakrishnan. 2007. Direct-touch vs. mouse input for tabletop displays. In Proceedings of the SIGCHI conference on Human factors in computing systems. 647–656.

Digital Library

[17]

Guilherme Corredato Guerino and Natasha Malveira Costa Valentim. 2020. Usability and user experience evaluation of natural user interfaces: a systematic mapping study. IET Software 14, 5 (2020), 451–467.

Digital Library

[18]

Hejar Gürlük, Malte-Levin Jauer, and Maria Uebbing-Rumke. 2014. Design and evaluation of a multi-touch interaction language for approach controllers. In Proceedings of the International Conference on Human-Computer Interaction in Aerospace. 1–4.

Digital Library

[19]

Marc Hassenzahl. 2004. The interplay of beauty, goodness, and usability in interactive products. Human–Computer Interaction 19, 4 (2004), 319–349.

[20]

ISO 9241-9. 2000. Ergonomics of Human System Interaction - Part 9: Requirements for non-keyboard input devices. International Organization for Standardization.

[21]

ISO/IEC 25010. 2011. Systems and Software Engineering - SquaRE - Software product Quality Requirements and Evaluation: System and Software Quality Models). International Organization for Standardization.

[22]

Giulio Jacucci, Ann Morrison, Gabriela T Richard, Jari Kleimola, Peter Peltonen, Lorenza Parisi, and Toni Laitinen. 2010. Worlds of information: designing for engagement at a public multi-touch display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2267–2276.

Digital Library

[23]

Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing systematic literature reviews in software engineering. (2007).

[24]

Guilherme Eduardo Konopatzki, Guilherme Guerino, and Natasha Valentim. 2023. Proposal and Preliminary Evaluation of a Usability and UX Multi-Touch Evaluation Technology. In Proceedings of the XIX Brazilian Symposium on Information Systems. 317–324.

Digital Library

[25]

Guilherme E Konopatzki Filho, Guilherme Corredato Guerino, and Natasha MC Valentim. 2022. A systematic mapping study on usability and user eXperience evaluation of multi-touch systems. Anais do XXI Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (2022).

[26]

Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and evaluation of a user experience questionnaire. In Symposium of the Austrian HCI and usability engineering group. Springer, 63–76.

Digital Library

[27]

Ankita Madan and Sanjay Kumar Dubey. 2012. Usability evaluation methods: a literature review. International Journal of Engineering Science and Technology 4, 2 (2012), 590–599.

[28]

Henry B Mann and Donald R Whitney. 1947. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics (1947), 50–60.

[29]

Vicente Nacher and Javier Jaen. 2015. Evaluating the accuracy of pre-kindergarten children multi-touch interaction. In IFIP Conference on Human-Computer Interaction. Springer, 549–556.

Digital Library

[30]

Fatih Nayebi, Jean-Marc Desharnais, and Alain Abran. 2012. The state of the art of mobile application usability evaluation. In 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). IEEE, 1–4.

[31]

Richard M Ryan and Edward L Deci. 2000. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.American psychologist 55, 1 (2000), 68.

[32]

Richard M Ryan, C Scott Rigby, and Andrew Przybylski. 2006. The motivational pull of video games: A self-determination theory approach. Motivation and emotion 30, 4 (2006), 344–360.

[33]

Gleison Santos, Ana Regina Rocha, Tayana Conte, Monalessa Perini Barcellos, and Rafael Prikladnicki. 2012. Strategic alignment between academy and Industry: a Virtuous Cycle to Promote Innovation in Technology. In 2012 26th Brazilian Symposium on Software Engineering. IEEE, 196–200.

Digital Library

[34]

Tim Schürmann, Christina Binder, Gesche Janzarik, and Joachim Vogt. 2015. Movement transformation on multi-touch devices: Intuition or instructional preparation?Applied Ergonomics 50 (2015), 251–255.

[35]

Forrest Shull, Jeffrey Carver, and Guilherme H Travassos. 2001. An empirical methodology for introducing software processes. ACM SIGSOFT Software Engineering Notes 26, 5 (2001), 288–296.

Digital Library

[36]

Elena Tuveri, Samuel A Iacolina, Fabio Sorrentino, L Davide Spano, and Riccardo Scateni. 2013. Controlling a planetarium software with a Kinect or in a multi-touch table: a comparison. In Proceedings of the Biannual Conference of the Italian Chapter of SIGCHI. 1–4.

Digital Library

[37]

Maria Uebbing-Rumke, Hejar Gürlük, Malte-Levin Jauer, Konrad Hagemann, and Andreas Udovic. 2014. Usability evaluation of multi-touch displays for TMA controller working positions. Proceedings of the 4th SESAR Innovation Days, Madrid, Spain (2014), 25–27.

[38]

Daniel Ullrich and Sarah Diefenbach. 2010. From magical experience to effortlessness: an exploration of the components of intuitive interaction. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries. 801–804.

Digital Library

[39]

Peter Vorderer, Werner Wirth, Feliz Ribeiro Gouveia, Frank Biocca, Timo Saari, Lutz Jäncke, Saskia Böcking, Holger Schramm, Andre Gysbers, Tilo Hartmann, 2004. Mec spatial presence questionnaire. Retrieved Sept 18 (2004), 2015.

[40]

David Watson and Lee Anna Clark. 1994. The PANAS-X: Manual for the positive and negative affect schedule-expanded form. (1994).

[41]

Diane Watson, Mark Hancock, Regan L Mandryk, and Max Birk. 2013. Deconstructing the touch experience. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces. 199–208.

Digital Library

[42]

Daniel Wigdor and Dennis Wixon. 2011. Brave NUI world: designing natural user interfaces for touch and gesture. Elsevier.

Digital Library

[43]

Bob G Witmer and Michael J Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence 7, 3 (1998), 225–240.

Digital Library

[44]

Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in software engineering. Springer Science & Business Media.

Cited By

Konopatzki GValentim NGuerino G(2024)A Qualitative Study and Improvement on UX and Usability Evaluation Questionnaire focused on Multi-touch SystemsProceedings of the XXIII Brazilian Symposium on Software Quality10.1145/3701625.3701674(362-372)Online publication date: 5-Nov-2024
https://dl.acm.org/doi/10.1145/3701625.3701674

Index Terms

A feasibility study of usability and UX evaluation technologies in multi-touch context: A quantitative and qualitative analysis
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
    1. Redundancy
  2. Embedded and cyber-physical systems
    1. Embedded systems
    2. Robotics
2. Networks
  1. Network properties
    1. Network reliability

Recommendations

Proposal and Preliminary Evaluation of a Usability and UX Multi-Touch Evaluation Technology
SBSI '23: Proceedings of the XIX Brazilian Symposium on Information Systems

Context: For multi-touch Information Systems (ISs) to reach the desired level of quality, it is necessary to have an evaluation of it. Two important quality aspects are usability and User eXperience (UX). A good assessment must take into account the ...
A systematic mapping study on usability and user eXperience evaluation of multi-touch systems
IHC '22: Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems

A significant increase in multi-touch interactions is notable with the expanded use of smartphones, tablets, and tabletops. Multi-touch systems are software that uses the user's fingers to perform some action. These systems need evaluation before being ...
A Qualitative Study and Improvement on UX and Usability Evaluation Questionnaire focused on Multi-touch Systems
SBQS '24: Proceedings of the XXIII Brazilian Symposium on Software Quality
Multi-touch systems have been popularizing in the last years. This kind of system needs to pass through an evaluation to ensure they reach a good quality level. The usability and User eXperience (UX) indicators are well-fitted to provide the necessary ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SBES '23: Proceedings of the XXXVII Brazilian Symposium on Software Engineering

September 2023

570 pages

ISBN:9798400707872

DOI:10.1145/3613372

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 September 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SBES 2023

SBES 2023: XXXVII Brazilian Symposium on Software Engineering

September 25 - 29, 2023

Campo Grande, Brazil

Acceptance Rates

Overall Acceptance Rate 147 of 427 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
75
Total Downloads

Downloads (Last 12 months)38
Downloads (Last 6 weeks)5

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Konopatzki GValentim NGuerino G(2024)A Qualitative Study and Improvement on UX and Usability Evaluation Questionnaire focused on Multi-touch SystemsProceedings of the XXIII Brazilian Symposium on Software Quality10.1145/3701625.3701674(362-372)Online publication date: 5-Nov-2024
https://dl.acm.org/doi/10.1145/3701625.3701674

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten