Abstract
Virtual reality (VR) and VR applications have reached the end-user and, hence, the demands on usability, also for novel applications, have increased. This situation requires VR usability evaluation methods that can be applied quickly, even after a first release of an application. In this article, we describe such an approach, which is fully automated and does not ask users to perform predefined tasks in a fixed test setting. Instead, it works on recordings of the actual usage of a VR application from which it generates task trees. Afterwards, it analyzes these task trees to search for usability smells, i.e., user behavior indicating usability issues. Our approach provides detailed descriptions of the usability issues that have been found and how they can be solved. We performed a large case study to evaluate our approach and show that it is capable of correctly identifying usability issues. Although our approach is applicable for different VR interaction modalities, such as gaze, controller, or hand interaction, it also has limitations. For example, it can detect diverse issues related to user efficiency, but specific misunderstandings of users cannot be uncovered.
- 1998. ISO 9241-11: Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability (ISO 9241-11:1998).Google Scholar
- 2018. VRTK - Virtual Reality Toolkit. Retrieved 1, 2018 from https://vrtoolkit.readme.io/.Google Scholar
- Diogo Almeida, José Creissac Campos, João Saraiva, and João Carlos Silva. 2015. Towards a catalog of usability smells. In Proceedings of the Volume I: Artificial Intelligence and Agents, Distributed Systems, and Information Systems (ACM SAC’15). ACM, 175--181. Google ScholarDigital Library
- Saleema Amershi, Jalal Mahmud, Jeffrey Nichols, Tessa Lau, and German Attanasio Ruiz. 2013. LiveAction: Automating web task model generation. ACM Trans. Interact. Intell. Syst. 3, 3, Article 14 (Oct. 2013), 23 pages. Google ScholarDigital Library
- Richard Atterer. 2008. Usability Tool Support for Model-Based Web Development. dissertation. http://nbn-resolving.de/urn:nbn:de:bvb:19-92963.Google Scholar
- Fiora T. W. Au, Simon Baker, Ian Warren, and Gillian Dobbie. 2008. Automated usability testing framework. In Proceedings of the 9th Conference on Australasian User Interface - Volume 76 (AUIC’08). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 55--64. http://dl.acm.org/citation.cfm?id=1378337.1378349. Google ScholarDigital Library
- Rebecca Baker. 2014. Remote Usability Testing: Thinking Outside the Lab. Retrieved 11, 2017 from http://www.uxmatters.com/mt/archives/2014/04/remote-usability-testing-thinking-outside-the-lab.php.Google Scholar
- S. Baker, F. Au, G. Dobbie, and I. Warren. 2008. Automated usability testing using HUI analyzer. In Proceedings of the 19th Australian Conference on Software Engineering (ASWEC’08). 579--588. Google ScholarDigital Library
- Aurora Bedford. 2015. Don’t Prioritize Efficiency Over Expectations. Retrieved 9, 2017 from http://www.nngroup.com/articles/efficiency-vs-expectations/.Google Scholar
- Aurora Bedford. 2015. No More Pogo Sticking: Protect Users from Wasted Clicks. Retrieved 9, 2017 from http://www.nngroup.com/articles/pogo-sticking/.Google Scholar
- Daniel J. Benjamin, James O. Berger, Magnus Johannesson, Brian A. Nosek, Eric-Jan Wagenmakers, Richard Berk, Kenneth A. Bollen, Björn Brembs, Lawrence Brown, Colin Camerer, et al. 2017. Redefine statistical significance. Nat. Hum. Behav. (2017).Google Scholar
- Nicholas Benson. 2017. Interaction Engine. Retrieved 9, 2017 from https://github.com/leapmotion/UnityModules/wiki/Interaction-Engine.Google Scholar
- Bootstrap core team and contributors. 2017. Bootstrap. Retrieved 11, 2017 from https://getbootstrap.com/.Google Scholar
- Doug A. Bowman, Joseph L. Gabbard, and Deborah Hix. 2002. A survey of usability evaluation in virtual environments: Classification and comparison of methods. Presence: Teleoper. Virtual Environ. 11, 4 (Aug. 2002), 404--424. Google ScholarDigital Library
- Paolo Burzacca and Fabio Paternò. 2013. Remote usability evaluation of mobile web applications. In Human-Computer Interaction. Human-Centred Design Approaches, Methods, Tools, and Environments, Masaaki Kurosu (Ed.). Lecture Notes in Computer Science, Vol. 8004. Springer Berlin Heidelberg, 241--248. Google ScholarDigital Library
- Tonio Carta, Fabio Paternò, and Vagner Santana. 2011. Support for remote usability evaluation of web mobile applications. In Proceedings of the 29th ACM International Conference on Design of Communication (SIGDOC’11). ACM, New York, NY, 129--136. Google ScholarDigital Library
- Cátedra SAES de la Universidad de Murcia. 2014. OHT Plus: An application framework for non-intrusive usability testing tools. Retrieved 6, 2014 from http://www.catedrasaes.org/wiki/OHTPlus.Google Scholar
- Selem Charfi, Houcine Ezzedine, Christophe Kolski, and Faouzi Moussa. 2011. Towards an automatic analysis of interaction data for HCI evaluation: Application to a transport network supervision system. In Proceedings of the 14th International Conference on Human-Computer Interaction: Design and Development Approaches - Volume Part I (HCII’11). Springer-Verlag, Berlin, Heidelberg, 175--184. http://dl.acm.org/citation.cfm?id=2022384.2022407. Google ScholarDigital Library
- HTC Corporation. 2017. Vive. Retrieved 9, 2017 from https://www.vive.com/.Google Scholar
- Allen Cypher. 1991. EAGER: Programming repetitive tasks by example. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’91). ACM, New York, NY, 33--39. Google ScholarDigital Library
- Vagner Figuerêdo de Santana and Maria Cecília Calani Baranauskas. 2015. WELFIT: A remote evaluation tool for identifying Web usage patterns through client-side logging. Int. J. Hum.-Comput. Studies 76 (2015), 40--49. Google ScholarDigital Library
- Deloitte. 2016. Head Mounted Displays in deutschen Unternehmen - Ein Virtual, Augmented und Mixed Reality Check. Retrieved 11, 2017 from http://www2.deloitte.com/de/de/pages/technology-media-and-telecommunications/articles/head-mounted-displays-in-deutschen-unternehmen.html.Google Scholar
- Alexiei Dingli and Justin Mifsud. 2011. USEFul: A framework to mainstream web site usability through automated evaluation. Int. J. Hum. Comput. Interact. 2, 1 (2011), 10--30. http://cscjournals.org/csc/manuscript/Journals/IJHCI/volume2/Issue1/IJHCI-19.pdf.Google Scholar
- Ralf Dörner, Wolfgang Broll, Paul Grimm, and Bernhard Jung (Eds.). 2013. Virtual und Augmented Reality (VR / AR): Grundlagen und Methoden der Virtuellen und Augmentierten Realität. Springer Berlin Heidelberg, Berlin, Heidelberg.Google Scholar
- Arianna D’Ulizia, Fernando Ferri, and Patrizia Grifoni. 2011. A survey of grammatical inference methods for natural language learning. Artif. Intell. Rev. 36, 1 (Jun. 2011), 1--27. Google ScholarDigital Library
- Sebastian Feuerstack, Marco Blumendorf, Maximilian Kern, Michael Kruppa, Michael Quade, Mathias Runge, and Sahin Albayrak. 2008. Automated usability evaluation during model-based interactive system development. In Proceedings of the 2nd Conference on Human-Centered Software Engineering and 7th International Workshop on Task Models and Diagrams (HCSE-TAMODIA’08). Springer-Verlag, Berlin, Heidelberg, 134--141. Google ScholarDigital Library
- FFsplit. 2017. FFSPLIT. Retrieved 9, 2017 from http://www.ffsplit.com.Google Scholar
- Software Engineering for Distributed System. 2017. AutoQUEST -- Testing, Analysing, and Observing Event-driven Software. Retrieved 9, 2017 from https://autoquest.informatik.uni-goettingen.de/.Google Scholar
- Peter Géczy, Noriaki Izumi, Shotaro Akaho, and Kôiti Hasida. 2007. Usability analysis framework based on behavioral segmentation. In E-Commerce and Web Technologies, Giuseppe Psaila and Roland Wagner (Eds.). Lecture Notes in Computer Science, Vol. 4655. Springer Berlin Heidelberg, 35--45. Google ScholarDigital Library
- Steven Gomez and David Laidlaw. 2012. Modeling task performance for a crowd of users from interaction histories. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, 2465--2468. Google ScholarDigital Library
- Google. 2017. Google Analytics. Retrieved 12, 2017 from http://www.google.com/analytics/.Google Scholar
- Google. 2018. Test auf Optimierung für Mobilgeräte. Retrieved 1, 2018 from https://search.google.com/test/mobile-friendly.Google Scholar
- Google LLC. 2017. Android Developers. Retrieved 11, 2017 from https://developer.android.com/index.html.Google Scholar
- Julián Grigera, Alejandra Garrido, and José Matías Rivero. 2014. A tool for detecting bad usability smells in an automatic way. In Web Engineering, Sven Casteleyn, Gustavo Rossi, and Marco Winckler (Eds.). Lecture Notes in Computer Science, Vol. 8541. Springer International Publishing, 490--493.Google Scholar
- Julián Grigera, Alejandra Garrido, José Matías Rivero, and Gustavo Rossi. 2017. Automatic detection of usability smells in web applications. Int. J. Hum.-Comput. Studies 97 (2017), 129--148.Google ScholarCross Ref
- Karl Grooves. 2007. The Limitations of Server Log Files for Usability Analysis. Retrieved 6, 2015 from http://boxesandarrows.com/the-limitations-of-server-log-files-for-usability-analysis/.Google Scholar
- Patrick Harms. 2016. Automated Field Usability Evaluation Using Generated Task Trees. Ph.D. Dissertation. http://hdl.handle.net/11858/00-1735-0000-0028-8684-1.Google Scholar
- Patrick Harms and Jens Grabowski. 2014. Usage-based automatic detection of usability smells. In Human-Centered Software Engineering, Stefan Sauer, Cristian Bogdan, Peter Forbrig, Regina Bernhaupt, and Marco Winckler (Eds.). Lecture Notes in Computer Science, Vol. 8742. Springer Berlin Heidelberg, 217--234. Google ScholarDigital Library
- Patrick Harms and Jens Grabowski. 2015. Consistency of task trees generated from website usage traces. In Proceedings of the 17th International Conference on System Design Languages (SDL’15). Springer Berlin Heidelberg, 8. Google ScholarDigital Library
- Patrick Harms, Steffen Herbold, and Jens Grabowski. 2014. Extended trace-based task tree generation. Int. J. Adv. Intel. Syst. 7, 3, 4 (Dec. 2014), 450--467. http://www.iariajournals.org/intelligent_systems/.Google Scholar
- Patrick Harms, Steffen Herbold, and Jens Grabowski. 2014. Trace-based task tree generation. In Proceedings of the 7th International Conference on Advances in Computer-Human Interactions (ACHI’14). XPS - Xpert Publishing Services, 6.Google Scholar
- N. Harrati, I. Bouchrika, A. Tari, and A. Ladjailia. 2015. Automating the evaluation of usability remotely for web applications via a model-based approach. In Proceedings of the 1st International Conference on New Technologies of Information and Communication (NTIC’15). 1--6.Google Scholar
- Marcus Hegner. 2003. Methoden zur Evaluation von Software. IZ, InformationsZentrum Sozialwiss. http://books.google.de/books?id=NhnMHAAACAAJ.Google Scholar
- David M. Hilbert and David F. Redmiles. 2000. Extracting usability information from user interface events. ACM Comput. Surv. 32, 4 (Dec. 2000), 384--421. Google ScholarDigital Library
- Yosef Hochberg. 1988. A sharper bonferroni procedure for multiple tests of significance. 75 (Dec. 1988), 800--802.Google Scholar
- Scott E. Hudson, Bonnie E. John, Keith Knudsen, and Michael D. Byrne. 1999. A tool for creating predictive performance models from user interface demonstrations. In Proceedings of the 12th Annual ACM Symposium on User Interface Software and Technology (UIST’99). ACM, New York, NY, 93--102. Google ScholarDigital Library
- Melody Y. Ivory and Marti A. Hearst. 2001. The state of the art in automating usability evaluation of user interfaces. ACM Comput. Surv. 33, 4 (Dec. 2001), 470--516. Google ScholarDigital Library
- Anthony Jameson, Angela Mahr, Michael Kruppa, Andreas Rieger, and Robert Schleicher. 2007. Looking for unexpected consequences of interface design decisions: The MeMo workbench. In Task Models and Diagrams for User Interface Design, Marco Winckler, Hilary Johnson, and Philippe Palanque (Eds.). Lecture Notes in Computer Science, Vol. 4849. Springer Berlin, Heidelberg, 279--286. Google ScholarDigital Library
- Kasper Løvborg Jensen and Lars Bo Larsen. 2007. Evaluating the usefulness of mobile services based on captured usage data from longitudinal field trials. In Proceedings of the 4th International Conference on Mobile Technology, Applications, and Systems and the 1st International Symposium on Computer Human Interaction in Mobile Technology (Mobility’07). ACM, New York, NY, 675--682. Google ScholarDigital Library
- Bonnie E. John. 2015. CogTool. Retrieved 6, 2015 from http://cogtool.com/.Google Scholar
- Bonnie E. John, Konstantine Prevas, Dario D. Salvucci, and Ken Koedinger. 2004. Predictive human performance modeling made easy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’04). ACM, New York, NY, 455--462. Google ScholarDigital Library
- Christos Katsanos, Nikolaos Tselios, and Nikolaos Avouris. 2006. InfoScent evaluator: A semi-automated tool to evaluate semantic appropriateness of hyperlinks in a web site. In Proceedings of the 18th Australia Conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments (OZCHI’06). ACM, New York, NY, 373--376. Google ScholarDigital Library
- Jun H. Kim, Daniel V. Gunn, Eric Schuh, Bruce Phillips, Randy J. Pagulayan, and Dennis Wixon. 2008. Tracking real-time user experience (TRUE): A comprehensive instrumentation solution for complex systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’08). ACM, New York, NY, 443--452. Google ScholarDigital Library
- Ralph Krimmel. 2014. Improving Automatic Task Tree Generation With Alignment Algorithms.Google Scholar
- Inc. Leap Motion. 2017. Unity. Retrieved 9, 2017 from https://developer.leapmotion.com/unity/.Google Scholar
- Andreas Lecerof and Fabio Paternò. 1998. Automatic support for usability evaluation. IEEE Trans. Softw. Eng. 24, 10 (Oct. 1998), 863--888. Google ScholarDigital Library
- Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte, Laurent Bouillon, Murielle Florins, and Daniela Trevisan. 2004. UsiXML: A user interface description language for context-sensitive user interfaces. In Proceedings of the ACM AVI’2004 Workshop “Developing User Interfaces with XML: Advances on User Interface Description Languages,” 55--62.Google Scholar
- Matomo.org. 2018. Matomo -- Liberating Analytics. Retrieved 1, 2018 from https://matomo.org/.Google Scholar
- David Maulsby. 1997. Inductive task modeling for user interface customization. In Proceedings of the 2nd International Conference on Intelligent User Interfaces (IUI’97). ACM, New York, NY, 233--236. Google ScholarDigital Library
- Thomas Memmel. 2009. User Interface Specification for Interactive Software Systems - Process-, Method- and Tool-Support for Interdisciplinary and Collaborative Requirements Modelling and Prototyping-Driven User Interface Specification. PhD thesis. University of Konstanz.Google Scholar
- Microsoft. 2018. Das neue Windows Mixed Reality. Retrieved 1, 2018 from https://www.microsoft.com/de-de/windows/windows-mixed-reality.Google Scholar
- C. Moser. 2012. User Experience Design: Mit erlebniszentrierter Softwareentwicklung zu Produkten, die begeistern. Springer Berlin Heidelberg.Google Scholar
- Donald A. Norman. 2002. The Design of Everyday Things (1. basic paperback ed., {repr..} ed.). Basic Books, {New York, NY}. Google ScholarDigital Library
- Oculus. 2018. oculus rift. Retrieved 1, 2018 from https://www.oculus.com/rift/.Google Scholar
- U.S. Dept. of Health and Human Services. 2006. The Research-Based Web Design 8 Usability Guidelines, Enlarged/Expanded edition. Retrieved 7, 2017 from http://guidelines.usability.gov/.Google Scholar
- Gary M. Olson, James D. Herbsleb, and Henry H. Rueter. 1994. Characterizing the sequential structure of interactive behaviors through statistical and grammatical techniques. Hum-Comput. Interact. 9 (1994), 427--472. Google ScholarDigital Library
- Oracle. 2017. Trail: Creating a GUI With JFC/Swing. Retrieved 11, 2017 from https://docs.oracle.com/javase/tutorial/uiswing/.Google Scholar
- Fabio Paternò. 2003. ConcurTaskTrees: An engineered notation for task models. In The Handbook of Task Analysis for Human Computer Interaction, Dan Diaper and Neville Stanton (Eds.). Lawrence Erlbaum Associates Publishers, 483--503.Google Scholar
- Fabio Paternò. 2003. Tools for remote web usability evaluation. In Proceedings of the 10th International Conference on Human-Computer Interaction., Vol. 1. Erlbaum, 828--832. http://giove.isti.cnr.it/attachments/publications/2003-A2-95.pdf.Google Scholar
- Fabio Paternò, Cristiano Mancini, and Silvia Meniconi. 1997. ConcurTaskTrees: A diagrammatic notation for specifying task models. In Proceedings of the IFIP TC13 International Conference on Human-Computer Interaction (INTERACT’97). Chapman 8 Hall, Ltd., London, UK, 362--369. Google ScholarDigital Library
- Fabio Paternò, Angela Piruzza, and Carmen Santoro. 2005. Remote usability analysis of MultiModal information regarding user behaviour. InInternational COST 294 Workshop on User Interface Quality Models, pp. 15--22. http://giove.isti.cnr.it/attachments/publications/2005-A2-134.pdf.Google Scholar
- Fabio Paternò, Angela Piruzza, and Carmen Santoro. 2007. Remote web usability evaluation exploiting multimodal information on user behavior. In Computer-Aided Design of User Interfaces V, Gaëlle Calvary, Costin Pribeanu, Giuseppe Santucci, and Jean Vanderdonckt (Eds.). Springer Netherlands, 287--298.Google Scholar
- Fabio Paternò, Andrea Russino, and Carmen Santoro. 2007. Remote evaluation of mobile applications. In Task Models and Diagrams for User Interface Design, Marco Winckler, Hilary Johnson, and Philippe Palanque (Eds.). Lecture Notes in Computer Science, Vol. 4849. Springer, Berlin, 155--169. Google ScholarDigital Library
- Fabio Paternò, Antonio Giovanni Schiavone, and Antonio Conti. 2017. Customizable automatic detection of bad usability smells in mobile accessed web applications. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’17). ACM, New York, NY, Article 42, 11 pages. Google ScholarDigital Library
- Michael Quade, Marco Blumendorf, and Sahin Albayrak. 2010. Towards model-based runtime evaluation and adaptation of user interfaces. In User Modeling and Adaptation for Daily Routines: Providing Assistance to People with Special and Specific Needs.Google Scholar
- Neil Ramsay, Stuart Marshall, and Alex Potanin. 2008. Annotating UI architecture with actual use. In Proceedings of the Ninth Conference on Australasian User Interface -- Volume 76 (AUIC’08). Australian Computer Society, Inc., Darlinghurst, Australia, 75--78. http://dl.acm.org/citation.cfm?id=1378337.1378351. Google ScholarDigital Library
- Karen Renaud and Phil Gray. 2004. Making sense of low-level usage data to understand user activities. In Proceedings of the Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists on IT Research in Developing Countries (SAICSIT’04). South African Institute for Computer Scientists and Information Technologists, Republic of South Africa, 115--124. http://dl.acm.org/citation.cfm?id=1035053.1035067. Google ScholarDigital Library
- Michael Richter and Markus D. Flückiger. 2013. Usability Engineering kompakt: Benutzbare Software gezielt entwickeln. Springer Berlin, Heidelberg.Google Scholar
- Joy Robinson, Candice Lanius, and Ryan Weber. 2017. The past, present, and future of UX empirical research. Commun. Des. Q. Rev 5, 3 (Nov. 2017), 10--23. Google ScholarDigital Library
- Stephanie Rosenbaum. 2008. The future of usability evaluation: Increasing impact on value. In Maturing Usability, Effie Lai-Chong Law, Ebba Thora Hvannberg, and Gilbert Cockton (Eds.). Springer, 344--378. http://dblp.uni-trier.de/db/series/hci/LawHC08.html#Rosenbaum08.Google Scholar
- Jean-David Ruvini et al. 2000. APE: Learning user’s habits to automate repetitive tasks. In Proceedings of the Conference on Intelligent User Interfaces (IUI’00). 229--232. Google ScholarDigital Library
- Dario D. Salvucci. 2009. Rapid prototyping and evaluation of in-vehicle interfaces. ACM Trans. Comput.-Hum. Interact. 16, 2, Article 9 (June 2009), 33 pages. Google ScholarDigital Library
- Florian Sarodnick and Henning Brau. 2006. Methoden der Usability Evaluation: Wissenschaftliche Grundlagen und praktische Anwendung (1st ed.). Huber, Bern.Google Scholar
- Andrew Sears. 1995. AIDE: A step toward metric-based interface development tools. In Proceedings of the 8th Annual ACM Symposium on User Interface and Software Technology (UIST’95). ACM, New York, NY, 101--110. Google ScholarDigital Library
- Antonio C. Siochi and Roger W. Ehrich. 1991. Computer analysis of user interfaces based on repetition in transcripts of user sessions. ACM Trans. Inf. Syst. 9, 4 (Oct. 1991), 309--335. Google ScholarDigital Library
- Antonio C. Siochi and Deborah Hix. 1991. A study of computer-supported user interface evaluation using maximal repeating pattern analysis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’91). ACM, New York, NY, 301--305. Google ScholarDigital Library
- Jenifer Tidwell. 2010. Designing Interfaces -- Patterns for Effective Interaction Design (2nd ed.). O’Reilly Media, Incorporated. http://books.google.de/books?id=5gvOU9X0fu0C. Google ScholarDigital Library
- Thomas Tiedtke, Christian Märtin, and Norbert Gerth. 2002. AWUSA -- A Tool for Automated Website Usability Analysis. In Proceedings of the 9th International Workshop on the Design, Specification and Verification of Interactive Systems (DSV-IS’02).Google Scholar
- H. Trætteberg. 2002. Model-based User Interface Design. Information Systems Group, Department of Computer and Information Sciences, Faculty of Information Technology, Mathematics and Electrical Engineering, Norwegian University of Science and Technology.Google Scholar
- Thomas Tullis and William Albert. 2008. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Morgan Kaufmann Publishers Inc., San Francisco, CA. Google ScholarDigital Library
- Unity Technologies. 2017. Unity 3D. Retrieved 9, 2017 from https://unity3d.com.Google Scholar
- Martijn Van Welie, Gerrit C. Van Der Veer, and Anton Eliëns. 1998. An ontology for task world models. In Proceedings of Design, Specification and Verification of Interactive Systems (DSV-IS’98), Abingdon. http://citeseerx.ist.psu.edu/viewdoc/summary.Google Scholar
- Google VR. 2017. Google Cardboard. Retrieved 9, 2017 from https://vr.google.com/cardboard/.Google Scholar
- Google VR. 2017. Google Daydream. Retrieved 9, 2017 from https://vr.google.com/daydream/.Google Scholar
- VRgluv. 2018. VRgluv. Retrieved 1, 2018 from https://vrgluv.com/.Google Scholar
- W3Schools. 2017. HTML5 Tutorial. Retrieved 11, 2017 from https://www.w3schools.com/html/default.asp.Google Scholar
- G. Wallner and S. Kriglstein. 2014. PLATO: A visual analytics system for gameplay data. Comput. Graph. 38 (2014), 341--356. Google ScholarDigital Library
- Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biomet. Bull. 1, 6 (1945), 80--83. http://www.jstor.org/stable/3001968.Google ScholarCross Ref
- D. Zwillinger and S. Kokoska. 1999. CRC Standard Probability and Statistics Tables and Formulae. CRC.Google Scholar
Index Terms
- Automated Usability Evaluation of Virtual Reality Applications
Recommendations
Evaluation of virtual reality systems for usability
CHI EA '99: CHI '99 Extended Abstracts on Human Factors in Computing SystemsThe adoption of 3D virtual interfaces is becoming more widespread. Although design guidelines and usability evaluation methods exist for the conventional 2 dimensional Graphical User Interface (GUI), they may or may not however, be appropriate in the ...
Automated Usability Smell Detection in VR Application with AutoQUEST
EICS '22 Companion: Companion of the 2022 ACM SIGCHI Symposium on Engineering Interactive Computing SystemsThe quality of software products is not only measured by their set of functionality and features, the User Experience (UX) and the quality of the applications’ User Interface (UI) gets more and more important. In order to measure and improve the ...
Usability Evaluation for Drone Mission Planning in Virtual Reality
Virtual, Augmented and Mixed Reality: Applications in Health, Cultural Heritage, and IndustryAbstractThere has been a lot of research conducted on human-robot interaction (HRI) with drones as well as human-computer interaction (HCI) with virtual reality (VR). However, little work was done on VR as HRI, even less for VR and drones in particular. ...
Comments