Corey Holland
Oleg Komogortsev
ABSTRACT
Automated detection of excessive visual search (ES) experienced by a user during software use presents the potential for substantial improvement in the efficiency of supervised usability analysis. This paper presents an objective evaluation of several methods for the automated segmentation and classification of ES intervals from an eye movement recording, a technique that can be utilized to aid in the identification of usability problems during software usability testing. Techniques considered for automated segmentation of the eye movement recording into unique intervals include mouse/keyboard events and eye movement scanpaths. ES is identified by a number of eye movement metrics, including: fixation count, saccade amplitude, convex hull area, scanpath inflections, scanpath length, and scanpath duration. The ES intervals identified by each algorithm are compared to those produced by manual classification to verify the accuracy, precision, and performance of each algorithm. The results indicate that automated classification can be successfully employed to substantially reduce the amount of recorded data reviewed by HCI experts during usability testing, with relatively little loss in accuracy.
Supplemental Material
- A. A. Witold, et al., "Consolidating the ISO Usability Models," presented at the 11th International Software Quality Management Conference and 8th Annual INSPIRE Conference, 2003.Google Scholar
- J. Rubin and D. Chisnell, Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, 2 ed. New York, NY: Wiley, 2008. Google ScholarDigital Library
- L. Vukelja, et al., "Are engineers condemned to design? a survey on software engineering and UI design in Switzerland," presented at the 11th IFIP TC 13 international conference on Human-computer interaction, Rio de Janeiro, Brazil, 2007. Google ScholarDigital Library
- J. S. Dumas and J. C. Redish, A Practical Guide to Usability Testing: Intellect Books, 1999. Google ScholarDigital Library
- R. J. Leigh and D. S. Zee, The Neurology of Eye Movements, 4 ed.: Oxford University Press, USA, 2006.Google Scholar
- A. Duchowski, Eye Tracking Methodology: Theory and Practice, 2nd ed.: Springer, 2007. Google ScholarDigital Library
- A. Poole and L. J. Ball, "Eye tracking in humancomputer interaction and usability research: current status and future prospects," in Encyclopedia of Human-Computer Interaction, C. Ghaoui, Ed., ed: Idea Group, 2005, pp. 211--219.Google Scholar
- L. J. Ball, et al., "Applying the Post-Experience EyeTracked Protocol (PEEP) Method in Usability Testing," Interfaces, vol. 67, pp. 15--19, 2006.Google Scholar
- M. C. Russell, "Hotspots and hyperlinks: using eyetracking to supplement usability testing," Usability News, vol. 7, 2005.Google Scholar
- J. M. Wolfe, "What Can 1 Million Trials Tell Us About Visual Search?," Psychological Science, vol. 9, pp. 33--39, 1998.Google ScholarCross Ref
- J. Shen, et al., "Distractor ratio influences patterns of eye movements during visual search," Perception, vol. 29, pp. 241--250, 2000.Google ScholarCross Ref
- I. D. Gilchrist and M. Harvey, "Refixation frequency and memory mechanisms in visual search," Current Biology, vol. 10, pp. 1209--1212, 2000.Google ScholarCross Ref
- A. J. Hornof and T. Halverson, "Cognitive strategies and eye movements for searching hierarchical computer displays," in SIGCHI conference on Human factors in computing systems, Ft. Lauderdale, Florida, USA, 2003, pp. 249--256. Google ScholarDigital Library
- O. V. Komogortsev, et al., "Eye movement driven usability evaluation via excessive search identification," in 14th International Conference on Human-Computer Interaction, 2011.Google Scholar
- O. Komogortsev, et al., "EMA: Automated eyemovement-driven approach for identification of usability issues," in Design, user experience, and usability. Theory, methods, tools and practice. vol. 6770, A. Marcus, Ed., ed: Springer Berlin / Heidelberg, 2011, pp. 459--468.Google Scholar
- O. Komogortsev, et al., "Aiding usability evaluation via detection of excessive visual search," presented at the 2011 ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), Vancouver, BC, Canada, 2011. Google ScholarDigital Library
- P. M. Fitts, et al., "Eye movements of aircraft pilots during instrument-landing approaches," Aeronautical Engineering Review, vol. 9, pp. 24--29, 1950.Google Scholar
- R. Jacob and K. Karn, "Commentary on Section 4. Eye Tracking in Human-Computer Interaction and Usability Research: Ready to Deliver the Promises," in The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research, ed: Elsevier, 2003, pp. 573--607.Google Scholar
- F. T. W. Au, et al., "Automated usability testing framework," presented at the Proceedings of the ninth conference on Australasian user interface Volume 76, Wollongong, Australia, 2008. Google ScholarDigital Library
- M. Ivory and A. Chevalier, "A Study of Automated Web Site Evaluation Tools," University of Washington, Department of Computer Science2002.Google Scholar
- J. H. Goldberg and X. P. Kotval, "Computer interface evaluation using eye movements: methods and constructs," International Journal of Industrial Ergonomics, vol. 24, pp. 631--645, 1999.Google ScholarCross Ref
- O. V. Komogortsev, et al., "Standardization of Automated Analyses of Oculomotor Fixation and Saccadic Behaviors," IEEE Transactions on Biomedical Engineering, vol. 57, pp. 2635--2645, 2010.Google ScholarCross Ref
- W. Sewell and O. Komogortsev, "Real-time eye gaze tracking with an unmodified commodity webcam employing a neural network," in 28th of the international conference extended abstracts on Human factors in computing systems, Atlanta, Georgia, USA, 2010, pp. 3739--3744. Google ScholarDigital Library
- J. S. Agustin, et al., "Low-cost gaze interaction: ready to deliver the promises," presented at the Proceedings of the 27th international conference extended abstracts on Human factors in computing systems, Boston, MA, USA, 2009. Google ScholarDigital Library
Index Terms
- Identifying usability issues via algorithmic detection of excessive visual search
Recommendations
Aiding usability evaluation via detection of excessive visual search
CHI EA '11: CHI '11 Extended Abstracts on Human Factors in Computing SystemsThis paper presents an objective evaluation of several methods for the automated classification of excessive visual search, a technique which has the potential to aid in the identification of usability problems during software usability testing. ...
Eye Tracking in Human-Computer Interaction and Usability Research
INTERACT '09: Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part IIThe objective of the tutorial is to give an overview on how eye tracking is currently used and how it can be used as a method in human computer interaction research and especially in usability research. An eye tracking system records how the eyes move ...
Eye Tracking Methodology in Screen-based Usability Testing
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing SystemsEye tracking is an important tool in usability testing of a screen-based user interface. Though eye tracking has been used in usability testing for quite a while, challenges remain. For example, how to accurately calibrate gaze point? How to interpret a ...
Comments