Skip to main content
SpringerLink
Log in

A practical guide on conducting eye tracking studies in software engineering

  • Experience Reports
  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

For several years, the software engineering research community used eye trackers to study program comprehension, bug localization, pair programming, and other software engineering tasks. Eye trackers provide researchers with insights on software engineers’ cognitive processes, data that can augment those acquired through other means, such as on-line surveys and questionnaires. While there are many ways to take advantage of eye trackers, advancing their use requires defining standards for experimental design, execution, and reporting. We begin by presenting the foundations of eye tracking to provide context and perspective. Based on previous surveys of eye tracking for programming and software engineering tasks and our collective, extensive experience with eye trackers, we discuss when and why researchers should use eye trackers as well as how they should use them. We compile a list of typical use cases—real and anticipated—of eye trackers, as well as metrics, visualizations, and statistical analyses to analyze and report eye-tracking data. We also discuss the pragmatics of eye tracking studies. Finally, we offer lessons learned about using eye trackers to study software engineering tasks. This paper is intended to be a one-stop resource for researchers interested in designing, executing, and reporting eye tracking studies of software engineering tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Notes

  1. https://imotions.com/blog/webcam-eye-tracking-vs-an-eye-tracker/

  2. https://github.com/brains-on-code

  3. http://www.ptidej.net/downloads/replications/

  4. http://www.cogain.org/eye-tracking/

  5. https://github.com/synesthesiam/eyecode

  6. http://www.ogama.net/

  7. http://www.ptidej.net/tools/programcomprehension/

  8. http://www.i-trace.org/

  9. http://www.i-trace.org/features/

  10. http://www.srcml.org/#home

  11. http://github.com/synesthesiam/eyecode

  12. http://github.com/hanav/PandasEye

  13. http://www.pygaze.org/

References

  • Abid NJ, Maletic JI, Sharif B (2019a) Using developer eye movements to externalize the mental model used in code summarization tasks. In: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, ACM, New York, ETRA ’19, pp 13:1–13:9. https://doi.org/10.1145/3314111.3319834

  • Abid NJ, Sharif B, Dragan N, Alrasheed H, Maletic JI (2019b) Developer reading behavior while summarizing java methods : Size and context matters. In: Proceedings of the 41th International Conference on Software Engineering, ACM, New York, ICSE 2019, p To Appear

  • Ali N, Sharafi Z, Guéhéneuc YG, Antoniol G (2015) An empirical study on the importance of source code entities for requirements traceability. Empir Softw Eng 20(2):442–478

    Article  Google Scholar 

  • Alkan S, Cagiltay K (2007) Studying computer game learning experience through eye tracking. Br J Educ Technol 38(3):538–542

    Article  Google Scholar 

  • Armaly A, Rodeghero P, Mcmillan C (2018) Audiohighlight: Code skimming for blind programmers,. In: 2018 IEEE International conference on software maintenance and evolution, ICSME, IEEE, pp 206–216

  • Ayres J, Flannick J, Gehrke J, Yiu T (2002) Sequential pattern mining using a bitmap representation. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, New York KDD ’02, pp 429–435. https://doi.org/10.1145/775047.775109

  • Barik T, Smith J, Lubick K, Holmes E, Feng J, Murphy-Hill E, Parnin C (2017) Do developers read compiler error messages? In: Proceedings of the 39th International Conference on Software Engineering, IEEE Press, Piscataway, ICSE ’17, pp 575–585 . https://doi.org/10.1109/ICSE.2017.59

  • Beatty J (1982) Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychol Bull 91(2):276

    Article  Google Scholar 

  • Bednarik R (2007) Methods to analyze visual attention strategies: Applications in the studies of programming. University of Joensuu

  • Bednarik R (2012) Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations. International Journal of Human-Computer Studies 70(2):143–155. https://doi.org/10.1016/j.ijhcs.2011.09.003

    Article  Google Scholar 

  • Bednarik R, Tukiainen M (2005) Effects of display blurring on the behavior of novices and experts during program debugging. In: CHI ’05 Extended Abstracts on Human Factors in Computing Systems, ACM, New York, CHI EA ’05, pp 1204–1207. https://doi.org/10.1145/1056808.1056877

  • Bednarik R, Tukiainen M (2006) An eye-tracking methodology for characterizing program comprehension processes. In: Proceedings of the 2006 Symposium on Eye Tracking Research & Applications, ACM, New York, NY, USA, ETRA ’06, pp 125–132

  • Bednarik R, Eivazi S, Hradis M (2012) Gaze and conversational engagement in multiparty video conversation: an annotation scheme and classification of high and low levels of engagement. In: Proceedings of the 4th workshop on eye gaze in intelligent human machine interaction, ACM, p 10

  • Begel A, Vrzakova H (2018) Eye movements in code review. In: Proceedings of the Workshop on Eye Movements in Programming, ACM, New York, EMIP ’18, pp 5:1–5:5. https://doi.org/10.1145/3216723.3216727

  • Berg-strom JR, Schall A (2014) Eye tracking in user experience design. Elsevier

  • Binkley D, Davis M, Lawrie D, Maletic JI, Morrell C, Sharif B (2013) The impact of identifier style on effort and comprehension. Empir Softw Eng 18(2):219–276. https://doi.org/10.1007/s10664-012-9201-4

    Article  Google Scholar 

  • Blascheck T, Sharif B (2019) Visually analyzing eye movements on natural language texts and source code snippets. In: ETRA 2019-ACM Symposium on Eye Tracking Research & Applications

  • Blascheck T, Kurzhals K, Raschke M, Burch M, Weiskopf D, Ertl T (2017) Visualization of eye tracking data: a taxonomy and survey. Computer Graphics Forum 36(8):260–284

    Article  Google Scholar 

  • Bojko A (2005) Eye tracking in user experience testing: How to make the most of it. In: Proceedings of the UPA 2005 Conference

  • Bojko AA (2009) Informative or misleading? heatmaps deconstructed. In: Proceedings of the 13th International Conference on Human-Computer Interaction. Part I: New Trends, Springer, Berlin, pp 30–39. https://doi.org/10.1007/978-3-642-02574-7_4

  • Buse RPL, Sadowski C, Weimer W (2011) Benefits and barriers of user evaluation in software engineering research, Object-oriented programming, Systems, Languages and Applications, pp 643–656

  • Busjahn T, Schulte C, Busjahn A (2011) Analysis of code reading to gain more insight in program comprehension. In: Proceedings of the 11th Koli Calling International Conference on Computing Education Research, ACM, New York, Koli Calling ’11, pp 1–9. https://doi.org/10.1145/2094131.2094133

  • Busjahn T, Schulte C, Sharif B, Simon BA, Hansen M, Bednarik R, Orlov P, Ihantola P, Shchekotova G, Antropova M (2014) Eye tracking in computing education. In: Proceedings of the Tenth Annual Conference on International Computing Education Research, ACM, New York, ICER ’14, pp 3–10 . https://doi.org/10.1145/2632320.2632344

  • Busjahn T, Bednarik R, Begel A, Crosby M, Paterson JH, Schulte C, Sharif B, Tamm S (2015) Eye movements in code reading: Relaxing the linear order. In: Proceedings of 22th International Conference on Program Comprehension, ICPC ’15

  • Buxton RB, Uludağ K, Dubowitz DJ, Liu TT (2004) Modeling the hemodynamic response to brain activation. Neuroimage 23:S220–S233

    Article  Google Scholar 

  • Cagiltay NE, Tokdemir G, Kilic O, Topalli D (2013) Performing and analyzing non-formal inspections of entity relationship diagram (erd). J Syst Softw 86 (8):2184–2195. https://doi.org/10.1016/j.jss.2013.03.106

    Article  Google Scholar 

  • Cepeda G, Guéhéneuc YG (2010) An empirical study on the efficiency of different design pattern representations in uml class diagrams. Empir Softw Eng 15 (5):493–522. https://doi.org/10.1007/s10664-009-9125-9

    Article  Google Scholar 

  • Clark B, Sharif B (2017) itracevis: Visualizing eye movement data within Eclipse. In: Working conference on software visualization, VISSOFT, IEEE, pp 22–32

  • Cristino F, Mathot S, Theeuwes J, Gilchrist ID (2010) Scanmatch: a novel method for comparing fixation sequences. Behaviour Res Meth 42:692–700

    Article  Google Scholar 

  • Crosby ME, Stelovsky J (1990) How do we read algorithms? a case study. Computer 23(1):24–35

    Article  Google Scholar 

  • Crosby ME, Scholtz J, Wiedenbeck S (2002) The roles beacons play in comprehension for novice and expert programmers. In: Proceeding of Programmers, 14th Workshop of the Psychology of Programming Interest Group, Brunel University, pp 18–21

  • Dalmaijer ES, Mathôt S, Van der Stigchel S (2014) Pygaze: an open-source, cross-platform toolbox for minimal-effort programming of eyetracking experiments. Behavior Res Meth 46(4):913–921

    Article  Google Scholar 

  • De Smet B, Lempereur L, Sharafi Z, Guéhéneuc YG, Antoniol G, Habra N (2014) Taupe: Visualizing and analyzing eye-tracking data. Sci Comput Program 79:260–278. https://doi.org/10.1016/j.scico.2012.01.004

    Article  Google Scholar 

  • Divjak M, Bischof H (2008) Real-time video-based eye blink analysis for detection of low blink-rate during computer use. In: First international workshop on tracking humans for the evaluation of their motion in image sequences (THEMIS 2008), pp 99-107

  • Duchowski AT (2002) A breadth-first survey of eye-tracking applications. Behavior Research Methods Instruments, & Computers 34(4):455–470

    Article  Google Scholar 

  • Duchowski AT (2007) Eye tracking methodology: Theory and practice. Springer, New York

    MATH  Google Scholar 

  • Fakhoury S, Ma Y, Arnaoudova V, Adesope O (2018) The effect of poor source code lexicon and readability on developers’ cognitive load. In: Proceedings of the 26th Conference on Program Comprehension, ACM, New York, ICPC ’18, pp 286–296. https://doi.org/10.1145/3196321.3196347

  • Farnsworth B (2019a) 10 Free Eye Tracking Software Programs [Pros and Cons]. https://imotions.com/blog/free-eye-tracking-software/, [Online; accessed 30-December-2019]

  • Farnsworth B (2019b) 10 Free Eye Tracking Software Programs [Pros and Cons]. https://imotions.com/blog/free-eye-tracking-software/, [Online; accessed 30-December-2019]

  • Fritz T, Begel A, Müller SC, Yigit-Elliott S, Züger M (2014) Using psycho-physiological measures to assess task difficulty in software development. In: Proceedings of the 36th International Conference on Software Engineering, ACM, New York, ICSE ’14, pp 402–413

  • Godfroid A (2019) Eye tracking in second language acquisition and bilingualism: A research synthesis and methodological guide. Routledge

  • Goldberg JH, Helfman JI (2010) Comparing information graphics: A critical look at eye tracking. In: Proceedings of the 3rd BELIV’10 Workshop: BEyond Time and Errors: Novel evaLuation Methods for Information Visualization, ACM, New York, BELIV ’10, pp 71–78. https://doi.org/10.1145/2110192.2110203

  • Goldberg JH, Kotval XP (1999) Computer interface evaluation using eye movements: methods and constructs. Int J Ind Ergon 24(6):631–645

    Article  Google Scholar 

  • Goldberg JH, Stimson MJ, Lewenstein M, Scott N, Wichansky AM (2002) Eye tracking in web search tasks: Design implications. In: Proceedings of the 2002 Symposium on Eye Tracking Research & Applications, ACM, New York, ETRA ’02, pp 51–58. https://doi.org/10.1145/507072.507082

  • Grace R, Byrne VE, Bierman DM, Legrand JM, Gricourt D, Davis RK, Staszewski JJ, Carnahan B (1998) A drowsy driver detection system for heavy vehicles. In: Digital avionics systems conference, 1998. Proceedings., 17th DASC. The AIAA/IEEE/SAE, vol 2. IEEE, pp I36-1

  • Guarnera DT, Bryant CA, Mishra A, Maletic JI, Sharif B (2018) itrace: eye tracking infrastructure for development environments. In: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications, ACM, p 105

  • Guéhéneuc YG (2006a) Taupe: Towards understanding program comprehension. In: Proceedings of the 2006 Conference of the Center for Advanced Studies on Collaborative Research, IBM Corp., Riverton, CASCON ’06. https://doi.org/10.1145/1188966.1188968

  • Guéhéneuc YG (2006b) Taupe: Towards understanding program comprehension. In: Proceedings of the 2006 Conference of the Center for Advanced Studies on Collaborative Research, IBM Corp., Riverton, NJ, USA, CASCON ’06

  • Haiduc S, Aponte J, Moreno L, Marcus A (2010) On the use of automated text summarization techniques for summarizing source code. In: 2010 17th Working Conference on Reverse Engineering, IEEE, pp 35–44

  • Haji-Abolhassani A, Clark JJ (2014) An inverse yarbus process: Predicting observers’ task from eye movement patterns. Vision Res 103:127–142

    Article  Google Scholar 

  • Hansen DW, Ji Q (2009) In the eye of the beholder: a survey of models for eyes and gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence 32 (3):478–500

    Article  Google Scholar 

  • Hansen M (2014) eyecode: An eye-tracking experimental framework for program comprehension. PhD thesis, School of Informatics and Computing, 2719 E. 10th Street Bloomington, IN 47408 USA

  • Hartridge H, Thomson L (1948) Methods of investigating eye movements. Brit J Ophthalmol 32(9):581

    Article  Google Scholar 

  • Hejmady P, Narayanan NH (2012) Visual attention patterns during program debugging with an IDE. In: Proceedings of the 2012 Symposium on Eye Tracking Research & Applications, ACM, New York, ETRA ’12, pp 197–200. https://doi.org/10.1145/2168556.2168592

  • Henderson JM, Pierce GL (2008) Eye movements during scene viewing: Evidence for mixed control of fixation durations. Psychonomic Bulletin & Review 15(3):566–573

    Article  Google Scholar 

  • Holmqvist K, Nyström M, Andersson R, Dewhurst R, Jarodzka H, Van de Weijer J (2011) Eye tracking: A comprehensive guide to methods and measures. OUP Oxford

  • Holmqvist K, Nyström M, Mulvey F (2012) Eye tracker data quality: what it is and how to measure it. In: Proceedings of the symposium on eye tracking research and applications, ACM, pp 45–52

  • Huey EB (1908) The psychology and pedagogy of reading. The Macmillan Company

  • Jacob RJ, Karn KS (2003) Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. Mind 2(3):4

    Google Scholar 

  • Jbara A, Feitelson DG (2017) How programmers read regular code: a controlled experiment using eye tracking. Empir Softw Eng 22(3):1440–1477

    Article  Google Scholar 

  • Jeanmart S, Guéhéneuc YG, Sahraoui HA, Habra N (2009) Impact of the visitor pattern on program comprehension and maintenance. In: Proceedings of 3rd International Symposium on Empirical Software Engineering and Measurement, pp 69–78

  • Just MA, Carpenter PA (1980) A theory of reading: from eye fixations to comprehension. Psychol Rev 87(4):329

    Article  Google Scholar 

  • Karn KS, Ellis S, Juliano C (1999) The hunt for usability: tracking eye movements. In: CHI’99 extended abstracts on Human factors in computing systems, ACM, pp 173–173

  • Kitchenham BA (2004) Procedures for undertaking systematic reviews. Tech. rep., Joint Technical Report, Computer Science Department, Keele University (TR/SE- 0401) and National ICT Australia Ltd

  • Kitchenham BA, Pfleeger SL, Pickard LM, Jones PW, Hoaglin DC, Emam KE, Rosenberg J (2002) Preliminary guidelines for empirical research in software engineering. IEEE Trans Softw Eng 28(8):721–734. https://doi.org/10.1109/TSE.2002.1027796

    Article  Google Scholar 

  • Ko AJ, Latoza TD, Burnett MM (2015) A practical guide to controlled experiments of software engineering tools with human participants. Empirical Softw Engg 20(1):110–141. https://doi.org/10.1007/s10664-013-9279-3

    Article  Google Scholar 

  • Lee S, Hooshyar D, Ji H, Nam K, Lim H (2018) Mining biometric data to predict programmer expertise and task difficulty. Clust Comput 21(1):1097–1107

    Article  Google Scholar 

  • Levenshtein V (1966) Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady 10:707

    MathSciNet  Google Scholar 

  • Lung J, Aranda J, Easterbrook SM, Wilson GV (2008) On the difficulty of replicating human subjects studies in software engineering. In: Proceedings of the 30th international conference on Software engineering, ACM, pp 191–200

  • Mackworth NH, Thomas EL (1962) Head-mounted eye-marker camera. JOSA 52(6):713–716

    Article  Google Scholar 

  • McChesney I, Bond R (2019) Eye tracking analysis of computer program comprehension in programmers with dyslexia. Empirical Softw Engg 24(3):1109–1154. https://doi.org/10.1007/s10664-018-9649-y

    Article  Google Scholar 

  • Murphy GC, Kersten M, Findlater L (2006) How are java software developers using the eclipse ide? IEEE Softw 23(4):76–83. https://doi.org/10.1109/MS.2006.105

    Article  Google Scholar 

  • Obaidellah U, Al Haek M, Cheng PCH (2018) A survey on the usage of eye-tracking in computer programming. ACM Comput Surv 51(1):5:1–5:58. https://doi.org/10.1145/3145904

    Article  Google Scholar 

  • Olsson P (2007) Real-time and offline filters for eye tracking

  • Orlov PA, Bednarik R (2017) The role of extrafoveal vision in source code comprehension. Perception 46(5):541–565

    Article  Google Scholar 

  • Peitek N, Siegmund J, Apel S, Kästner C, Parnin C, Bethmann A, Leich T, Saake G, Brechmann A (2018a) A look into programmers’ heads. IEEE Trans Softw Eng, pp 1–1

  • Peitek N, Siegmund J, Parnin C, Apel S, Hofmeister J, Brechmann A (2018b) Simultaneous Measurement of Program Comprehension with fMRI and Eye tracking: A Case Study. In: Symposium on Empirical Software Engineering and Measurement, to appear

  • Pernice K, Nielsen J (2009) Eyetracking methodology: How to conduct and evaluate usability studies using eyetracking. Nielsen Norman Group Technical Report

  • Peterson C, Saddler J, Blascheck T, Sharif B (2019) Visually analyzing students’ gaze on c++ code snippets. In: EMIP 2019-6th International Workshop on Eye Movements in Programming

  • Petrusel R, Mendling J (2012) Eye-tracking the factors of process model comprehension tasks. In: Proceedings of the Conference on the Advanced Information Systems Engineering, Springer, CAiSE ’13, pp 224–239

  • Pfleeger SL (1995) Experimental design and analysis in software engineering, part 5: Analyzing the data. SIGSOFT Softw Eng Notes 20 (5):14–17. https://doi.org/10.1145/217030.217032

    Article  Google Scholar 

  • Poole A, Ball LJ (2005) Eye tracking in human-computer interaction and usability research: Current status and future. In: Prospects”, Chapter in C. Ghaoui (Ed.): Encyclopedia of Human-Computer Interaction. Pennsylvania: Idea Group, Inc

  • Privitera CM, Stark LW (2000) Algorithms for defining visual regions-of-interest: Comparison with eye fixations. IEEE Trans Pattern Anal Mach Intell 22:970–982

    Article  Google Scholar 

  • Rayner K (1978) Eye movements in reading and information processing. Psychol Bull 85(3):618–660

    Article  Google Scholar 

  • Rayner K (1998) Eye movements in reading and information processing: 20 years of research. Psychol Bull 124(3):372

    Article  Google Scholar 

  • Rodeghero P, McMillan C, McBurney PW, Bosch N, D’Mello S (2014) Improving automated source code summarization via an eye-tracking study of programmers. In: Proceedings of the 36th International Conference on Software Engineering, ACM, New York, ICSE, 2014, pp 390-401. https://doi.org/10.1145/2568225.2568247

  • Ross J (2009) Eyetracking: Is It Worth It? http://www.uxmatters.com/mt/archives/2009/10/eyetracking-is-it-worth-it.php/, [Online; accessed 20-March-2019]

  • Sajaniemi J (2004) Comparison of three eye tracking devices in psychology of programming research. In: Proceedings of the 16th Annual Psychology of Programming Interest Group Workshop, PPIG ’04, pp 151–158

  • Salvucci DD, Goldberg JH (2000) Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 Symposium on Eye Tracking Research & Applications, ACM, New York, ETRA ’00, pp 71–78. https://doi.org/10.1145/355017.355028

  • Shackel B (1960) Note on mobile eye viewpoint recording. JOSA 50(8):763–768

    Article  Google Scholar 

  • Shaffer TR, Wise JL, Walters BM, Müller SC, Falcone M, Sharif B (2015) itrace: Enabling eye tracking on software artifacts within the ide to support software engineering tasks. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ACM, pp 954–957

  • Shapiro JR, Neuberg SL (2007) From stereotype threat to stereotype threats: Implications of a multi-threat framework for causes, moderators, mediators, consequences, and interventions. Personal Soc Psychol Rev 11(2):107–130

    Article  Google Scholar 

  • Sharafi Z, Soh Z, Guéhéneuc YG, Antoniol G (2012) Women and men - different but equal: On the impact of identifier style on source code reading. In: Proceedings of 20th International Conference on Program Comprehension, ICPC ’13, pp 27–36

  • Sharafi Z, Marchetto A, Susi A, Antoniol G, Guéhéneuc YG (2013) An empirical study on the efficiency of graphical vs.textual representations in requirements comprehension. In: Proceedings of 21st International Conference on Program Comprehension, ICPC ’13, pp 33–42

  • Sharafi Z, Shaffer T, Bonita S, Guéhéneuc YG (2015a) Eye-tracking metrics in software engineering. In: Proceedings of 22nd Asia-Pacific Software Engineering Conference, IEEE CS Press, APSEC ’15

  • Sharafi Z, Soh Z, Guéhéneuc YG (2015b) A systematic literature review on the usage of eye-tracking in software engineering. Information and Software Technology (IST)

  • Sharif B, Maletic JI (2010a) An eye tracking study on camelcase and under_score identifier styles. In: Proceeding of 18th IEEE International Conference on Program Comprehension, IEEE Computer Society, ICPC ’10, pp 196–205

  • Sharif B, Maletic JI (2010b) An eye tracking study on the effects of layout in understanding the role of design patterns. In: Proceedings of the 26th IEEE International Conference on Software Maintenance, IEEE Computer Society, pp 1–10

  • Sharif B, Falcone M, Maletic JI (2012) An eye-tracking study on the role of scan time in finding source code defects. In: Proceedings of the Symposium on Eye Tracking Research & Applications, ACM, New York, ETRA’12, pp 381–384

  • Sharif B, Jetty G, Aponte J, Parra E (2013) An empirical study assessing the effect of seeit 3D on comprehension. In: Proceeding of 1st IEEE Working Conference on Software Visualization, IEEE, VISSOFT ’13, pp 1–10

  • Siegmund J, Siegmund N, Apel S (2015) Views on internal and external validity in empirical software engineering. In: Proceedings of the 37th International Conference on Software Engineering-Volume 1, IEEE Press, pp 9–19

  • Soh Z, Sharafi Z, den Plas BV, Porras GC, Guéhéneuc YG, Antoniol G (2012) Professional status and expertise for UML class diagram comprehension: An empirical study. In: Proceedings of 20th International Conference on Program Comprehension, ICPC ’13, pp 163–172

  • Soh Z, Khomh F, Guéhéneuc YG, Antoniol G, Adams B (2013) On the effect of program exploration on maintenance tasks. In: 2013 20th Working Conference on Reverse Engineering (WCRE), pp 391–400. https://doi.org/10.1109/WCRE.2013.6671314

  • Soh Z, Khomh F, Guéhéneuc YG, Antoniol G (2018) Noise in mylyn interaction traces and its impact on developers and recommendation systems. Empir Softw Eng 23(2):645–692. https://doi.org/10.1007/s10664-017-9529-x

    Article  Google Scholar 

  • Spencer SJ, Steele CM, Quinn DM (1999) Stereotype threat and women’s math performance. J Exp Soc Psychol 35(1):4–28

    Article  Google Scholar 

  • Steele CM, Aronson J (1995) Stereotype threat and the intellectual test performance of african americans. J Pers Soc Psychol 69(5):797

    Article  Google Scholar 

  • Stein R, Brennan SE (2004) Another person’s eye gaze as a cue in solving programming problems. In: Proceedings of the 6th International Conference on Multimodal Interfaces, ACM, New York, ICMI ’04, pp 9–15 https://doi.org/10.1145/1027933.1027936

  • Strandvall T (2009) Eye tracking in human-computer interaction and usability research. In: Gross T, Gulliksen J, Kotzé P, Oestreicher L, Palanque P, Prates R O, Winckler M (eds) Human-Computer Interaction–INTERACT 2009: 12th IFIP TC 13 International Conference, Uppsala, Sweden, August 24-28, 2009, Proceedings, Part II. Springer, Berlin, pp 936–937

  • Sundstedt V (2010) Gazing at games: Using eye tracking to control virtual characters. In: ACM SIGGRAPH 2010 Courses, ACM, New York, SIGGRAPH ’10, pp 5:1–5:160 https://doi.org/10.1145/1837101.1837106

  • Turner R, Falcone M, Sharif B, Lazar A (2014a) An eye-tracking study assessing the comprehension of C++ and Python source code. In: Proceedings of the Symposium on Eye Tracking Research & Applications, ACM, New York, ETRA ’14, pp 231–234

  • Turner R, Falcone M, Sharif B, Lazar A (2014b) An eye-tracking study assessing the comprehension of c++ and python source code. In: Proceedings of the Symposium on Eye Tracking Research and Applications, ACM, New York, ETRA ’14, pp 231–234. https://doi.org/10.1145/2578153.2578218

  • Uwano H, Nakamura M, Monden A, Matsumoto K (2006) Analyzing individual performance of source code review using reviewers’ eye movement. In: Proceedings of the 2006 symposium on Eye tracking research & applications, ACM, ETRA ’06, pp 133–140

  • Vrzakova H (2019) Machine learning methods in interaction inference from gaze. In: Dissertations in Forestry and Natural Sciences, University of Eastern Finland

  • Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer Science & Business Media

  • Yarbus AL (1967) Eye movements during perception of complex objects. Springer

  • Yusuf S, Kagdi HH, Maletic JI (2007) Assessing the comprehension of UML class diagrams via eye tracking. In: Proceeding of 15th IEEE International Conference on Program Comprehension, IEEE Computer Society, ICPC ’07, pp 113–122

  • Zhang Z, Zhang J (2010) A new real-time eye tracking based on nonlinear unscented kalman filter for monitoring driver fatigue. Journal of Control Theory and Applications 8(2):181–188

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their insightful comments and suggestions. This work has been partly funded by the US NSF under Grant Numbers CCF 18-55756 and CCF 15-53573, as well as the NSERC Discovery Grant program and the Canada Research Chair in Software Patterns and Patterns of Software.

Author information

Authors and Affiliations

  1. University of Michigan, Ann Arbor, MI, USA

    Zohreh Sharafi

  2. University of Nebraska–Lincoln, Lincoln, NE, USA

    Bonita Sharif

  3. Concordia University, Montréal, Québec, Canada

    Yann-Gaël Guéhéneuc

  4. Microsoft Research, Redmond, WA, USA

    Andrew Begel

  5. University of Eastern Finland, Joensuu, Finland

    Roman Bednarik

  6. University of Hawai’i at Mānoa, Honolulu, HI, USA

    Martha Crosby

Authors
  1. Zohreh Sharafi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bonita Sharif
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yann-Gaël Guéhéneuc
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrew Begel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Roman Bednarik
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Martha Crosby
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yann-Gaël Guéhéneuc.

Additional information

Communicated by: Denys Poshyvanyk

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sharafi, Z., Sharif, B., Guéhéneuc, YG. et al. A practical guide on conducting eye tracking studies in software engineering. Empir Software Eng 25, 3128–3174 (2020). https://doi.org/10.1007/s10664-020-09829-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-020-09829-4

Keywords

Search

Navigation