research-article

Analysing the API learning process through the use of eye tracking

Authors:

Sören Sparmann,

Carsten SchulteAuthors Info & Claims

ETRA '23: Proceedings of the 2023 Symposium on Eye Tracking Research and Applications

Article No.: 86, Pages 1 - 6

https://doi.org/10.1145/3588015.3589194

Published: 30 May 2023 Publication History

Abstract

We conducted an exploratory study in which participants had to acquire knowledge of unfamiliar application programming interfaces (APIs) in order to complete two programming tasks. Eye tracking was used to monitor participants’ attention throughout the study. We analysed the learning process using the COIL model, which includes three stages: Information Collection, Information Organisation and Solution Testing. Using this model, we describe patterns in the sequences of actions derived from the data. We discuss whether the Solution Testing stage has two distinct functions: constructing the solution and gathering information simultaneously. This would be an exploratory Solution Testing Stage. In addition, the use of eye tracking provided further insight into the API learning process.

References

[1]

Richard Andersson, Linnea Larsson, Kenneth Holmqvist, Martin Stridh, and Marcus Nyström. 2017. One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms. Behavior Research Methods 49, 2 (April 2017), 616–637. https://doi.org/10.3758/s13428-016-0738-9

[2]

Benjamin T. Carter and Steven G. Luke. 2020. Best practices in eye tracking research. International Journal of Psychophysiology 155 (2020), 49–62. https://doi.org/10.1016/j.ijpsycho.2020.05.010

[3]

Ekwa Duala-Ekoko and Martin P Robillard. 2010. The information gathering strategies of API learners. Technical Report. Technical report, TR-2010.6, School of Computer Science, McGill University.

[4]

Ekwa Duala-Ekoko and Martin P. Robillard. 2012. Asking and answering questions about unfamiliar APIs: An exploratory study. In 2012 34th International Conference on Software Engineering (ICSE). IEEE, Zurich, 266–276. https://doi.org/10.1109/ICSE.2012.6227187

[5]

Andrew T. Duchowski. 2017. Eye tracking methodology: theory and practice (third edition ed.). Springer, Cham.

[6]

Janet Feigenspan, Christian Kastner, Jorg Liebig, Sven Apel, and Stefan Hanenberg. 2012. Measuring programming experience. In 2012 20th IEEE International Conference on Program Comprehension (ICPC). IEEE, Passau, Germany, 73–82. https://doi.org/10.1109/ICPC.2012.6240511

[7]

Gao Gao, Finn Voichick, Michelle Ichinco, and Caitlin L. Kelleher. 2020. Exploring Programmers’ API Learning Processes: Collecting Web Resources as External Memory. 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (2020), 1–10.

[8]

Max Goldman and Robert C. Miller. 2009. Codetrail: Connecting source code and web resources. Journal of Visual Languages & Computing 20, 4 (Aug. 2009), 223–235. https://doi.org/10.1016/j.jvlc.2009.04.003

Digital Library

[9]

Drew T. Guarnera, Corey A. Bryant, Ashwin Mishra, Jonathan I. Maletic, and Bonita Sharif. 2018. iTrace: eye tracking infrastructure for development environments. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications. ACM, Warsaw Poland, 1–3. https://doi.org/10.1145/3204493.3208343

Digital Library

[10]

Ava Heinonen. 2023. Component Comprehension in Context. ACM SIGSOFT Software Engineering Notes 48, 1 (Jan. 2023), 44–48. https://doi.org/10.1145/3573074.3573086

Digital Library

[11]

Daqing Hou and Lin Li. 2011. Obstacles in Using Frameworks and APIs: An Exploratory Study of Programmers’ Newsgroup Discussions. In 2011 IEEE 19th International Conference on Program Comprehension. 91–100. https://doi.org/10.1109/ICPC.2011.21 ISSN: 1092-8138.

Digital Library

[12]

Margaret Jean Intons-Peterson and George L. Newsome. 1992. External memory aids: Effects and effectiveness. In Memory improvement: Implications for memory theory, Douglas J. Herrmann, Herbert Weingartner, Alan Searleman, and Cathy McEvoy (Eds.). Springer New York, New York, NY, 101–121. https://doi.org/10.1007/978-1-4612-2760-1_7

[13]

Caitlin Kelleher and Michelle Ichinco. 2019. Towards a Model of API Learning. In 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, Memphis, TN, USA, 163–168. https://doi.org/10.1109/VLHCC.2019.8818850

[14]

Amy J. Ko and Yann Riche. 2011. The role of conceptual knowledge in API usability. In 2011 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, Pittsburgh, PA, USA, 173–176. https://doi.org/10.1109/VLHCC.2011.6070395

[15]

Essi Lahtinen, Kirsti Ala-Mutka, and Hannu-Matti Järvinen. 2005. A study of the difficulties of novice programmers. Acm sigcse bulletin 37, 3 (2005), 14–18. Publisher: ACM New York, NY, USA.

[16]

Brad A Myers and Jeffrey Stylos. 2016. Improving API usability. Commun. ACM 59, 6 (2016), 62–69.

Digital Library

[17]

Peter Pirolli and Stuart Card. 1995. Information foraging in information access environments. In Proceedings of the SIGCHI conference on Human factors in computing systems. 51–58.

Digital Library

[18]

Martin P. Robillard. 2009. What Makes APIs Hard to Learn? Answers from Developers. IEEE Software 26, 6 (Nov. 2009), 27–34. https://doi.org/10.1109/MS.2009.193 Conference Name: IEEE Software.

Digital Library

[19]

Martin P. Robillard and Robert DeLine. 2011. A field study of API learning obstacles. Empirical Software Engineering 16, 6 (Dec. 2011), 703–732. https://doi.org/10.1007/s10664-010-9150-8

Digital Library

[20]

Dario D. Salvucci and Joseph H. Goldberg. 2000. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the symposium on Eye tracking research & applications - ETRA ’00. ACM Press, Palm Beach Gardens, Florida, United States, 71–78. https://doi.org/10.1145/355017.355028

Digital Library

[21]

Noah L. Schroeder and Ada T. Cenkci. 2018. Spatial Contiguity and Spatial Split-Attention Effects in Multimedia Learning Environments: a Meta-Analysis. Educational Psychology Review 30, 3 (Sept. 2018), 679–701. https://doi.org/10.1007/s10648-018-9435-9

[22]

Janet Siegmund, Christian Kästner, Jörg Liebig, Sven Apel, and Stefan Hanenberg. 2014. Measuring and modeling programming experience. Empirical Software Engineering 19, 5 (Oct. 2014), 1299–1334. https://doi.org/10.1007/s10664-013-9286-4

Digital Library

[23]

John Sweller, Jeroen JG Van Merrienboer, and Fred GWC Paas. 1998. Cognitive architecture and instructional design. Educational psychology review (1998), 251–296.

[24]

Kyle Thayer, Sarah E. Chasins, and Amy J. Ko. 2021. A Theory of Robust API Knowledge. ACM Transactions on Computing Education 21, 1 (March 2021), 1–32. https://doi.org/10.1145/3444945

Digital Library

[25]

Finn Voichick, Gao Gao, Michelle Ichinco, and Caitlin Kelleher. 2019. Towards Validation of a Model of API Learning. In 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (Memphis, TN, USA, 2019-10). IEEE, 267–269. https://doi.org/10.1109/VLHCC.2019.8818795

[26]

Xin Xia, Lingfeng Bao, David Lo, Pavneet Singh Kochhar, Ahmed E. Hassan, and Zhenchang Xing. 2017. What do developers search for on the web?Empirical Software Engineering 22, 6 (Dec. 2017), 3149–3185. https://doi.org/10.1007/s10664-017-9514-4

Digital Library

Cited By

Sparmann SHüsing SSchulte C(2023)JuGaze: A Cell-based Eye Tracking and Logging Tool for Jupyter NotebooksProceedings of the 23rd Koli Calling International Conference on Computing Education Research10.1145/3631802.3631824(1-11)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3631802.3631824

Index Terms

Analysing the API learning process through the use of eye tracking
1. Information systems
  1. World Wide Web
    1. Web searching and information discovery
2. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
        Information technology education

Recommendations

Analysing EOG signal features for the discrimination of eye movements with wearable devices
PETMEI '11: Proceedings of the 1st international workshop on pervasive eye tracking & mobile eye-based interaction

Eye tracking research in human-computer interaction and experimental psychology traditionally focuses on stationary devices and a small number of common eye movements. The advent of pervasive eye tracking promises new applications, such as eye-based ...
An eye tracking dataset for point of gaze detection
ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

This paper presents a new, publicly available eye tracking dataset, aimed to be used as a benchmark for Point of Gaze (PoG) detection algorithms. The dataset consists of a set of videos recording the eye motion of human test subjects as they were ...
An Affordable Solution for Binocular Eye Tracking and Calibration in Head-mounted Displays
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Immersion is the ultimate goal of head-mounted displays (HMD) for Virtual Reality (VR) in order to produce a convincing user experience. Two important aspects in this context are motion sickness, often due to imprecise calibration, and the integration ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '23: Proceedings of the 2023 Symposium on Eye Tracking Research and Applications

May 2023

441 pages

ISBN:9798400701504

DOI:10.1145/3588015

Editors:
Enkelejda Kasneci
Technical University Munich, Germany
,
Frederick Shic
University of Washington, USA
,
Mohamed Khamis
University of Glasgow, Scotland

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 May 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ETRA '23

Sponsor:

ETRA '23: 2023 Symposium on Eye Tracking Research and Applications

May 30 - June 2, 2023

Tubingen, Germany

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

ETRA '25

Sponsor:
sigchi
sigchi

2025 Symposium on Eye Tracking Research and Applications

May 26 - 29, 2025

Tokyo , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
96
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sparmann SHüsing SSchulte C(2023)JuGaze: A Cell-based Eye Tracking and Logging Tool for Jupyter NotebooksProceedings of the 23rd Koli Calling International Conference on Computing Education Research10.1145/3631802.3631824(1-11)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3631802.3631824

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten