skip to main content
research-article

Can Synchronous Code Editing and Awareness Tools Support Remote Tutoring? Effects on Learning and Teaching

Published: 04 October 2023 Publication History

Abstract

One-on-one tutoring is effective for learning computer science since a tutor can work alongside a student and provide tailored feedback on their code. However, translating this type of instruction to a remote setting is challenging. Traditional methods such as screensharing lack key pedagogical functionality and most available tools are designed for collaboration rather than instruction. To identify tools that can support remote tutoring, we conducted an experiment to assess two resources: synchronous editing and awareness tools. In our study, an instructor teaches a learner introductory programming concepts remotely, collaborating through screensharing alone, a shared notebook with real-time collaborative editing, or a shared notebook with additional awareness tools overlaid. To embed the awareness tools, we designed a Chrome extension that enables real-time sharing of gaze and cursor data. Our results show that synchronous editing combined with awareness tools significantly improved learning. The awareness tools also helped tutors better communicate with the student, track their understanding, and establish a sense of presence. As a final contribution, we also assessed the efficacy of gaze-sharing using a webcam eye-tracker. While the accuracy was not as precise as a dedicated sensor, instructors described instances when the gaze was useful for gauging student attention and establishing presence. We discuss implications for remote tutoring in computer science and scaling awareness technology.

References

[1]
Rohana Abdul Karim, Nor Farizan Zakaria, Mohd Asyraf Zulkifley, Mohd Marzuki Mustafa, Ismail Sagap, and Nani Harlina Md Latar. 2013. Telepointer technology in telemedicine: a review. Biomed. Eng. Online, Vol. 12 (March 2013), 21. https://doi.org/10.1186/1475--925X-12--21
[2]
Deepak Akkil and Poika Isokoski. 2016. Gaze Augmentation in Egocentric Video Improves Awareness of Intention. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 1573--1584. https://doi.org/10.1145/2858036.2858127
[3]
Deepak Akkil and Poika Isokoski. 2019. Comparison of Gaze and Mouse Pointers for Video-based Collaborative Physical Task. Interact. Comput., Vol. 30, 6 (Feb. 2019), 524--542. https://doi.org/10.1093/iwc/iwy026
[4]
A H Anderson, E G Bard, C Sotillo, A Newlands, and G Doherty-Sneddon. 1997. Limited visual control of the intelligibility of speech in face-to-face dialogue. Percept. Psychophys., Vol. 59, 4 (May 1997), 580--592. https://doi.org/10.3758/bf03211866
[5]
Ellen Gurman Bard, Robin L Hill, Mary Ellen Foster, and Manabu Arai. 2014. Tuning accessibility of referring expressions in situated dialogue. Language, Cognition and Neuroscience, Vol. 29, 8 (Sept. 2014), 928--949. https://doi.org/10.1080/23273798.2014.895845
[6]
M Bauer, G Kortuem, and Z Segall. 1999. "Where are you pointing at?" A study of remote collaboration in a wearable videoconference system. In Digest of Papers. Third International Symposium on Wearable Computers. ieeexplore.ieee.org, 151--158. https://doi.org/10.1109/ISWC.1999.806696
[7]
Roman Bednarik and Markku Tukiainen. 2005. Effects of display blurring on the behavior of novices and experts during program debugging. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (Portland, OR, USA) (CHI EA '05). Association for Computing Machinery, New York, NY, USA, 1204--1207. https://doi.org/10.1145/1056808.1056877
[8]
Benjamin S Bloom. 1984. The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Educ. Res., Vol. 13, 6 (June 1984), 4--16. https://doi.org/10.3102/0013189X013006004
[9]
Kristy Elizabeth Boyer, August A Dwight, R Taylor Fondren, Mladen A Vouk, and James C Lester. 2008. A development environment for distributed synchronous collaborative programming. SIGCSE bull., Vol. 40, 3 (Aug. 2008), 158--162. https://doi.org/10.1145/1597849.1384315
[10]
Kristy Elizabeth Boyer, Robert Phillips, Michael D Wallis, Mladen A Vouk, and James C Lester. 2009. Investigating the role of student motivation in computer science education through one-on-one tutoring. Computer Science Education, Vol. 19, 2 (June 2009), 111--135. https://doi.org/10.1080/08993400902937584
[11]
Susan E Brennan, Xin Chen, Christopher A Dickinson, Mark B Neider, and Gregory J Zelinsky. 2008. Coordinating cognition: the costs and benefits of shared gaze during collaborative search. Cognition, Vol. 106, 3 (March 2008), 1465--1477. https://doi.org/10.1016/j.cognition.2007.05.012
[12]
Shiwei Cheng, Jialing Wang, Xiaoquan Shen, Yijian Chen, and Anind Dey. 2021. Collaborative eye tracking based code review through real-time shared gaze visualization. Frontiers of Computer Science, Vol. 16, 3 (Nov. 2021), 163704. https://doi.org/10.1007/s11704-020-0422--1
[13]
Michelene T H Chi, Stephanie A Siler, Heisawn Jeong, Takashi Yamauchi, and Robert G Hausmann. 2001. Learning from human tutoring. Cogn. Sci., Vol. 25, 4 (July 2001), 471--533. https://doi.org/10.1207/s15516709cog2504_1
[14]
Herbert H Clark and Susan E Brennan. 1991. Grounding in communication. In Perspectives on socially shared cognition, (pp, Lauren B Resnick (Ed.). Vol. 429. American Psychological Association, xiii, Washington, DC, US, 127--149. https://doi.org/10.1037/10096-006
[15]
Software Freedom Conservancy. 2023. Git. https://git-scm.com/
[16]
Bernardo José da Silva Estácio and Rafael Prikladnicki. 2015. Distributed Pair Programming: A Systematic Literature Review. Information and Software Technology, Vol. 63 (July 2015), 1--10. https://doi.org/10.1016/j.infsof.2015.02.011
[17]
S D'Angelo and A Begel. 2017. Improving communication between pair programmers using shared gaze awareness. Proceedings of the 2017 CHI conference on (2017).
[18]
Sarah D'Angelo and Darren Gergle. 2016. Gazed and Confused: Understanding and Designing Shared Gaze for Remote Collaboration. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 2492--2496. https://doi.org/10.1145/2858036.2858499
[19]
S D'angelo and B Schneider. 2021. Shared gaze visualizations in collaborative interactions: past, present and future. Interact. Comput. (2021).
[20]
Deepnote. 2023. Deepnote. https://deepnote.com/
[21]
Qing Ding and Sitan Cao. 2017. RECT: A Cloud-Based Learning Tool for Graduate Software Engineering Practice Courses With Remote Tutor Support. IEEE Access, Vol. 5 (2017), 2262--2271. https://doi.org/10.1109/ACCESS.2017.2664070
[22]
P Dourish and V Bellotti. 1992. Awareness and coordination in shared workspaces. conference on Computer-supported cooperative ? (1992).
[23]
Selina N Emhardt, Ellen M Kok, Halszka Jarodzka, Saskia Brand-Gruwel, Christian Drumm, and Tamara van Gog. 2020. How Experts Adapt Their Gaze Behavior When Modeling a Task to Novices. Cogn. Sci., Vol. 44, 9 (Sept. 2020), e12893. https://doi.org/10.1111/cogs.12893
[24]
Hongfei Fan and Chengzheng Sun. 2012. Achieving integrated consistency maintenance and awareness in real-time collaborative programming environments: The CoEclipse approach. In Proceedings of the 2012 IEEE 16th International Conference on Computer Supported Cooperative Work in Design (CSCWD). ieeexplore.ieee.org, 94--101. https://doi.org/10.1109/CSCWD.2012.6221803
[25]
Hongfei Fan, Chengzheng Sun, and Haifeng Shen. 2012. ATCoPE: any-time collaborative programming environment for seamless integration of real-time and non-real-time teamwork in software development. In Proceedings of the 17th ACM international conference on Supporting group work (Sanibel Island, Florida, USA) (GROUP '12). Association for Computing Machinery, New York, NY, USA, 107--116. https://doi.org/10.1145/2389176.2389194
[26]
Amber N Finn and Paul Schrodt. 2012. Students' Perceived Understanding Mediates the Effects of Teacher Clarity and Nonverbal Immediacy on Learner Empowerment. Commun. Educ., Vol. 61, 2 (April 2012), 111--130. https://doi.org/10.1080/03634523.2012.656669
[27]
Brandi N Frisby and Matthew M Martin. 2010. Instructor--Student and Student--Student Rapport in the Classroom. Commun. Educ., Vol. 59, 2 (April 2010), 146--164. https://doi.org/10.1080/03634520903564362
[28]
Susan R Fussell, Leslie D Setlock, Jie Yang, Jiazhi Ou, Elizabeth Mauer, and Adam D I Kramer. 2004. Gestures over video streams to support remote collaboration on physical tasks. Hum.-Comput. Interact., Vol. 19, 3 (Sept. 2004), 273--309. https://doi.org/10.1207/s15327051hci1903_3
[29]
GazeRecorder. 2023. GazeCloud API. https://gazerecorder.com/gazecloudapi/
[30]
Max Goldman, Greg Little, and Robert C Miller. 2011. Real-time collaborative coding in a web IDE. In Proceedings of the 24th annual ACM symposium on User interface software and technology (Santa Barbara, California, USA) (UIST '11). Association for Computing Machinery, New York, NY, USA, 155--164. https://doi.org/10.1145/2047196.2047215
[31]
Jamie Gorson and Eleanor O'Rourke. 2019. How Do Students Talk About Intelligence? An Investigation of Motivation, Self-efficacy, and Mindsets in Computer Science. In Proceedings of the 2019 ACM Conference on International Computing Education Research (Toronto ON, Canada) (ICER '19). Association for Computing Machinery, New York, NY, USA, 21--29. https://doi.org/10.1145/3291279.3339413
[32]
Jamie Gorson and Eleanor O'Rourke. 2020. Why do CS1 Students Think They're Bad at Programming? Investigating Self-efficacy and Self-assessments at Three Universities. In Proceedings of the 2020 ACM Conference on International Computing Education Research (Virtual Event, New Zealand) (ICER '20). Association for Computing Machinery, New York, NY, USA, 170--181. https://doi.org/10.1145/3372782.3406273
[33]
Neil A Granitz, Stephen K Koernig, and Katrin R Harich. 2009. Now It's Personal: Antecedents and Outcomes of Rapport Between Business Faculty and Their Students. Journal of Marketing Education, Vol. 31, 1 (April 2009), 52--65. https://doi.org/10.1177/0273475308326408
[34]
Philip J Guo, Jeffery White, and Renan Zanelatto. 2015. Codechella: Multi-user program visualizations for real-time tutoring and collaborative learning. In 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (Atlanta, GA). IEEE. https://doi.org/10.1109/vlhcc.2015.7357201
[35]
Kunal Gupta, Gun A Lee, and Mark Billinghurst. 2016. Do You See What I See? The Effect of Gaze Tracking on Task Space Remote Collaboration. IEEE Trans. Vis. Comput. Graph., Vol. 22, 11 (Nov. 2016), 2413--2422. https://doi.org/10.1109/TVCG.2016.2593778
[36]
Mark Guzdial. 2014. Limitations of MOOCs for Computing Education- Addressing our needs: MOOCs and technology to advance learning and learning research (Ubiquity symposium). Ubiquity, Vol. 2014, July (July 2014), 1--9. https://doi.org/10.1145/2591683
[37]
Richard R Hake. 1998. Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. Am. J. Phys., Vol. 66, 1 (1998), 64--74.
[38]
Lile Hattori and Michele Lanza. 2010. Syde: a tool for collaborative software development. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2 (Cape Town, South Africa) (ICSE '10). Association for Computing Machinery, New York, NY, USA, 235--238. https://doi.org/10.1145/1810295.1810339
[39]
Sublime HQ. 2023. SublimeText. https://www.sublimetext.com/
[40]
Jeroen Janssen, Gijsbert Erkens, and Paul A Kirschner. 2011. Group awareness tools: It's what you do with it that matters. Comput. Human Behav., Vol. 27, 3 (May 2011), 1046--1058. https://doi.org/10.1016/j.chb.2010.06.002
[41]
J Kawash, J Horacsek, and N Wong. 2021. What We Learned from the Abrupt Switch to Online Teaching Due to the COVID-19 Pandemic in a Post-secondary Computer Science Program. CSEDU (2) (2021).
[42]
Karel Kreijns, Monique Bijker, and Joshua Weidlich. 2020. A Rasch Analysis Approach to the Development and Validation of a Social Presence Measure. In Rasch Measurement: Applications in Quantitative Educational Research, Myint Swe Khine (Ed.). Springer Singapore, Singapore, 197--221. https://doi.org/10.1007/978--981--15--1800--3_11
[43]
Carmen Lacave, M Angeles Garc'ia, Ana I Molina, Santiago Sánchez, Miguel A Redondo, and Manuel Ortega. 2019. COLLECE-2.0: A real-time collaborative programming system on Eclipse. In 2019 International Symposium on Computers in Education (SIIE). 1--6. https://doi.org/10.1109/SIIE48397.2019.8970132
[44]
Xiao Liu and Gyun Woo. 2021. CodeHelper: A Web-Based Lightweight IDE for E-Mentoring in Online Programming Courses. In 2021 3rd International Conference on Computer Communication and the Internet (ICCCI). 220--224. https://doi.org/10.1109/ICCCI51764.2021.9486772
[45]
Microsoft. 2023. Visual Studio Code. https://code.visualstudio.com/
[46]
Ana Isabel Molina, Jesús Gallardo, Miguel Ángel Redondo, and Crescencio Bravo. 2014. Evaluating the awareness support of COLLECE, a collaborative programming tool. In Proceedings of the XV International Conference on Human Computer Interaction (Puerto de la Cruz, Tenerife, Spain) (Interacción '14, Article 11). Association for Computing Machinery, New York, NY, USA, 1--2. https://doi.org/10.1145/2662253.2662264
[47]
Neema Moraveji, Robb Lindgren, and Roy Pea. 2009. Organized Mischief: Comparing shared and private displays on a collaborative learning task. (June 2009).
[48]
Eva Ostertagová, Oskar Ostertag, and Jozef Kováv c. 2014. Methodology and application of the Kruskal-Wallis test. Appl. Mech. Mater., Vol. 611 (Aug. 2014), 115--120. https://doi.org/10.4028/www.scientific.net/amm.611.115
[49]
Yizhou Qian and James Lehman. 2017. Students' misconceptions and other difficulties in introductory programming. ACM trans. comput. educ., Vol. 18, 1 (Dec. 2017), 1--24. https://doi.org/10.1145/3077618
[50]
Inc. Replit. 2023. Replit. https://replit.com/
[51]
Riese and Kann. 2020. Teaching assistants' experiences of tutoring and assessing in computer science education. IEEE Frontiers in Education Conference (FIE ? (2020).
[52]
Emma Riese and Viggo Kann. 2020. Teaching assistants' experiences of tutoring and assessing in computer science education. In 2020 IEEE Frontiers in Education Conference (FIE). IEEE, 1--9.
[53]
Stephan Salinger, Christopher Oezbek, Karl Beecher, and Julia Schenk. 2010. Saros: an eclipse plug-in for distributed party programming. In Proceedings of the 2010 ICSE Workshop on Cooperative and Human Aspects of Software Engineering (Cape Town, South Africa) (CHASE '10). Association for Computing Machinery, New York, NY, USA, 48--55. https://doi.org/10.1145/1833310.1833319
[54]
Bertrand Schneider and Roy Pea. 2013. Real-time mutual gaze perception enhances collaborative learning and collaboration quality. International Journal of Computer-Supported Collaborative Learning, Vol. 8, 4 (Dec. 2013), 375--397. https://doi.org/10.1007/s11412-013--9181--4
[55]
Schoolhouse. 2023. Schoolhouse. https://schoolhouse.world/
[56]
Azin Semsar, Hannah McGowan, Yuanyuan Feng, H Reza Zahiri, Adrian Park, Andrea Kleinsmith, and Helena Mentis. 2019. How Trainees Use the Information from Telepointers in Remote Instruction. Proc. ACM Hum.-Comput. Interact., Vol. 3, CSCW (Nov. 2019), 1--20. https://doi.org/10.1145/3359195
[57]
Haifeng Shen and Chengzheng Sun. 2000. RECIPE: a prototype for Internet-based real-time collaborative programming. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=6689d673adabe5fed215ee5b27aacb41e37b1d60. Accessed: 2022--12--30.
[58]
Oleg v Spakov, Diederick Niehorster, Howell Istance, Kari-Jouko R"aih"a, and Harri Siirtola. 2019. Two-Way Gaze Sharing in Remote Teaching. In Human-Computer Interaction -- INTERACT 2019. Springer International Publishing, 242--251. https://doi.org/10.1007/978--3-030--29384-0_16
[59]
Danaë Stanton, Helen Neale, Victor Bayon, and N Rd. 2002. Interfaces to support children's co-present collaboration: multiple mice and tangible technologies. In CSCL, Vol. 2. books.google.com, 342--351.
[60]
Randy Stein and Susan E Brennan. 2004. Another person's eye gaze as a cue in solving programming problems. In Proceedings of the 6th international conference on Multimodal interfaces (State College, PA, USA) (ICMI '04). Association for Computing Machinery, New York, NY, USA, 9--15. https://doi.org/10.1145/1027933.1027936
[61]
Shari L Stockero. 2014. Transitions in Prospective Mathematics Teacher Noticing. In Research Trends in Mathematics Teacher Education, Jane-Jane Lo, Keith R Leatham, and Laura R Van Zoest (Eds.). Springer International Publishing, Cham, 239--259. https://doi.org/10.1007/978--3--319-02562--9_13
[62]
Gahyun Sung, Tianyi Feng, and Bertrand Schneider. 2021. Learners Learn More and Instructors Track Better with Real-time Gaze Sharing. Proc. ACM Hum.-Comput. Interact., Vol. 5, CSCW1 (April 2021), 1--23. https://doi.org/10.1145/3449208
[63]
Alaaeddin Swidan, Felienne Hermans, and Marileen Smit. 2018. Programming Misconceptions for School Students. In Proceedings of the 2018 ACM Conference on International Computing Education Research (Espoo, Finland) (ICER '18). Association for Computing Machinery, New York, NY, USA, 151--159. https://doi.org/10.1145/3230977.3230995
[64]
Udacity. 2023. Udacity. https://www.udacity.com/
[65]
Jason Vandeventer and Benjamin Barbour. 2012. CodeWave: a real-time, collaborative IDE for enhanced learning in computer science. In Proceedings of the 43rd ACM technical symposium on Computer Science Education (Raleigh, North Carolina, USA) (SIGCSE '12). Association for Computing Machinery, New York, NY, USA, 75--80. https://doi.org/10.1145/2157136.2157160
[66]
April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019. How Data Scientists Use Computational Notebooks for Real-Time Collaboration. Proc. ACM Hum.-Comput. Interact., Vol. 3, CSCW (Nov. 2019), 1--30. https://doi.org/10.1145/3359141
[67]
Jeremy Warner and Philip J Guo. 2017. CodePilot: Scaffolding End-to-End Collaborative Software Development for Novice Programmers. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 1136--1141. https://doi.org/10.1145/3025453.3025876
[68]
Stephanie Yang, Allan Jiang, and Bertrand Schneider. 2022. Supporting Remote Programming Instruction With Real-Time Collaboration and Awareness Tools. In Proceedings of the 16th International Conference of the Learning Sciences-ICLS 2022, pp. 1349--1352. International Society of the Learning Sciences.
[69]
Nancy Yao, Jeff Brewer, Sarah D'Angelo, Mike Horn, and Darren Gergle. 2018. Visualizing Gaze Information from Multiple Students to Support Remote Instruction. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA '18, Paper LBW051). Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3170427.3188453
[70]
Kimberly Michelle Ying and Kristy Elizabeth Boyer. 2020. Understanding Students' Needs for Better Collaborative Coding Tools. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--8. https://doi.org/10.1145/3334480.3383068
[71]
Yanxia Zhang, Ken Pfeuffer, Ming Ki Chong, Jason Alexander, Andreas Bulling, and Hans Gellersen. 2017. Look together: using gaze for assisting co-located collaborative search. Pers. Ubiquit. Comput., Vol. 21, 1 (Feb. 2017), 173--186. https://doi.org/10.1007/s00779-016-0969-x
[72]
Inc. Zoom Video Communications. 2023. Zoom. https://zoom.us/

Cited By

View all
  • (2024)GazeMolVR: Sharing Eye-Gaze Cues in a Collaborative VR Environment for Molecular VisualizationProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701599(7-23)Online publication date: 1-Dec-2024
  • (2024)Predicting Code Comprehension: A Novel Approach to Align Human Gaze with Code using Deep Neural NetworksProceedings of the ACM on Software Engineering10.1145/36607951:FSE(1982-2004)Online publication date: 12-Jul-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 7, Issue CSCW2
CSCW
October 2023
4055 pages
EISSN:2573-0142
DOI:10.1145/3626953
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 October 2023
Published in PACMHCI Volume 7, Issue CSCW2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. awareness tools
  2. gaze visualization
  3. synchronous code editing

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)112
  • Downloads (Last 6 weeks)5
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)GazeMolVR: Sharing Eye-Gaze Cues in a Collaborative VR Environment for Molecular VisualizationProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701599(7-23)Online publication date: 1-Dec-2024
  • (2024)Predicting Code Comprehension: A Novel Approach to Align Human Gaze with Code using Deep Neural NetworksProceedings of the ACM on Software Engineering10.1145/36607951:FSE(1982-2004)Online publication date: 12-Jul-2024

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media