Runlong Ye
Pan Chen
Angela Zavaleta Bernuy
Joseph Jay Williams
ABSTRACT
Post-secondary institutions have experienced a continuous trend of student procrastination on course work, thus leading to lower academic performance and potentially worse knowledge retention and other longer-term impacts. Sending reminders about deliverables is a simple approach, but it has the potential to be a valuable tool to mitigate such issues and assist students with time management. This paper will introduce specific processes of conducting such experiments, especially email designs and randomization, to help instructors and researchers conduct similar experiments or field-deploying reminders. To evaluate homework reminder messages, we designed and deployed a real-world randomized A/B experiment at a North American university in a CS1 course where students were randomly assigned to either receive these reminder messages or not. Our findings suggest that students who received the reminder messages have a higher homework completion rate (p < .05) and performed significantly better (p < .01) on the following midterm test than students who did not receive the reminder message. Finally, we discuss how a homework reminder can improve student behaviours, as well as how this type of reminder can be enhanced for future interventions.
- Safia Afzal and Humaira Jami. 2019. Prevalence of Academic Procrastination and Reasons for Academic Procrastination in University Students. Journal of Behavioural Sciences 28, 1 (2019), 51–69.Google Scholar
- Mojeed Kolawole Akinsola, Adedeji Tella, and Adeyinka Tella. 2007. Correlates of academic procrastination and mathematics achievement of university undergraduate students. Eurasia Journal of Mathematics, science and technology education 3, 4(2007), 363–370.Google Scholar
- Albert Bandura. 1997. Self-efficacy: the exercise of control. New York: W.H. Freeman.Google Scholar
- Allison Banko. 2014. Email Marketing: Dell lifts revenue 109% via GIF-centric campaign. https://www.marketingsherpa.com/article/case-study/gif-centric-email-campaign.Google Scholar
- Brett L Beck, Susan R Koons, and Debra L Milgrim. 2000. Correlates and consequences of behavioral procrastination: The effects of academic procrastination, self-consciousness, self-esteem and self-handicapping. Journal of social behavior and personality 15, 5 (2000), 3.Google Scholar
- Angela Zavaleta Bernuy, Qi Yin Zheng, Hammad Shaikh, Andrew Peterson, and Joseph Jay Williams. 2021. Investigating the Impact of Online Homework Reminders Using Randomized A/B Comparisons. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. ACM, Virtual Event, USA, 921–927.Google ScholarDigital Library
- Linda L Caldwell, Edward A Smith, Linda M Collins, John W Graham, Mary Lai, Lisa Wegner, Tania Vergnani, Catherine Matthews, and Joachim Jacobs. 2012. Translational research in South Africa: Evaluating implementation quality using a factorial design. In Child & youth care forum, Vol. 41. Springer, 119–136.Google Scholar
- Rebeca Cerezo, Miguel Sánchez-Santillán, M Puerto Paule-Ruiz, and J Carlos Núñez. 2016. Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education. Computers & Education 96 (2016), 42–54.Google ScholarDigital Library
- Ching-Wen Chang, Beth Hurst, and Annice McLean. 2015. You’ve Got Mail: Student Preferences of Instructor Communication in Online Courses in an Age of Advancing Technologies. Journal of Educational Technology Development and Exchange 8, 1(2015), 1–20.Google ScholarCross Ref
- Linda M Collins, John J Dziak, and Runze Li. 2009. Design of experiments with multiple independent variables: a resource management perspective on complete and reduced factorial designs.Psychological methods 14, 3 (2009), 202.Google Scholar
- Sophie Cormac, Laurence A. Eagle, and Mark Davies. 2020. A large-scale test of the relationship between procrastination and performance using learning analytics. In Assessment & Evaluation in Higher Education 45(1). 1–14.Google Scholar
- Hermann Ebbinghaus. 1913. Memory: a contribution to experimental psychology. Teachers College Press. https://doi.org/10.1037/10011-000Google Scholar
- Stephen H Edwards, Joshua Martin, and Clfford A Shaffer. 2015. Examining classroom interventions to reduce procrastination. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education. 254–259.Google ScholarDigital Library
- Stephen H. Edwards, Joshua Martin, and Clfford A. Shaffer. 2015. Examining Classroom Interventions to Reduce Procrastination. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education. Association for Computing Machinery, New York, NY, USA, 254–259.Google ScholarDigital Library
- GetResponse. [n.d.]. Email marketing benchmarks (2020) by GetResponse. http://www.getresponse.com/resources/reports/email-marketing-benchmarksGoogle Scholar
- Ezatollah Ghadampour and Kobra Beiranvand. 2019. Effect of cognitive and metacognitive learning strategies training on academic procrastination and self efficacy in students. Advances in Cognitive Science 21, 3 (2019), 31–41.Google ScholarCross Ref
- Peter Gollwitzer. 1999. Implementation intentions: Strong effects of simple plans. American Psychologist 54, 7 (1999), 493–503.Google ScholarCross Ref
- Jeffery C Huffman, Ariana M Albanese, Kirsti A Campbell, Christopher M Celano, Rachel A Millstein, Carol A Mastromauro, Brian C Healy, Wei-Jean Chung, James L Januzzi, Linda M Collins, 2017. The Positive Emotions after Acute Coronary Events behavioral health intervention: Design, rationale, and preliminary feasibility of a factorial design study. Clinical trials 14, 2 (2017), 128–139.Google Scholar
- Jr. William Humphrey, Debbie Laverie, and Alison Shields. 2019. Exploring the Effects of Encouraging Student Performance with Text Assignment Reminders. Journal of Marketing Education 0, 0 (2019).Google Scholar
- Kari C Kugler, David L Wyrick, Milroy Tanner, Amanda E., Jeffrey John, Brittany D. Chambers, Alice Ma, Kate M. Guastaferro, and Linda M Collins. 2018. Using the Multiphase Optimization Strategy (MOST) to Develop an Optimized Online STI Preventive Intervention Aimed at College Students: Description of Conceptual Model and Iterative Approach to Optimization: Advanced Topics. Springer, New York, NY USA.Google Scholar
- Jørgen Melgaard, Rubina Monir, Lester Allan Lasrado, and Asle Fagerstrøm. 2022. Academic Procrastination and Online Learning During the COVID-19 Pandemic. Procedia computer science 196 (2022), 117–124.Google Scholar
- Nicolas Michinov, Sophie Brunot, Olivier Le Bohec, Jacques Juhel, and Marine Delaval. 2011. Procrastination, participation, and performance in online learning environments. Computers & Education 56, 1 (2011), 243–252.Google ScholarDigital Library
- William Kirkman O’Brien. 2002. Applying the transtheoretical model to academic procrastination. In Dissertation Abstracts International: Section B: The Sciences and Engineering, 62(11-B).Google Scholar
- Anthony J Onwuegbuzie. 2004. Academic procrastination and statistics anxiety. Assessment & Evaluation in Higher Education 29, 1 (2004), 3–19.Google ScholarCross Ref
- Megan E Piper, Michael C Fiore, Stevens S Smith, David Fraser, Daniel M Bolt, Linda M Collins, Robin Mermelstein, Tanya R Schlam, Jessica W Cook, Douglas E Jorenby, 2016. Identifying effective intervention components for smoking cessation: a factorial screening experiment. Addiction 111, 1 (2016), 129–141.Google ScholarCross Ref
- Timothy A. Pychyl, Jonathan M. Lee, Rachelle Thibodeau, and Allan Blunt. 2000. Five days of emotion: An experience sampling study of undergraduate student procrastination. Journal of Social Behavior & Personality 15, 5 (2000), 239–254.Google Scholar
- Navdeep Sahni, Christian S. Wheeler, and Pradeep Chintagunta. 2018. Personalization in Email Marketing: The Role of Noninformative Advertising Content. Marketing Science 37, 2 (2018), 177–311.Google ScholarDigital Library
- Navdeep S Sahni, S Christian Wheeler, and Pradeep Chintagunta. 2018. Personalization in email marketing: The role of noninformative advertising content. Marketing Science 37, 2 (2018), 236–258.Google ScholarDigital Library
- Vivian C Sheer and Timothy K Fung. 2007. Can email communication enhance professor-student relationship and student evaluation of professor?: Some empirical evidence. Journal of Educational Computing Research 37, 3 (2007), 289–306.Google ScholarCross Ref
- Dan Siroker and Pete Koomen. 2013. A/B testing: The most powerful way to turn clicks into customers. John Wiley & Sons.Google ScholarDigital Library
- Ben O. Smith, Dustin R. White, Patricia C. Kuzyk, and James E. Tierney. 2018. Improved Grade Outcomes with an E-mailed “Grade Nudge”. The Journal of Economic Education 49, 4 (2018), 1–7.Google ScholarCross Ref
- Laura J. Solomon and Esther D. Rothblum. 1984. Academic procrastination: Frequency and cognitive-behavioral correlates.Journal of Counseling Psychology 31, 4 (1984), 503–509. https://doi.org/10.1037/0022-0167.31.4.503Google Scholar
- Piers Steel. 2007. The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure. Psychological Bulletin 133, 1 (2007), 65–94.Google ScholarCross Ref
- Piers Steel. 2007. The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure.Psychological Bulletin 133, 1 (2007), 65–94. https://doi.org/10.1037/0033-2909.133.1.65Google Scholar
- Claudia Szabo, Nickolas Falkner, Antti Knutas, and Mohsen Dorochi. 2018. Understanding the Effects of Lecturer Intervention on Computer Science Student Behaviour. In Proceedings of the 2017 ITiCSE Conference on Working Group Reports. Association for Computing Machinery, New York, NY, USA, 105–124.Google ScholarDigital Library
- Dianne M Tice and Roy F Baumeister. 1997. Longitudinal study of procrastination, performance, stress, and health: The costs and benefits of dawdling. Psychological science 8, 6 (1997), 454–458.Google Scholar
- Navid Hashemi Tonekaboni, Sahar Voghoei, and Delaram Yazdansepas. 2020. How Personalized Feedback Emails Can Enhance Participation. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. ACM, Portland, OR, USA, 1376.Google ScholarDigital Library
- E Sophía Valenzuela-Gálvez, Alvaro Garrido-Morgado, and Óscar González-Benito. 2022. Boost your email marketing campaign! Emojis as visual stimuli to influence customer engagement. JOURNAL OF RESEARCH IN INTERACTIVE MARKETING (2022).Google Scholar
- Wendelien Van Eerde. 2003. A meta-analytically derived nomological network of procrastination. Personality and individual differences 35, 6 (2003), 1401–1418.Google Scholar
- Sahar Voghoei, Navid Hashemi Tonekaboni, Delaram Yazdansepas, Saber Soley-mani, Abolfazl Farahani, and Hamid R. Arabnia. 2018. Personalized Feedback Emails: A Case Study on Online Introductory Computer Science Courses. In Proceedings of the 2020 ACM Southeast Conference. Association for Computing Machinery, New York, NY, USA, 18–25.Google Scholar
- Christopher Watson and Frederick Li. 2014. Failure rates in introductory programming revisited. In Proceedings of the 2014 conference on Innovation & technology in computer science education. Association for Computing Machinery, New York, NY, USA, 39–44.Google ScholarDigital Library
- Joseph C Wesley. 1994. Effects of ability, high school achievement, and procrastinatory behavior on college performance. Educational and Psychological Measurement 54, 2 (1994), 404–408.Google ScholarCross Ref
- M. Betul Yilmaz. 2017. The Relation between Academic Procrastination of Uni- versity Students and Their Assignment and Exam Performances: The Situation in Distance and Face-to-Face Learning Environments. Journal of Education and Training Studies 5, 9 (2017), 146–157.Google ScholarCross Ref
- M Betul Yilmaz. 2017. The Relation between Academic Procrastination of University Students and Their Assignment and Exam Performances: The Situation in Distance and Face-to-Face Learning Environments.Journal of Education and Training Studies 5, 9 (2017), 146–157.Google ScholarCross Ref
- Angela Zavaleta Bernuy, Ziwen Han, Hammad Shaikh, Qi Yin Zheng, Lisa-Angelique Lim, Anna Rafferty, Andrew Petersen, and Joseph Jay Williams. 2022. How can Email Interventions Increase Students’ Completion of Online Homework? A Case Study Using A/B Comparisons. In LAK22: 12th International Learning Analytics and Knowledge Conference. 107–118.Google ScholarDigital Library
Index Terms
- Behavioral Consequences of Reminder Emails on Students’ Academic Performance: a Real-world Deployment
Recommendations
Elderly mental model of reminder system
APCHI '12: Proceedings of the 10th asia pacific conference on Computer human interactionThe growing numbers of elderly is inevitable. As we get older, we will experience some memory declines, thus an assistive technology such as reminder system is recommended. However, the uptake of reminder system is still low. Many researchers from the ...
Designing a smartpen reminder system for older adults
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing SystemsDesigning interactive systems for older adults often means designing with older adults from the earliest stages of development. This paper describes the co-design of a smartpen and paper calendar-based reminder system for the home. The design sessions ...
Students' behavioural engagement in reviewing their tele-consultation feedback within an online clinical communication skills platform
AbstractThe benefit of reviewing personal feedback to students' learning of clinical communication skills is well researched. Less is known about the factors that related to students' engagement in reviewing non-compulsory online feedback, and ...
Highlights- Students showing higher engagement achieved higher assessment results.
- More ...
Comments