skip to main content
10.1145/3303772.3303790acmotherconferencesArticle/Chapter ViewAbstractPublication PageslakConference Proceedingsconference-collections
research-article

On multi-device use: Using technological modality profiles to explain differences in students' learning

Published: 04 March 2019 Publication History

Abstract

With increasing abundance and ubiquity of mobile phones, desktop PCs, and tablets in the last decade, we are seeing students intermixing these modalities to learn and regulate their learning. However, the role of these modalities in educational settings is still largely under-researched. Similarly, little attention has been paid to the research on the extension of learning analytics to analyze the learning processes of students adopting various modalities during a learning activity. Traditionally, research on how modalities affect the way in which activities are completed has mainly relied upon self-reported data or mere counts of access from each modality. We explore the use of technological modalities in regulating learning via learning management systems (LMS) in the context of blended courses. We used data mining techniques to analyze patterns in sequences of actions performed by learners (n = 120) across different modalities in order to identify technological modality profiles of sequences. These profiles were used to detect the technological modality strategies adopted by students. We found a moderate effect size (∈2 = 0.12) of students' adopted strategies on the final course grade. Furthermore, when looking specifically at online discussion engagement and performance, students' adopted technological modality strategies explained a large amount of variance (η2 = 0.68) in their engagement and quality of contributions. The result implications and further research are discussed.

References

[1]
2012. ECAR study of undergraduate students and information technology. https://library.educause.edu/resources/2012/9/ecar-study-of-\undergraduate-students-and-information-technology-2012
[2]
Mohamed Ally and Avgoustos Tsinakos. 2014. Increasing access through mobile learning.
[3]
James H Bray, Scott E Maxwell, and Scott E Maxwell. 1985. Multivariate analysis of variance. Number 54. Sage.
[4]
Guy Brock, Vasyl Pihur, Susmita Datta, Somnath Datta, et al. 2011. clValid, an R package for cluster validation. Journal of Statistical Software (Brock et al., March 2008) (2011).
[5]
D Christopher Brooks. 2016. ECAR study of undergraduate students and information technology. Technical Report. 2016.
[6]
Geraldine Clarebout, Jan Elen, Norma A Juarez Collazo, Griet Lust, and Lai Jiang. 2013. Metacognition and the use of tools. In International handbook of metacognition and learning technologies. Springer, 187--195.
[7]
Jacob Cohen. 1992. A power primer. Psychological bulletin 112, 1 (1992), 155.
[8]
Simon Cross, Mike Sharples, and Graham Healing. 2016. Learning with mobile devices: the changing place and space of distance learners' study. (2016).
[9]
Eden Dahlstrom, JD Walker, and Charles Dziuban. 2013. ECAR study of undergraduate students and information technology. Technical Report. 2013.
[10]
Joseph C Dunn. 1974. Well-separated clusters and optimal fuzzy partitions. Journal of cybernetics 4, 1 (1974), 95--104.
[11]
Rachel H Ellaway, Patricia Fink, Lisa Graves, and Alanna Campbell. 2014. Left to their own devices: medical learnersâĂŹ use of mobile technologies. Medical teacher 36, 2 (2014), 130--138.
[12]
Helen Farley, Angela Murphy, Chris Johnson, Brad Carter, Michael Lane, Warren Midgley, Abdul Hafeez-Baig, Stijn Dekeyser, and Andy Koronios. 2015. How Do Students Use Their Mobile Devices to Support Learning? A Case Study from an Australian Regional University. Journal of Interactive Media in Education 2015, 1 (2015).
[13]
Oliver Edmund Fincham, Dragan V Gasevic, Jelena M Jovanovic, and Abelardo Pardo. 2018. From Study Tactics to Learning Strategies: An Analytical Method for Extracting Interpretable Representations. IEEE Transactions on Learning Technologies (2018).
[14]
Alexis Gabadinho, Gilbert Ritschard, Nicolas Séverin Mueller, and Matthias Studer. 2011. Analyzing and visualizing state sequences in R with TraMineR. Journal of Statistical Software 40, 4 (2011), 1--37.
[15]
Dragan Gašević, Shane Dawson, Tim Rogers, and Danijela Gasevic. 2016. Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education 28 (2016), 68--84.
[16]
Gene V Glass, Percy D Peckham, and James R Sanders. 1972. Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Review of educational research 42, 3 (1972), 237--288.
[17]
Arthur C Graesser, Danielle S McNamara, and Jonna M Kulikowich. 2011. Coh-Metrix: Providing multilevel analyses of text characteristics. Educational researcher 40, 5 (2011), 223--234.
[18]
Arthur C Graesser, Danielle S McNamara, Max M Louwerse, and Zhiqiang Cai. 2004. Coh-Metrix: Analysis of text on cohesion and language. Behavior research methods, instruments, & computers 36, 2 (2004), 193--202.
[19]
Houston Heflin, Jennifer Shewmaker, and Jessica Nguyen. 2017. Impact of mobile technology on student attitudes, engagement, and learning. Computers & Education 107 (2017), 91--99.
[20]
Whitney Hess. 2015. Location Agnostic, Context Specific. https://whitneyhess.com/blog/2012/02/07/location-agnostic-context-specific/
[21]
Matissa Hollister. 2009. Is optimal matching suboptimal? Sociological Methods & Research 38, 2 (2009), 235--264.
[22]
Lai Jiang, Jan Elen, and Geraldine Clarebout. 2008. Learner Variables, Tool-Usage Behaviour and Performance in an Open Learning Environment. In Beyond Knowledge: The Legacy of Competence. Springer, 257--266.
[23]
Srećko Joksimović, Nia Dowell, Oleksandra Poquet, Vitomir Kovanović, Dragan Gašević, Shane Dawson, and Arthur C Graesser. 2018. Exploring development of social capital in a CMOOC through language and discourse. The Internet and Higher Education 36 (2018), 54--64.
[24]
Jelena Jovanovic, Dragan Gasevic, Shane Dawson, Abelardo Pardo, and Negin Mirriahi. 2017. Learning analytics to unveil learning strategies in a flipped classroom. 33 (02 2017).
[25]
Leonard Kaufman and Peter Rousseeuw. 1990. Finding Groups in Data: An Introduction To Cluster Analysis.
[26]
James Knaub. 1987. Practical Interpretation of Hypothesis Tests - letter to the editor - TAS. 41 (08 1987), 246--247.
[27]
Jasper Knight. 2010. Distinguishing the learning approaches adopted by undergraduates in their use of online resources. Active Learning in Higher Education 11, 1 (2010), 67--76.
[28]
Vitomir Kovanović, Dragan Gašević, Shane Dawson, Srećko Joksimović, and Ryan Baker. 2016. Does time-on-task estimation matter? Implications on validity of learning analytics findings. Journal of Learning Analytics 2, 3 (2016), 81--110.
[29]
Vitomir Kovanović, Dragan Gašević, Srećko Joksimović, Marek Hatala, and Olusola Adesope. 2015. Analytics of communities of inquiry: Effects of learning technology use on cognitive presence in asynchronous online discussions. The Internet and Higher Education 27 (2015), 74--89.
[30]
Vitomir Kovanović, Srećko Joksimović, Zak Waters, Dragan Gašević, Kirsty Kitto, Marek Hatala, and George Siemens. 2016. Towards automated content analysis of discussion transcripts: A cognitive presence case. In Proceedings of the sixth international conference on learning analytics & knowledge. ACM, 15--24.
[31]
Greig Emil Krull. 2017. Supporting seamless learning: Students' use of multiple devices in open and distance learning universities. Ph.D. Dissertation. http://openaccess.uoc.edu/webapps/o2/handle/10609/72507
[32]
Yu-Ju Lan, Yao-tin Sung, and Kuo-En Chang. 2007. A mobile-device-supported peer-assisted learning system for collaborative early EFL reading. (2007).
[33]
Griet Lust, Norma A Juarez Collazo, Jan Elen, and Geraldine Clarebout. 2012. Content Management Systems: Enriched learning opportunities for all? Computers in Human Behavior 28, 3 (2012), 795--808.
[34]
Griet Lust, Jan Elen, and Geraldine Clarebout. 2013. Regulation of tool-use within a blended course: Student differences and performance effects. Computers & Education 60, 1 (2013), 385--395.
[35]
Griet Lust, Jan Elen, and Geraldine Clarebout. 2013. StudentsâĂŹ tool-use within a web enhanced course: Explanatory mechanisms of studentsâĂŹ tool-use pattern. Computers in Human Behavior 29, 5 (2013).
[36]
Griet Lust, Mieke Vandewaetere, Eva Ceulemans, Jan Elen, and Geraldine Clarebout. 2011. Tool-use in a blended undergraduate course: In Search of user profiles. Computers & Education 57, 3 (2011), 2135--2144.
[37]
Leah P Macfadyen and Shane Dawson. 2010. Mining LMS data to develop an âĂIJearly warning systemâĂİ for educators: A proof of concept. Computers & education 54, 2 (2010), 588--599.
[38]
Jun Nakahara, Shinichi Hisamatsu, Kazaru Yaegashi, and Yuhei Yamauchi. 2005. iTree: Does the mobile phone encourage learners to be more involved in collaborative learning?. In Proceedings of th 2005 conference on Computer support for collaborative learning: learning 2005: the next 10 years! International Society of the Learning Sciences, 470--478.
[39]
Jeffrey Pomerantz and D. Christopher Brooks. 2017. ECAR Study of Faculty and Information Technology, 2017. https://library.educause.edu/resources/2017/10/ecar-study-of-faculty-and-information-technology-2017
[40]
Peter J Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics 20 (1987), 53--65.
[41]
Richard M Royall. 1986. The effect of sample size on the meaning of significance tests. The American Statistician 40, 4 (1986), 313--315.
[42]
Glenn Stockwell. 2010. Using mobile phones for vocabulary activities: Examining the effect of platform. (2010).
[43]
Glenn Stockwell. 2013. Tracking learner usage of mobile phones for language learning outside of the classroom. CALICO Journal 30, 1 (2013), 118--136.
[44]
Yao-Ting Sung, Kuo-En Chang, Huei-Tse Hou, and Pin-Fu Chen. 2010. Designing an electronic guidebook for learning engagement in a museum of history. Computers in Human Behavior 26, 1 (2010), 74--83.
[45]
Yao-Ting Sung, Kuo-En Chang, and Tzu-Chien Liu. 2016. The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis. Computers & Education 94 (2016), 252--275.
[46]
Bernardo Tabuenca, Marco Kalz, Hendrik Drachsler, and Marcus Specht. 2015. Time will tell: The role of mobile learning analytics in self-regulated learning. Computers & Education 89 (2015), 53--74.
[47]
Katrien Verbert, Sten Govaerts, Erik Duval, Jose Luis Santos, Frans Van Assche, Gonzalo Parra, and Joris Klerkx. 2014. Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing 18, 6 (2014), 1499--1514.
[48]
Philip H Winne. 2006. How software technologies can improve research on learning and bolster school reform. Educational Psychologist 41, 1 (2006), 5--17.
[49]
Gary KW Wong. 2016. A new wave of innovation using mobile learning analytics for flipped classroom. In Mobile Learning Design. Springer, 189--218.
[50]
Jerrold H Zar. 2010. Biostatistical Analysis. Biostatistical Analysis (5th Edition) 9 (2010), 09685.

Cited By

View all
  • (2023)College Students’ Perceptions and Preferences Regarding Intelligent Advisory Systems in Multi-device Learning EnvironmentsHuman Aspects of IT for the Aged Population10.1007/978-3-031-34866-2_12(154-166)Online publication date: 23-Jul-2023
  • (2021)Conception centrée utilisateur d’un environnement virtuel pour la prise de décision collaborative : état de l’art pluridisciplinaire et analyse des besoins.Proceedings of the 17th “Ergonomie et Informatique Avancée” Conference10.1145/3486812.3486834(1-12)Online publication date: 6-Oct-2021
  • (2021)Study Behavior in Computing Education—A Systematic Literature ReviewACM Transactions on Computing Education10.1145/346912922:1(1-40)Online publication date: 18-Oct-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
LAK19: Proceedings of the 9th International Conference on Learning Analytics & Knowledge
March 2019
565 pages
ISBN:9781450362566
DOI:10.1145/3303772
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Blended learning
  2. Learning analytics
  3. Mobile Learning
  4. Multi-device use
  5. Online discussions
  6. Trace Analysis

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

LAK19

Acceptance Rates

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)27
  • Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2023)College Students’ Perceptions and Preferences Regarding Intelligent Advisory Systems in Multi-device Learning EnvironmentsHuman Aspects of IT for the Aged Population10.1007/978-3-031-34866-2_12(154-166)Online publication date: 23-Jul-2023
  • (2021)Conception centrée utilisateur d’un environnement virtuel pour la prise de décision collaborative : état de l’art pluridisciplinaire et analyse des besoins.Proceedings of the 17th “Ergonomie et Informatique Avancée” Conference10.1145/3486812.3486834(1-12)Online publication date: 6-Oct-2021
  • (2021)Study Behavior in Computing Education—A Systematic Literature ReviewACM Transactions on Computing Education10.1145/346912922:1(1-40)Online publication date: 18-Oct-2021
  • (2021)Video Consumption with Mobile Applications in a Global Enterprise MOOC ContextInnovations in Learning and Technology for the Workplace and Higher Education10.1007/978-3-030-90677-1_5(49-60)Online publication date: 13-Nov-2021
  • (2020)How patterns of students dashboard use are related to their achievement and self-regulatory engagementProceedings of the Tenth International Conference on Learning Analytics & Knowledge10.1145/3375462.3375472(340-349)Online publication date: 23-Mar-2020
  • (2020)Analyzing the consistency in within-activity learning patterns in blended learningProceedings of the Tenth International Conference on Learning Analytics & Knowledge10.1145/3375462.3375470(1-10)Online publication date: 23-Mar-2020
  • (2020)Multi-Device Use: Understanding the Motivations behind Switching between Multiple Devices during a TaskInternational Journal of Human–Computer Interaction10.1080/10447318.2020.172610636:12(1178-1193)Online publication date: 13-Feb-2020
  • (2020)Integration of BYOD Technology in Traditional Classroom: A Statistical ApproachAdvances in Computational Intelligence, Security and Internet of Things10.1007/978-981-15-3666-3_9(98-110)Online publication date: 5-Mar-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media