Abstract
Rain Classroom, as a mobile app designed and developed by Tsinghua University, has been catching increasingly intense attention since her birth in 2016. Her usability has, however, seldom been explored. The purpose of this study is to compare the usability of Rain Classroom assisted learning with other learning approaches. A mixed design was adopted to collect both quantitative and qualitative data. Via pre- and post-questionnaires, we collected quantitative data. Through analyzing the data, we concluded that the usability of Rain Classroom was significantly higher than that of the traditional learning in terms of eight attributes: effectiveness, efficiency, satisfaction, learnability, memorability, errors, cognitive load and timeliness. Via the semi-structured interviews, we collected qualitative data, based on which the findings are generally consistent with those from questionnaires. Future research may focus on development of more advanced mobile apps, between which the usability comparison may be conducted assisted with interdisciplinary cooperation such as statistics, education, psychology and computer technology.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Al-Fahad, F. N. (2009). Students’ attitudes and perceptions towards the effectiveness of mobile learning in king Saud university, Saudi Arabia. Turkish Online Journal of Educational Technology, 8(2), 9
Ardito, C., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., Rossano, V., & Tersigni, M et al. (2004). Usability of e-learning tools. In Proceedings of the working conference on Advanced visual interfaces (pp. 80–84). ACM.
Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability Evaluation in Industry, 189(194), 4–7
Burkes, K. M. E. (2007). Applying cognitive load theory to the design of online learning. Dissertation Abstracts International, 68–08, Section: A, p. 3354. Adviser: Jeff M. Allen.
Chiou, W. C., Lin, C. C., Perng, C., & Tsai, J. T. (2009). E-Learning usability measurement-using technology acceptance model and usability test. In 15th International conference on Industry, engineering, & management systems (IEMS) (pp. 9–11).
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillside. Lawrence Earlbaum Associates.
Costabile, M. F., Marsico, M. D., Lanzilotti, R., Plantamura, V. L., & Roselli, T. (2005). On the Usability Evaluation of E-Learning Applications. In Hawaii international conference on system sciences. IEEE Computer Society.
De Villiers, R. (2004). Usability evaluation of an e-learning tutorial: Criteria, questions and case study. In Proceedings of the 2004 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries (pp. 284–291). South African Institute for Computer Scientists and Information Technologists.
Ellis, T., & Hafner, W. (2008). Building a framework to support project-based collaborative learning experiences in an asynchronous learning network. Interdisciplinary Journal of E-learning and Learning Objects, 4(1), 167–190
Fozdar, B. I., & Kumar, L. S. (2007). Mobile learning and student retention. International Review of Research in Open and Distance Learning, 8(2), 1–18
Georgieva, E. S., Smrikarov, A. S., & Georgiev, T. S. (2011). Evaluation of mobile learning system. Procedia Computer Science, 3, 632–637
Guillen-Gamez, F. D., Mayorga-Fernandez, M. J., Bravo-Agapito, J., & Escribano-Ortiz, D. (2020). Analysis of teachers’ pedagogical digital competence: Identification of factors predicting their acquisition. Technology, Knowledge, and Learning. https://doi.org/10.1007/s10758-019-09432-7
Han, M. (2018). The blending teaching of ideological and political course in colleges and universities based on MOOC and rain classroom-taking the course of “introduction to Mao Zedong thought and the theoretical system of socialism with Chinese characteristics” as an example. Modern Educational Technology, 28(7), 65–70
Harrison, R., Flood, D., & Duce, D. (2013). Usability of mobile applications: Literature review and rationale for a new usability model. Journal of Interaction Science, 1(1), 1–16
ISO 9241. (1997). Ergonomics Requirements for Office Work with Visual Display Terminals (VDTs) International Standards Organization, Geneva.
Kiget, N. K., Wanyembi, G., & Peters, A. I. (2014). Evaluating usability of e-learning systems in universities. International Journal of Advanced Computer Science and Applications (IJACSA). https://doi.org/10.14569/IJACSA.2014.050815
Lan, Y. F., & Sie, Y. S. (2010). Using RSS to support mobile learning based on media richness theory. Computers & Education, 55(2), 723–732
Li, X. M., Zhang, C. P., & Yuan, B. (2017). An Experimental study on the application of rain classroom-elucidating the fundamental rules of the connectivism learning theory. Modern Educational Technology, 27(5), 40–45. https://doi.org/10.3969/j.issn.1009-8097.2017.05.006
Li, X., & Song, S. (2017). Mobile technology affordance and its social implications: A case of “Rain Classroom.” British Journal of Educational Technology, 49(2), 276–291. https://doi.org/10.1111/bjet.12586
Liao, Y. D., & Ding, Y. F. (2018). Exploration and practice of teaching design under rain classroom mode: A case study of undergraduate architectural english. Educational Modernization, 5(32), 254–256
Lippert, A., Shubeck, K., Morgan, B., Hampton, A., & Graesser, A. (2019). Multiple agent designs in conversational intelligent tutoring systems. Technology, Knowledge, and Learning. https://doi.org/10.1007/s10758-019-09431-8
Lou, R. J., Li, Z., Peng, B. P., Du, J. F., & Luo, L. Z. (2018). Application of rain classroom in college english classroom teaching. Education for Chinese After-school (Theory), 21, 107–108
Motiwalla, L. F. (2007). Mobile learning: A framework and evaluation. Computers & Education, 49(3), 581–596
Nacenta, M. A., Kamber, Y., Qiang, Y., & Kristensson, P. O. (2013). Memorability of pre-designed and user-defined gesture sets. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1099–1108). ACM.
Nielsen, J. (1994). Usability engineering. Elsevier. https://doi.org/10.1016/C2009-0-21512-1
Nielsen, J., & Budiu, R. (2013). Mobile usability. MITP-Verlags GmbH & Co.
Othman, M. K. (2012). Measuring visitors’ experiences with mobile guide technology in cultural spaces. University of York.
Pallant, J. (2013). SPSS survival manual. McGraw-Hill Education.
Parsazadeh, N., Ali, R., & Rezaei, M. (2018). A framework for cooperative and interactive mobile learning to improve online information evaluation skills. Computers & Education, 120, 75–89
Paas, F., & van Merrienboer, J. (1994). Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6, 51–71
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38, 1–4
Qureshi, K., & Irfan, M. (2009). Usability evaluation of e-learning applications, A case study of It's Learning from a student's perspective. Master Thesis of Blekinge Institute of Technology.
Reeves, T. C., Benson, L., Elliott, D., Grant, M., Holschuh, D., Kim, B., Kim, H., Lauber, E., & Loh, S. (2002). Usability and instructional design heuristics for e-learning evaluation. In Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002 (pp. 1615–1621).
Salvucci, D. D. (2001). Predicting the effects of in-car interface use on driver performance: An integrated model approach. International Journal of Human-Computer Studies, 55(1), 85–107
Santosa, P. I. (2009). Usability of E-learning portal and how it affects students' attitude and satisfaction, an exploratory study. PACIS 2009 Proceedings71.
Schildbach, B., & Rukzio, E. (2010). Investigating selection and reading performance on a mobile phone while walking. In Proceedings of the 12th international conference on human computer interaction with mobile devices and services. Lisbon, Portugal: ACM. 2010.
Seufert, S., Meier, C., Soellner, M., & Rietsche, R. (2019). A pedagogical perspective on big data and learning analytics: A conceptual model for digital learning support. Technology, Knowledge, and Learning, 24(4), 599–619. https://doi.org/10.1007/s10758-019-09399-5
Shih, J. L., Chuang, C. W., & Hwang, G. J. (2010). An inquiry-based mobile learning approach to enhancing social science learning effectiveness. Journal of Educational Technology and Society, 13(4), 50–62
Sun, P. C., Tsai, R. J., Finger, G., Chen, Y. Y., & Yeh, D. (2008). What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50(4), 1183–1202
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–285
Tan, T. H., & Liu, T. Y. (2004). The mobile-based interactive learning environment (MOBILE) and a case study for assisting elementary school english learning. Advanced learning technologies, 2004. In Proceedings. IEEE International conference on (pp. 530–534). IEEE.
Wang, S. G. (2017). Rain classroom: An intelligent teaching tool in the context of mobile internet and big data. Modern Educational Technology, 27(5), 26–32
Wang, Y. S. (2003). Assessment of learner satisfaction with asynchronous electronic learning systems. Information Management, 41(1), 75–86
Wang, M., Shen, R., Novak, D., & Pan, X. (2009). The impact of mobile learning on students’ learning behaviours and performance: Report from a large blended classroom. British Journal of Educational Technology, 40(4), 673–695
Wendeson, S., Fatimah, W., Ahmad, B. W., Samiha, N., & Haron, B. (2010). University students awareness on m-learning. Waset Org.
Windell, D., & Wiebe, E. (2007). Measuring cognitive load in multimedia instruction: A comparison of two instruments. Annual meeting of the American educational research association.
Wixom, B. H., & Todd, P. A. (2005). A theoretical integration of user satisfaction and technology acceptance. Information Systems Research, 16(1), 85–102
Xu, J. D., Benbasat, I., & Cenfetelli, R. T. (2013). Integrating service quality with system and information quality: An empirical test in the e-service context. MIS Quarterly, 37(3), 777–794
Yang, F., Zhang, H. R., & Zhang, W. X. (2017). A Study on the Blended Learning based on MOOC and Rain Classroom-taking the teaching practice of “conversational english skills” MOOC and rain classroom as an example. Modern Educational Technology, 27(5), 33–39
Yang, G. B., & Yuan, G. H. (2016). Application of Rain Classroom to “Oral Developmental Biology” for graduates. Electronic Journal of General Dentistry, 3(24), 1-2+4
Yao, J., & Wang, W. L. (2017). Application of blended learning of WeChat and rain classroom to higher education. Higher Education Exploration, 9, 50–54
Yu, H. T. (2018). A five-step intelligent pedagogical approach in colleges and universities based on rain classroom-a case of “application of network education.” Modern Educational Technology, 28(9), 54–58
Yu, Z., Zhu, Y., Yang, Z., & Chen, W. (2019). Student satisfaction, learning outcomes, and cognitive loads with a mobile learning platform. Computer Assisted Language Learning, 32(4), 323–341. https://doi.org/10.1080/09588221.2018.1517093
Yu, Z. (2020). Visualizing co-citations of technology acceptance models in education. Journal of Information Technology Research, 13(1), 77–95. https://doi.org/10.4018/JITR.2020010106
Yuan, B., Zhao, H. M., Zhang, C. P., & Li, X. M. (2018). The visual analysis of postgraduate’ english learning behavior based on rain classroom. Modern Educational Technology, 28(5), 68–74. https://doi.org/10.3969/j.issn.1009-8097.2018.05.010
Zeng, R. X. (2016). An online conference was held by school online to announce the launch of a smart teaching tool-rain classroom. Asia-Pacific Education, 24, 3
Zhu, L. Y. (2016). Teaching of business english translation based on rain classroom. Teaching of Forestry Region, 12, 47–48
Acknowledgements
MOOCs of Beijing Language and Culture University (Important) “An introduction to Linguistics” in 2019 (MOOC201902); An online and offline hybrid course “Introduction to Linguistics” of Beijing Language and Culture University in 2020; The research and reform fund of the “Undergraduate Teaching Reform and Innovation Project” of Beijing higher education in 2020—innovative “multilingual +” excellent talent training system (202010032003).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Human or Animal Rights
All participants are voluntary to participate in the study and the study complies with ethical standards.
Informed Consent
The research obtained consent and the data were analyzed anonymously. The research has been approved by the authors’ institutional review board–Beijing Language and Culture University, which waived the need for written informed consent from the participants.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yu, Z., Yu, L. Identifying Tertiary Students’ Perception of Usabilities of Rain Classroom. Tech Know Learn 27, 1215–1235 (2022). https://doi.org/10.1007/s10758-021-09524-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10758-021-09524-3