Skip to main content
Springer Nature Link
Log in

Which Algorithms Suit Which Learning Environments? A Comparative Study of Recommender Systems in TEL

  • Conference paper
  • First Online:
Adaptive and Adaptable Learning (EC-TEL 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9891))

Included in the following conference series:

Abstract

In recent years, a number of recommendation algorithms have been proposed to help learners find suitable learning resources on-line. Next to user-centered evaluations, offline-datasets have been used to investigate new recommendation algorithms or variations of collaborative filtering approaches. However, a more extensive study comparing a variety of recommendation strategies on multiple TEL datasets is missing. In this work, we contribute with a data-driven study of recommendation strategies in TEL to shed light on their suitability for TEL datasets. To that end, we evaluate six state-of-the-art recommendation algorithms for tag and resource recommendations on six empirical datasets: a dataset from European Schoolnets TravelWell, a dataset from the MACE portal, which features access to meta-data-enriched learning resources from the field of architecture, two datasets from the social bookmarking systems BibSonomy and CiteULike, a MOOC dataset from the KDD challenge 2015, and Aposdle, a small-scale workplace learning dataset. We highlight strengths and shortcomings of the discussed recommendation algorithms and their applicability to the TEL datasets. Our results demonstrate that the performance of the algorithms strongly depends on the properties and characteristics of the particular dataset. However, we also find a strong correlation between the average number of users per resource and the algorithm performance. A tag recommender evaluation experiment reveals that a hybrid combination of a cognitive-inspired and a popularity-based approach consistently performs best on all TEL datasets we utilized in our study.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/learning-layers/TagRec/.

  2. 2.

    http://www.kde.cs.uni-kassel.de/bibsonomy/dumps/.

  3. 3.

    http://www.citeulike.org/faq/data.adp.

  4. 4.

    http://kddcup2015.com/information.html.

  5. 5.

    http://lreforschools.eun.org.

References

  1. Drachsler, H., Verbert, K., Santos, O.C., Manouselis, N.: Panorama of recommender systems to support learning. In: Ricci, F., Rokach, L., Shapira, B. (eds.) Recommender Systems Handbook, pp. 421–451. Springer, Heidelberg (2015)

    Chapter  Google Scholar 

  2. Khribi, M.K., Jemni, M., Nasraoui, O.: Recommendation systems for personalized technology-enhanced learning. In: Kinshuk, Huang, R. (eds.) Ubiquitous Learning Environments and Technologies, pp. 159–180. Springer, Heidelberg (2015)

    Google Scholar 

  3. Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H., Koper, R.: Recommender systems in technology enhanced learning. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 387–415. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  4. Verbert, K., Manouselis, N., Drachsler, H., Duval, E.: Dataset-driven research to support learning and knowledge analytics. Educ. Technol. Soc. 15(3), 133–148 (2012)

    Google Scholar 

  5. Verbert, K., Drachsler, H., Manouselis, N., Wolpers, M., Vuorikari, R., Duval, E.: Dataset-driven research for improving recommender systems for learning. In: Proceedings of LAK 2011, pp. 44–53. ACM (2011)

    Google Scholar 

  6. Fazeli, S., Loni, B., Drachsler, H., Sloep, P.: Which recommender system can best fit social learning platforms? In: Rensing, C., de Freitas, S., Ley, T., Muñoz-Merino, P.J. (eds.) EC-TEL 2014. LNCS, vol. 8719, pp. 84–97. Springer, Heidelberg (2014)

    Google Scholar 

  7. Niemann, K., Wolpers, M.: Usage context-boosted filtering for recommender systems in TEL. In: Hernández-Leo, D., Ley, T., Klamma, R., Harrer, A. (eds.) EC-TEL 2013. LNCS, vol. 8095, pp. 246–259. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  8. Manouselis, N., Vuorikari, R., Van Assche, F.: Collaborative recommendation of e-learning resources: an experimental investigation. J. Comput. Assist. Learn. 26(4), 227–242 (2010)

    Article  Google Scholar 

  9. Bateman, S., Brooks, C., Mccalla, G., Brusilovsky, P.: Applying collaborative tagging to e-learning. In: Proceedings WWW 2007 (2007)

    Google Scholar 

  10. Kuhn, A., McNally, B., Schmoll, S., Cahill, C., Lo, W.-T., Quintana, C., Delen, I.: How students find, evaluate and utilize peer-collected annotated multimedia data in science inquiry with Zydeco. In: Proceedings of SIGCHI 2012, pp. 3061–3070. ACM (2012)

    Google Scholar 

  11. Klašnja-Milićević, A., Ivanović, M., Nanopoulos, A.: Recommender systems in e-learning environments: a survey of the state-of-the-art and possible extensions. Artif. Intell. Rev. 44(4), 571–604 (2015)

    Article  Google Scholar 

  12. Manouselis, N., Drachsler, H., Verbert, K., Duval, E.: Recommender Systems for Learning. Springer, New York (2012)

    Google Scholar 

  13. Erdt, M., Fernandez, A., Rensing, C.: Evaluating recommender systems for technology enhanced learning: a quantitative survey. IEEE Trans. Learn. Technol. 8(4), 326–344 (2015)

    Article  Google Scholar 

  14. Lohmann, S., Thalmann, S., Harrer, A., Maier, R.: Learner-generated annotation of learning resources-lessons from experiments on tagging. J. Univ. Comput. Sci. 304, 312 (2007)

    Google Scholar 

  15. Diaz-Aviles, E., Fisichella, M., Kawase, R., Nejdl, W., Stewart, A.: Unsupervised auto-tagging for learning object enrichment. In: Kloos, C.D., Gillet, D., Crespo García, R.M., Wild, F., Wolpers, M. (eds.) EC-TEL 2011. LNCS, vol. 6964, pp. 83–96. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  16. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  17. Niemann, K.: Automatic tagging of learning objects based on their usage in web portals. In: Conole, G., Klobučar, T., Rensing, C., Konert, J., Lavoué, E. (eds.) Design for Teaching and Learning in a Networked World, vol. 9307, pp. 240–253. Springer, Heidelberg (2015)

    Google Scholar 

  18. Kowald, D., Lex, E.: Evaluating tag recommender algorithms in real-world folksonomies: a comparative study. In: Proceedings of RecSys 2015, pp. 265–268. ACM (2015)

    Google Scholar 

  19. Seitlinger, P., Kowald, D., Kopeinik, S., Hasani-Mavriqi, I., Ley, T., Lex, E.: Attention please! a hybrid resource recommender mimicking attention-interpretation dynamics. In: Proceedings of International World Wide Web Conferences Steering Committee, WWW 2015, pp. 339–345 (2015)

    Google Scholar 

  20. Trattner, C., Kowald, D., Seitlinger, P., Kopeinik, S., Ley, T.: Modeling activation processes in human memory to predict the use of tags in social bookmarking systems. J. Web Sci. 2(1), 1–16 (2016)

    Article  Google Scholar 

  21. Kowald, D., Lacic, E., Trattner, C.: Tagrec: towards a standardized tagrecommender benchmarking framework. In: Proceedings of HT 2014. ACM, New York (2014)

    Google Scholar 

  22. Jäschke, R., Marinho, L., Hotho, A., Schmidt-Thieme, L., Stumme, G.: Tag recommendations in Folksonomies. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 506–514. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  23. Schafer, J.B., Frankowski, D., Herlocker, J., Sen, S.: Collaborative filtering recommender systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) Adaptive Web 2007. LNCS, vol. 4321, pp. 291–324. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  24. Marinho, L.B., Schmidt-Thieme, L.: Collaborative tag recommendations. In: Preisach, C., Burkhardt, H., Schmidt-Thieme, L., Decker, R. (eds.) Data Analysis, Machine Learning and Applications, pp. 533–540. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  25. Gemmell, J., Schimoler, T., Ramezani, M., Christiansen, L., Mobasher, B.: Improving folkrank with item-based collaborative filtering. In: Recommender Systems & the Social Web (2009)

    Google Scholar 

  26. Basilico, J., Hofmann, T.: Unifying collaborative and content-based filtering. In: Proceedings of ICML 2004, p. 9. ACM (2004)

    Google Scholar 

  27. Friedrich, M., Niemann, K., Scheffel, M., Schmitz, H.-C., Wolpers, M.: Object recommendation based on usage context. Educ. Technol. Soc. 10(3), 106–121 (2007)

    Google Scholar 

  28. Niemann, K., Wolpers, M.: Creating usage context-based object similarities to boost recommender systems in technology enhanced learning. IEEE Trans. Learn. Technol. 8(3), 274–285 (2015)

    Article  Google Scholar 

  29. Kowald, D., Kopeinik, S., Seitlinger, P., Ley, T., Albert, D., Trattner, C.: Refining frequency-based tag reuse predictions by means of time and semantic context. In: Atzmueller, M., Chin, A., Scholz, C., Trattner, C. (eds.) MUSE/MSM 2013, LNAI 8940. LNCS, vol. 8940, pp. 55–74. Springer, Heidelberg (2015)

    Google Scholar 

  30. Anderson, J.R., Schooler, L.J.: Reflections of the environment in memory. Psychol. Sci. 2(6), 396–408 (1991)

    Article  Google Scholar 

  31. Love, B.C., Medin, D.L., Gureckis, T.M.: Sustain: a network model of category learning. Psychol. Rev. 111(2), 309 (2004)

    Article  Google Scholar 

  32. Benchmark folksonomy data from bibsonomy, Knowledge and Data Engineering Group. University of Kassel, 2013/2015. http://www.kde.cs.uni-kassel.de/bibsonomy/dumps

  33. Stefaner, M., Dalla Vecchia, E., Condotta, M., Wolpers, M., Specht, M., Apelt, S., Duval, E.: MACE – enriching architectural learning objects for experience multiplication. In: Duval, E., Klamma, R., Wolpers, M. (eds.) EC-TEL 2007. LNCS, vol. 4753, pp. 322–336. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  34. Vuorikari, R., Massart, D.: Datatel challenge: European schoolnet’s travel well dataset. In: Proceedings of RecSysTEL 2010 (2010)

    Google Scholar 

  35. Beham, G., Stern, H., Lindstaedt, S.: Aposdle-ds a dataset from the Aposdle work integrated learning system. In: Proceedings of RecSysTEL 2010 (2010)

    Google Scholar 

  36. Marinho, L.B., Hotho, A., Jäschke, R., Nanopoulos, A., Rendle, S., Schmidt-Thieme, L., Stumme, G., Symeonidis, P.: Recommender Systems for Social Tagging Systems. Springer, New York (2012)

    Book  Google Scholar 

  37. Sakai, T.: On the reliability of information retrieval metrics based on graded relevance. Inf. Process. Manage. 43(2), 531–548 (2007)

    Article  Google Scholar 

Download references

Acknowledgments

We would like to gratefully acknowledge Katja Niemann who provided us with the MACE and TravelWell datasets, as well as the organizers of KDD Cup 2015 and XuetangX for making the KDD dataset available. This work is funded by the Know-Center, the EU-IP Learning Layers (Grant Agreement: 318209) and the EU-IP AFEL (Grant Agreement: 687916). The Know-Center is funded within the Austrian COMET Program under the auspices of the Austrian Ministry of Transport, Innovation and Technology, the Austrian Ministry of Economics and Labor and by the State of Styria.

Author information

Authors and Affiliations

  1. Knowledge Technologies Institute, Graz University of Technology, Graz, Austria

    Simone Kopeinik, Dominik Kowald & Elisabeth Lex

Authors
  1. Simone Kopeinik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dominik Kowald
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elisabeth Lex
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simone Kopeinik .

Editor information

Editors and Affiliations

  1. KU Leuven , Leuven, Belgium

    Katrien Verbert

  2. The Open University , Milton Keynes, United Kingdom

    Mike Sharples

  3. Jožef Stefan Institute , Ljubljana, Slovenia

    Tomaž Klobučar

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Kopeinik, S., Kowald, D., Lex, E. (2016). Which Algorithms Suit Which Learning Environments? A Comparative Study of Recommender Systems in TEL. In: Verbert, K., Sharples, M., Klobučar, T. (eds) Adaptive and Adaptable Learning. EC-TEL 2016. Lecture Notes in Computer Science(), vol 9891. Springer, Cham. https://doi.org/10.1007/978-3-319-45153-4_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-45153-4_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-45152-7

  • Online ISBN: 978-3-319-45153-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics