skip to main content
10.1145/3627217.3627224acmotherconferencesArticle/Chapter ViewAbstractPublication PagescomputeConference Proceedingsconference-collections
research-article

Learning to Rank for Search Results Re-ranking in Learning Experience Platforms

Published: 19 December 2023 Publication History

Abstract

The ability to search and retrieve the right resources in a Learning Experience Platform (LXP) is critical in helping the workforce of an enterprise to upskill and deepen their expertise effectively. To ensure the best resources are shown as high in the result set as possible to catch learners’ attention, a supervised learning approach of training and deploying a Learning to Rank (LTR) model for re-ranking is proposed. This work specifically focuses on judgement list preparation taking advantage of the learning progress data available in LXPs, as well as on defining and measuring model performance through metrics in both test and production setups. In particular, it highlights the positive impact of the deployed LTR model in production using the defined metrics like average search result click position and percentage top N clicks.

References

[1]
Christopher JC Burges. 2010. From ranknet to lambdarank to lambdamart: An overview. Learning 11, 23-581 (2010), 81.
[2]
Olivier Chapelle and Ya Zhang. 2009. A dynamic bayesian network click model for web search ranking. In Proceedings of the 18th international conference on World wide web. 1–10.
[3]
Aleksandr Chuklin, Ilya Markov, and Maarten De Rijke. 2022. Click models for web search. Springer Nature.
[4]
Nick Craswell. 2018. Mean Reciprocal Rank. Springer New York, New York, NY, 2217–2217. https://doi.org/10.1007/978-1-4614-8265-9_488
[5]
Janeve R Desy, Darcy A Reed, and Alexandra P Wolanskyj. 2017. Milestones and millennials: a perfect pairing—competency-based medical education and the learning preferences of generation Y. In Mayo Clinic Proceedings, Vol. 92. Elsevier, 243–250.
[6]
Adrián Domínguez, Joseba Saenz-de Navarrete, Luis De-Marcos, Luis Fernández-Sanz, Carmen Pagés, and José-Javier Martínez-Herráiz. 2013. Gamifying learning experiences: Practical implications and outcomes. Computers & education 63 (2013), 380–392.
[7]
Georges E Dupret and Benjamin Piwowarski. 2008. A user browsing model to predict search engine click data from past observations. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval. 331–338.
[8]
Luminiţa Giurgiu. 2017. Microlearning an evolving elearning trend. Scientific Bulletin 22, 1 (2017), 18–23.
[9]
Fan Guo, Chao Liu, Anitha Kannan, Tom Minka, Michael Taylor, Yi-Min Wang, and Christos Faloutsos. 2009. Click chain model in web search. In Proceedings of the 18th international conference on World wide web. 11–20.
[10]
Luis Gutiérrez and Brian Keith. 2019. A systematic literature review on word embeddings. In Trends and Applications in Software Engineering: Proceedings of the 7th International Conference on Software Process Improvement (CIMPS 2018) 7. Springer, 132–141.
[11]
Venkateshprasanna HM, Ayush Kataria, Arnav Saraf, and Makrand Pawar. 2022. Metadata Interoperability and Multilingual Information Retrieval in Multitenant Enterprise Learning Platforms: An Approach. In Proceedings of the 8th International Conference on e-Society, e-Learning and e-Technologies. 14–19.
[12]
Liangjie Hong, Ron Bekkerman, Joseph Adler, and Brian D Davison. 2012. Learning to rank social update streams. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. 651–660.
[13]
Yujing Hu, Qing Da, Anxiang Zeng, Yang Yu, and Yinghui Xu. 2018. Reinforcement learning to rank in e-commerce search engine: Formalization, analysis, and application. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 368–377.
[14]
Kalervo Järvelin and Jaana Kekäläinen. 2017. IR evaluation methods for retrieving highly relevant documents. In ACM SIGIR Forum, Vol. 51. ACM New York, NY, USA, 243–250.
[15]
Shubhra Kanti Karmaker Santu, Parikshit Sondhi, and ChengXiang Zhai. 2017. On application of learning to rank for e-commerce search. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval. 475–484.
[16]
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33 (2020), 9459–9474.
[17]
Tie-Yan Liu 2009. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval 3, 3 (2009), 225–331.
[18]
Craig Macdonald, Rodrygo LT Santos, and Iadh Ounis. 2013. The whens and hows of learning to rank for web search. Information Retrieval 16 (2013), 584–628.
[19]
Patrick E Parrish, Brent G Wilson, and Joanna C Dunlap. 2011. Learning experience as transaction: A framework for instructional design. Educational Technology (2011), 15–22.
[20]
Tao Qin, Tie-Yan Liu, Xu-Dong Zhang, De-Sheng Wang, Wen-Ying Xiong, and Hang Li. 2008. Learning to rank relational objects and its application to web search. In Proceedings of the 17th international conference on World Wide Web. 407–416.
[21]
Stephen Robertson, Hugo Zaragoza, 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389.
[22]
Daniel E Rose and Danny Levinson. 2004. Understanding user goals in web search. In Proceedings of the 13th international conference on World Wide Web. 13–19.
[23]
Rafael Solis. 2017. Employee training strategies for today’s workforce. Strategic HR Review 16, 6 (2017), 250–254.
[24]
Mihai Surdeanu, Massimiliano Ciaramita, and Hugo Zaragoza. 2011. Learning to rank answers to non-factoid questions from web collections. Computational linguistics 37, 2 (2011), 351–383.
[25]
JK Suresh and Kavi Mahesh. 2006. Ten steps to maturity in knowledge management: Lessons in economy. Elsevier.
[26]
Arif Usta, Ismail Sengor Altingovde, Rifat Ozcan, and Özgür Ulusoy. 2021. Learning to rank for educational search engines. IEEE Transactions on Learning Technologies 14, 2 (2021), 211–225.
[27]
Ryen White. 2013. Beliefs and biases in web search. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. 3–12.
[28]
Jun Yu, Dacheng Tao, Meng Wang, and Yong Rui. 2014. Learning to rank using user clicks and visual features for image retrieval. IEEE transactions on cybernetics 45, 4 (2014), 767–779.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
COMPUTE '23: Proceedings of the 16th Annual ACM India Compute Conference
December 2023
120 pages
ISBN:9798400708404
DOI:10.1145/3627217
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 December 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Information Retrieval
  2. Learning Experience Platforms
  3. Learning to Rank
  4. Supervised Learning

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

COMPUTE '23
COMPUTE '23: 16th Annual ACM India Compute Conference
December 9 - 11, 2023
Hyderabad, India

Acceptance Rates

Overall Acceptance Rate 114 of 622 submissions, 18%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 33
    Total Downloads
  • Downloads (Last 12 months)28
  • Downloads (Last 6 weeks)1
Reflects downloads up to 11 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media