skip to main content
research-article

Multi-Task Learning for Entity Recommendation and Document Ranking in Web Search

Published: 26 July 2020 Publication History

Abstract

Entity recommendation, providing users with an improved search experience by proactively recommending related entities to a given query, has become an indispensable feature of today’s Web search engine. Existing studies typically only consider the query issued at the current timestep while ignoring the in-session user search behavior (short-term search history) or historical user search behavior across all sessions (long-term search history) when generating entity recommendations. As a consequence, they may fail to recommend entities of interest relevant to a user’s actual information need. In this work, we believe that both short-term and long-term search history convey valuable evidence that could help understand the user’s search intent behind a query, and take both of them into consideration for entity recommendation. Furthermore, there has been little work on exploring whether the use of other companion tasks in Web search such as document ranking as auxiliary tasks could improve the performance of entity recommendation. To this end, we propose a multi-task learning framework with deep neural networks (DNNs) to jointly learn and optimize two companion tasks in Web search engines: entity recommendation and document ranking, which can be easily trained in an end-to-end manner. Specifically, we regard document ranking as an auxiliary task to improve the main task of entity recommendation, where the representations of queries, sessions, and users are shared across all tasks and optimized by the multi-task objective during training. We evaluate our approach using large-scale, real-world search logs of a widely-used commercial Web search engine. We also performed extensive ablation experiments over a number of facets of the proposed multi-task DNN model to figure out their relative importance. The experimental results show that both short-term and long-term search history can bring significant improvements in recommendation effectiveness, and the combination of both outperforms using either of them individually. In addition, the experiments show that the performance of both entity recommendation and document ranking can be significantly improved, which demonstrates the effectiveness of using multi-task learning to jointly optimize the two companion tasks in Web search.

References

[1]
Wasi Ahmad, Kai-Wei Chang, and Hongning Wang. 2018. Multi-task learning for document ranking and query suggestion. In Proceedings of International Conference on Learning Representations. Vancouver, BC, Canada.
[2]
Ziv Bar-Yossef and Naama Kraus. 2011. Context-sensitive query auto-completion. In Proceedings of the 20th International Conference on World Wide Web. Hyderabad, India, 107--116.
[3]
Paul N. Bennett, Ryen W. White, Wei Chu, Susan T. Dumais, Peter Bailey, Fedor Borisyuk, and Xiaoyuan Cui. 2012. Modeling the impact of short-and long-term behavior on search personalization. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. Portland, OR, 185--194.
[4]
Bin Bi, Hao Ma, Bo-June (Paul) Hsu, Wei Chu, Kuansan Wang, and Junghoo Cho. 2015. Learning to recommend related entities to search users. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining. Shanghai, China, 139--148.
[5]
Roi Blanco, Berkant Barla Cambazoglu, Peter Mika, and Nicolas Torzec. 2013. Entity recommendations in web search. In Proceedings of the 12th International Semantic Web Conference. Sydney, Australia, 33--48.
[6]
Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2012. Joint learning of words and meaning representations for open-text semantic parsing. In Proceedings of the 15th International Conference on Artificial Intelligence and Statistics. La Palma, Canary Islands, 127--135.
[7]
Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. 2005. Learning to rank using gradient descent. In Proceedings of the 22nd International Conference on Machine Learning. Bonn, Germany, 89--96.
[8]
Rich Caruana. 1997. Multitask learning. Machine Learning 28, 1 (1997), 41--75.
[9]
Jia Chen, Jiaxin Mao, Yiqun Liu, Min Zhang, and Shaoping Ma. 2020. A context-aware click model for web search. In Proceedings of the 13th International Conference on Web Search and Data Mining. Houston, TX, 88--96.
[10]
Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12 (2011), 2493--2537.
[11]
Mostafa Dehghani, Sascha Rothe, Enrique Alfonseca, and Pascal Fleury. 2017. Learning to attend, copy, and generate for session-based query suggestion. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. Singapore, 1747--1756.
[12]
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015. Multi-task learning for multiple language translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Beijing, China, 1723--1732.
[13]
Ignacio Fernandez-Tobias and Roi Blanco. 2016. Memory-based recommendations of entities for web search users. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management. Indianapolis, IN, 35--44.
[14]
Nicolas Fiorini and Zhiyong Lu. 2018. Personalized neural language models for real-world query auto completion. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers). New Orleans, LA, 208--215.
[15]
Jianfeng Gao, Xiaodong He, and Jian-Yun Nie. 2010. Clickthrough-based translation models for web search: From word models to phrase models. In Proceedings of the 19th ACM Conference on Information and Knowledge Management. Toronto, Ontario, Canada, 1139--1148.
[16]
Songwei Ge, Zhicheng Dou, Zhengbao Jiang, Jian-Yun Nie, and Ji-Rong Wen. 2018. Personalizing search results using hierarchical RNN with query-aware attention. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. Torino, Italy, 347--356.
[17]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735--1780.
[18]
Sha Hu, Zhicheng Dou, Xiaojie Wang, Tetsuya Sakai, and Ji-Rong Wen. 2015. Search result diversification based on hierarchical intents. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management. Melbourne, Australia, 63--72.
[19]
Jizhou Huang, Shiqiang Ding, Haifeng Wang, and Ting Liu. 2018. Learning to recommend related entities with serendipity for web search users. ACM Transactions on Asian and Low-Resource Language Information Processing 17, 3, 25 (April 2018), 22 pages.
[20]
Jizhou Huang, Wei Zhang, Yaming Sun, Haifeng Wang, and Ting Liu. 2018. Improving entity recommendation with search log and multi-task learning. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. Stockholm, Sweden, 4107--4114.
[21]
Aaron Jaech and Mari Ostendorf. 2018. Personalized language model for query auto-completion. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Melbourne, Australia, 700--705.
[22]
Bernard J. Jansen, Amanda Spink, Chris Blakely, and Sherry Koshman. 2007. Defining a session on web search engines. Journal of the American Society for Information Science and Technology 58, 6 (2007), 862--871.
[23]
Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20, 4 (2002), 422--446.
[24]
Di Jiang, Kenneth Wai-Ting Leung, and Wilfred Ng. 2011. Context-aware search personalization with concept preference. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management. Glasgow, Scotland, UK, 563--572.
[25]
Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the 31st International Conference on Machine Learning —Volume 32. Beijing, China, 1188--1196.
[26]
Liangda Li, Hongbo Deng, Anlei Dong, Yi Chang, Ricardo Baeza-Yates, and Hongyuan Zha. 2017. Exploring query auto-completion and click logs for contextual-aware web search and query suggestion. In Proceedings of the 26th International Conference on World Wide Web. Perth, Australia, 539--548.
[27]
Zhen Liao, Daxin Jiang, Enhong Chen, Jian Pei, Huanhuan Cao, and Hang Li. 2011. Mining concept sequences from large-scale search logs for context-aware query suggestion. ACM Transactions on Intelligent Systems and Technology 3, 1, Article 17 (Oct. 2011), 40 pages.
[28]
Zhen Liao, Daxin Jiang, Jian Pei, Yalou Huang, Enhong Chen, Huanhuan Cao, and Hang Li. 2013. A VlHMM approach to context-aware search. ACM Transactions on the Web 7, 4, Article 22 (Nov 2013), 38 pages.
[29]
Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon.com Recommendations: Item-to-item collaborative filtering. IEEE Internet Computing 7, 1 (2003), 76--80.
[30]
Fang Liu, Clement Yu, and Weiyi Meng. 2004. Personalized web search for improving retrieval effectiveness. IEEE Transactions on Knowledge and Data Engineering 16, 1 (Jan. 2004), 28--40.
[31]
Xiaodong Liu, Jianfeng Gao, Xiaodong He, Li Deng, Kevin Duh, and Ye-Yi Wang. 2015. Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Denver, CO, 912--921.
[32]
Julia Luxenburger, Shady Elbassuoni, and Gerhard Weikum. 2008. Matching task profiles and user needs in personalized web search. In Proceedings of the 17th ACM Conference on Information and Knowledge Management. Napa Valley, CA, 689--698.
[33]
Nicolaas Matthijs and Filip Radlinski. 2011. Personalizing web search using long term browsing history. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining. Hong Kong, China, 25--34.
[34]
Dae Hoon Park and Rikio Chiba. 2017. A neural language model for query auto-completion. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. Shinjuku, Tokyo, Japan, 1189--1192.
[35]
Feng Qiu and Junghoo Cho. 2006. Automatic identification of user interest for personalized search. In Proceedings of the 15th International Conference on World Wide Web. Edinburgh, Scotland, 727--736.
[36]
Filip Radlinski and Susan Dumais. 2006. Improving personalized web search using result diversification. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Seattle, WA, 691--692.
[37]
Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web. Hong Kong, China, 285--295.
[38]
Xuehua Shen, Bin Tan, and ChengXiang Zhai. 2005. Context-sensitive information retrieval using implicit feedback. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Salvador, Brazil, 43--50.
[39]
Milad Shokouhi. 2013. Learning to personalize query auto-completion. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval. Dublin, Ireland, 103--112.
[40]
Jian-Tao Sun, Hua-Jun Zeng, Huan Liu, Yuchang Lu, and Zheng Chen. 2005. CubeSVD: A novel approach to personalized web search. In Proceedings of the 14th International Conference on World Wide Web. Chiba, Japan, 382--390.
[41]
Bin Tan, Xuehua Shen, and ChengXiang Zhai. 2006. Mining long-term search history to improve search accuracy. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Philadelphia, PA, 718--723.
[42]
Jaime Teevan, Susan T. Dumais, and Eric Horvitz. 2010. Potential for personalization. ACM Transactions on Computer-Human Interaction (TOCHI) 17, 1 (2010), 4.
[43]
Yury Ustinovskiy and Pavel Serdyukov. 2013. Personalization of web-search using short-term browsing context. In Proceedings of the 22nd ACM International Conference on Information and Knowledge Management. San Francisco, CA, 1979--1988.
[44]
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph and text jointly embedding. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar, 1591--1601.
[45]
Ryen W. White, Paul N. Bennett, and Susan T. Dumais. 2010. Predicting short-term interests using activity-based search context. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management. Toronto, ON, Canada, 1009--1018.
[46]
Ryen W. White, Mikhail Bilenko, and Silviu Cucerzan. 2007. Studying the use of popular destinations to enhance web search interaction. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Amsterdam, The Netherlands, 159--166.
[47]
Ryen W. White, Wei Chu, Ahmed Hassan, Xiaodong He, Yang Song, and Hongning Wang. 2013. Enhancing personalized search by mining and modeling task behavior. In Proceedings of the 22nd International Conference on World Wide Web. Rio de Janeiro, Brazil, 1411--1420.
[48]
Biao Xiang, Daxin Jiang, Jian Pei, Xiaohui Sun, Enhong Chen, and Hang Li. 2010. Context-aware ranking in web search. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Geneva, Switzerland, 451--458.
[49]
Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. 2016. Representation learning of knowledge graphs with entity descriptions. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. Phoenix, AZ, 2659--2665.
[50]
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego, CA, 1480--1489.
[51]
Xiao Yu, Hao Ma, Bo-June Paul Hsu, and Jiawei Han. 2014. On building entity recommender systems using user click log and freebase knowledge. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining. New York, NY, 263--272.
[52]
Dongxu Zhang, Bin Yuan, Dong Wang, and Rong Liu. 2015. Joint semantic relevance learning with text data and graph knowledge. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality. Beijing, China, 32--40.
[53]
Huaping Zhong, Jianwen Zhang, Zhen Wang, Hai Wan, and Zheng Chen. 2015. Aligning knowledge and text embeddings by entity descriptions. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal, 267--272.
[54]
Yadong Zhu, Yanyan Lan, Jiafeng Guo, Xueqi Cheng, and Shuzi Niu. 2014. Learning for search result diversification. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval. Gold Coast, Queensland, Australia, 293--302.

Cited By

View all
  • (2024)A Simple yet Effective Framework for Active Learning to RankMachine Intelligence Research10.1007/s11633-023-1422-z21:1(169-183)Online publication date: 15-Jan-2024
  • (2022)DuIVA: An Intelligent Voice Assistant for Hands-free and Eyes-free Voice Interaction with the Baidu Maps AppProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539030(3040-3050)Online publication date: 14-Aug-2022
  • (2022)Entity Recommendation With Negative Feedback Memory Networks for Topic-Oriented Knowledge Graph ExplorationIEEE Transactions on Reliability10.1109/TR.2022.316909271:2(788-802)Online publication date: Jun-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology
ACM Transactions on Intelligent Systems and Technology  Volume 11, Issue 5
Survey Paper and Regular Paper
October 2020
325 pages
ISSN:2157-6904
EISSN:2157-6912
DOI:10.1145/3409643
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2020
Online AM: 07 May 2020
Accepted: 01 April 2020
Revised: 01 March 2020
Received: 01 February 2019
Published in TIST Volume 11, Issue 5

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Entity recommendation
  2. Web search
  3. context-aware
  4. document ranking
  5. multi-task learning
  6. neural networks
  7. personalized

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)34
  • Downloads (Last 6 weeks)1
Reflects downloads up to 18 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Simple yet Effective Framework for Active Learning to RankMachine Intelligence Research10.1007/s11633-023-1422-z21:1(169-183)Online publication date: 15-Jan-2024
  • (2022)DuIVA: An Intelligent Voice Assistant for Hands-free and Eyes-free Voice Interaction with the Baidu Maps AppProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539030(3040-3050)Online publication date: 14-Aug-2022
  • (2022)Entity Recommendation With Negative Feedback Memory Networks for Topic-Oriented Knowledge Graph ExplorationIEEE Transactions on Reliability10.1109/TR.2022.316909271:2(788-802)Online publication date: Jun-2022
  • (2022)Active tag recommendation for interactive entity search: Interaction effectiveness and retrieval performanceInformation Processing & Management10.1016/j.ipm.2021.10285659:2(102856)Online publication date: Mar-2022
  • (2022)NGPR: A comprehensive personalized point-of-interest recommendation method based on heterogeneous graphsMultimedia Tools and Applications10.1007/s11042-022-13088-481:27(39207-39228)Online publication date: 1-Nov-2022
  • (2021)Information Search Trail Recommendation Based on Markov Chain Model and Case-based ReasoningData and Information Management10.2478/dim-2020-00475:1(228-241)Online publication date: Jan-2021
  • (2021)A Weakly Supervised Academic Search Model Based on Knowledge-Enhanced Feature RepresentationWireless Communications and Mobile Computing10.1155/2021/44115242021(1-12)Online publication date: 22-Sep-2021
  • (2021)GEDITProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3481924(4135-4144)Online publication date: 26-Oct-2021
  • (2021)HGAMN: Heterogeneous Graph Attention Matching Network for Multilingual POI Retrieval at Baidu MapsProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining10.1145/3447548.3467059(3032-3040)Online publication date: 14-Aug-2021
  • (2021)Meta-Learned Spatial-Temporal POI Auto-Completion for the Search Engine at Baidu MapsProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining10.1145/3447548.3467058(2822-2830)Online publication date: 14-Aug-2021

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media