research-article

A hierarchical Dirichlet model for taxonomy expansion for search engines

Authors:

Changsung Kang,

Jiawei HanAuthors Info & Claims

WWW '14: Proceedings of the 23rd international conference on World wide web

Pages 961 - 970

https://doi.org/10.1145/2566486.2568037

Published: 07 April 2014 Publication History

Abstract

Emerging trends and products pose a challenge to modern search engines since they must adapt to the constantly changing needs and interests of users. For example, vertical search engines, such as Amazon, eBay, Walmart, Yelp and Yahoo! Local, provide business category hierarchies for people to navigate through millions of business listings. The category information also provides important ranking features that can be used to improve search experience. However, category hierarchies are often manually crafted by some human experts and they are far from complete. Manually constructed category hierarchies cannot handle the ever-changing and sometimes long-tail user information needs. In this paper, we study the problem of how to expand an existing category hierarchy for a search/navigation system to accommodate the information needs of users more comprehensively. We propose a general framework for this task, which has three steps: 1) detecting meaningful missing categories; 2) modeling the category hierarchy using a hierarchical Dirichlet model and predicting the optimal tree structure according to the model; 3) reorganizing the corpus using the complete category structure, i.e., associating each webpage with the relevant categories from the complete category hierarchy. Experimental results demonstrate that our proposed framework generates a high-quality category hierarchy and significantly boosts the retrieval performance.

References

[1]

R. Agrawal and R. Srikant. Fast algorithms for mining association rules. In Proc. of 20th Intl. Conf. on VLDB, pages 487--499, 1994.

Digital Library

[2]

D. M. Blei, T. L. Griffiths, and M. I. Jordan. The nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies. Journal of the ACM (JACM), 57(2):7, 2010.

Digital Library

[3]

C. Blundell, Y. W. Teh, and K. A. Heller. Bayesian rose trees. arXiv preprint arXiv:1203.3468, 2012.

[4]

R. J. Brachman. What is-a is and isn't: An analysis of taxonomic links in semantic networks. IEEE Computer, 16(10):30--36, 1983.

Digital Library

[5]

A. Clauset, C. Moore, and M. E. Newman. Hierarchical structure and the prediction of missing links in networks. Nature, 453(7191):98--101, 2008.

[6]

J. Edmonds. Optimum branchings. Journal of Research of the National Bureau of Standards B, 71(4):233--240, 1967.

[7]

T. Fountain and M. Lapata. Taxonomy induction using hierarchical random graphs. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 466--476. Association for Computational Linguistics, 2012.

Digital Library

[8]

H. N. Gabow, Z. Galil, T. Spencer, and R. E. Tarjan. Efficient algorithms for finding minimum spanning trees in undirected and directed graphs. Combinatorica, 6(2):109--122, 1986.

Digital Library

[9]

T. Griffiths. Hierarchical topic models and the nested chinese restaurant process. Advances in neural information processing systems, 16:106--114, 2004.

[10]

W. K. Hastings. Monte carlo sampling methods using markov chains and their applications. Biometrika, 57(1):97--109, 1970.

[11]

C. Kang, J. Lee, and Y. Chang. Predicting primary categories of business listings for local search. In Proceedings of the 21st ACM international conference on Information and knowledge management, CIKM '12, pages 2591--2594, New York, NY, USA, 2012. ACM.

Digital Library

[12]

S. Kotz, N. Balakrishnan, and N. Johnson. Continuous Multivariate Distributions, Models and Applications. Continuous Multivariate Distributions. Wiley, 2004.

[13]

Z. Kozareva, E. Riloff, and E. H. Hovy. Semantic class learning from the web with hyponym pattern linkage graphs. In ACL, volume 8, pages 1048--1056, 2008.

[14]

W. Li, D. Blei, and A. McCallum. Nonparametric bayes pachinko allocation. arXiv preprint arXiv:1206.5270, 2012.

[15]

X. Liu, Y. Song, S. Liu, and H. Wang. Automatic taxonomy construction from keywords. In KDD, pages 1433--1441, 2012.

Digital Library

[16]

X.-L. Mao, Z.-Y. Ming, T.-S. Chua, S. Li, H. Yan, and X. Li. Sshlda: a semi-supervised hierarchical topic model. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 800--809. Association for Computational Linguistics, 2012.

Digital Library

[17]

D. Mimno, W. Li, and A. McCallum. Mixtures of hierarchical topics with pachinko allocation. In Proceedings of the 24th international conference on Machine learning, pages 633--640. ACM, 2007.

Digital Library

[18]

R. Navigli, P. Velardi, and S. Faralli. A graph-based algorithm for inducing lexical taxonomies from scratch. In Proceedings of the Twenty-Second international joint conference on Artificial Intelligence-Volume Volume Three, pages 1872--1877. AAAI Press, 2011.

Digital Library

[19]

J. Nivre, J. Hall, J. Nilsson, A. Chanev, G. Eryigit, S. Kubler, S. Marinov, and E. Marsi. Maltparser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 13(2):95--135, 2007.

[20]

Y. Petinot, K. McKeown, and K. Thadani. A hierarchical model of web summaries. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers - Volume 2, HLT '11, pages 670--675, Stroudsburg, PA, USA, 2011. Association for Computational Linguistics.

Digital Library

[21]

S. P. Ponzetto and M. Strube. Deriving a large scale taxonomy from wikipedia. In Proceedings of the 22nd national conference on Artificial intelligence - Volume 2, AAAI'07, pages 1440--1445. AAAI Press, 2007.

Digital Library

[22]

A. Ratnaparkhi. A maximum entropy model for Part-Of-speech tagging. In E. Brill and K. Church, editors, Proceedings of the Empirical Methods in Natural Language Processing, pages 133--142, 1996.

[23]

R. Snow, D. Jurafsky, and A. Y. Ng. Semantic taxonomy induction from heterogenous evidence. In ACL, 2006.

Digital Library

[24]

D. Stewart. Building Enterprise Taxonomies. Mokita Press, 2011.

Digital Library

[25]

C. Wang, M. Danilevsky, N. Desai, Y. Zhang, P. Nguyen, T. Taula, and J. Han. A phrase mining framework for recursive construction of a topical hierarchy. In KDD, pages 437--445, New York, NY, USA, 2013. ACM.

Digital Library

[26]

F. Wu and D. S. Weld. Automatically refining the wikipedia infobox ontology. In WWW, pages 635--644, 2008.

Digital Library

[27]

E. Zavitsanos, G. Paliouras, and G. A. Vouros. Non-parametric estimation of topic hierarchies from texts with hierarchical dirichlet processes. The Journal of Machine Learning Research, 12:2749--2775, 2011.

Digital Library

[28]

C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to information retrieval. ACM Transactions on Information Systems (TOIS), 22(2):179--214, 2004.

Digital Library

Cited By

Galhotra SFirmani DSaha BSrivastava DIves ZBonifati AEl Abbadi A(2022)Hierarchical Entity Resolution using an OracleProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3526147(414-428)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3526147
Shen JHan JShen JHan J(2022)Taxonomy EnrichmentAutomated Taxonomy Discovery and Exploration10.1007/978-3-031-11405-2_4(49-81)Online publication date: 22-Sep-2022
https://doi.org/10.1007/978-3-031-11405-2_4
Zalmout NZhang CLi XLiang YDong XZhu FChin Ooi BMiao CWang HSkrypnyk IHsu WChawla S(2021)All You Need to Know to Build a Product Knowledge GraphProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining10.1145/3447548.3470825(4090-4091)Online publication date: 14-Aug-2021
https://dl.acm.org/doi/10.1145/3447548.3470825
Show More Cited By

Index Terms

A hierarchical Dirichlet model for taxonomy expansion for search engines
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing
    2. Retrieval tasks and goals
      1. Clustering and classification
  2. Information systems applications
    1. Data mining
      1. Clustering

Recommendations

Overlap Among Major Web Search Engines
ITNG '06: Proceedings of the Third International Conference on Information Technology: New Generations

Our study examined the overlap among results retrieved by three major Web search engines for a large set of more than 10,316 queries. Previous smaller studies have discussed the lack of overlap in results returned by Web search engines for the same ...
A study of results overlap and uniqueness among major web search engines

The performance and capabilities of Web search engines is an important and significant area of research. Millions of people world wide use Web search engines very day. This paper reports the results of a major study examining the overlap among results ...
Query Expansion Based on a Personalized Web Search Model
SKG '07: Proceedings of the Third International Conference on Semantics, Knowledge and Grid

A novel query expansion algorithm is proposed in this paper. It is based on a model of personalized web search system. The new system, as a middleware between a user and a Web search engine, is set up on the client machine. It can learn a user's ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '14: Proceedings of the 23rd international conference on World wide web

April 2014

926 pages

ISBN:9781450327442

DOI:10.1145/2566486

General Chair:
Chin-Wan Chung
Korea Advanced Institute of Science and Technology, Korea
,
Program Chairs:
Andrei Broder
Google Inc., USA
,
Kyuseok Shim
Seoul National University, Korea
,
Torsten Suel
New York University, USA

Copyright © 2014 Copyright is held by the International World Wide Web Conference Committee (IW3C2).

Sponsors

IW3C2: International World Wide Web Conference Committee

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 April 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WWW '14

Sponsor:

IW3C2

WWW '14: 23rd International World Wide Web Conference

April 7 - 11, 2014

Seoul, Korea

Acceptance Rates

WWW '14 Paper Acceptance Rate 84 of 645 submissions, 13%;

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
349
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Galhotra SFirmani DSaha BSrivastava DIves ZBonifati AEl Abbadi A(2022)Hierarchical Entity Resolution using an OracleProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3526147(414-428)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3526147
Shen JHan JShen JHan J(2022)Taxonomy EnrichmentAutomated Taxonomy Discovery and Exploration10.1007/978-3-031-11405-2_4(49-81)Online publication date: 22-Sep-2022
https://doi.org/10.1007/978-3-031-11405-2_4
Zalmout NZhang CLi XLiang YDong XZhu FChin Ooi BMiao CWang HSkrypnyk IHsu WChawla S(2021)All You Need to Know to Build a Product Knowledge GraphProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining10.1145/3447548.3470825(4090-4091)Online publication date: 14-Aug-2021
https://dl.acm.org/doi/10.1145/3447548.3470825
Wang SZhao RChen XZheng YLiu B(2021)Enquire One’s Parent and Child Before Decision: Fully Exploit Hierarchical Structure for Self-Supervised Taxonomy ExpansionProceedings of the Web Conference 202110.1145/3442381.3449948(3291-3304)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3449948
Giabelli AMalandri LMercorio FMezzanzanica MSeveso A(2020)NEO: A Tool for Taxonomy Enrichment with New Emerging OccupationsThe Semantic Web – ISWC 202010.1007/978-3-030-62466-8_35(568-584)Online publication date: 2-Nov-2020
https://dl.acm.org/doi/10.1007/978-3-030-62466-8_35
Shin SMoon I(2017)Guided HTMIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2016.262579029:2(330-343)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1109/TKDE.2016.2625790
Zhu XMing ZHao YZhu X(2015)Tackling data sparseness in recommendation using social media based topic hierarchy modelingProceedings of the 24th International Conference on Artificial Intelligence10.5555/2832581.2832586(2415-2421)Online publication date: 25-Jul-2015
https://dl.acm.org/doi/10.5555/2832581.2832586

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten