short-paper

Summary of the NTCIR-10 INTENT-2 task: subtopic mining and search result diversification

Authors:

Takehiro Yamamoto,

Makoto P. Kato,

Mayu IwataAuthors Info & Claims

SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

Pages 761 - 764

https://doi.org/10.1145/2484028.2484104

Published: 28 July 2013 Publication History

Abstract

The NTCIR INTENT task comprises two subtasks: {\em Subtopic Mining}, where systems are required to return a ranked list of {\em subtopic strings} for each given query; and {\em Document Ranking}, where systems are required to return a diversified web search result for each given query. This paper summarises the novel features of the Second INTENT task at NTCIR-10 and its main findings, and poses some questions for future diversified search evaluation.

References

[1]

R. Agrawal, G. Sreenivas, A. Halverson, and S. Leong. Diversifying search results. In Proceedings of ACM WSDM 2009, pages 5--14, 2009.

Digital Library

[2]

O. Chapelle, S. Ji, C. Liao, E. Velipasaoglu, L. Lai, and S.-L. Wu. Intent-based diversification of web search results: Metrics and algorithms. Information Retrieval, 14(6):572--592, 2011.

Digital Library

[3]

C. L. A. Clarke, N. Craswell, I. Soboroff, and A. Ashkan. A comparative analysis of cascade measures for novelty and diversity. In Proceedings of ACM WSDM 2011, pages 75--84, 2011.

Digital Library

[4]

C. L. A. Clarke, N. Craswell, and E. M. Voorhees. Overview of the TREC 2012 web track. In Proceedings of TREC 2012, 2013.

[5]

K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of IR techniques. ACM TOIS, 20(4):422--446, 2002.

Digital Library

[6]

T. Sakai. Evaluating evaluation metrics based on the bootstrap. In Proceedings of ACM SIGIR 2006, pages 525--532, 2006.

Digital Library

[7]

T. Sakai. A note on progress in document retrieval technology based on the official ntcir results (in japanese). In Proceedings of FIT 2006, pages 67--70, 2006.

[8]

T. Sakai. Evaluation with informational and navigational intents. In Proceedings of ACM WWW 2012, pages 499--508, 2012.

Digital Library

[9]

T. Sakai. The unreusability of diversified search test collections. In Proceedings of EVIA 2013, 2013.

[10]

T. Sakai, Z. Dou, and C. L. Clarke. The impact of intent selection on diversified search evaluation. In Proceedings of ACM SIGIR 2013, 2013.

Digital Library

[11]

T. Sakai, Z. Dou, R. Song, and N. Kando. The reusability of a diversified search test collection. In Proceedings of AIRS 2012, pages 26--38, 2012.

[12]

T. Sakai, Z. Dou, T. Yamamoto, Y. Liu, M. Zhang, M. P. Kato, R. Song, and M. Iwata. Overview of the NTCIR-10 INTENT-2 task. In Proceedings of NTCIR-10, 2013.

[13]

T. Sakai and R. Song. Evaluating diversified search results using per-intent graded relevance. In Proceedings of ACM SIGIR 2011, pages 1043--1042, 2011.

Digital Library

[14]

T. Sakai and R. Song. Diversified search evaluation: Lessons from the NTCIR-9 INTENT task. Information Retrieval, 2013.

Digital Library

[15]

R. L. Santos, C. Macdonald, and I. Ounis. Selectively diversifying web search results. In Proceedings of CIKM 2010, pages 1179--1188, 2010.

Digital Library

[16]

R. Song, M. Zhang, T. Sakai, M. P. Kato, Y. Liu, M. Sugimoto, Q. Wang, and N. Orii. Overview of the NTCIR-9 INTENT task. In Proceedings of NTCIR-9, pages 82--105, 2011.

[17]

C. Zhai, W. W. Cohen, and J. Lafferty. Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. In Proceedings of ACM SIGIR 2003, pages 10--17, 2003.

Digital Library

Cited By

Shajalal MAono M(2020)Coverage-based query subtopic diversification leveraging semantic relevanceKnowledge and Information Systems10.1007/s10115-020-01470-362:7(2873-2891)Online publication date: 27-Apr-2020
https://doi.org/10.1007/s10115-020-01470-3
Dou ZGuo J(2020)Query Intent UnderstandingQuery Understanding for Search Engines10.1007/978-3-030-58334-7_4(69-101)Online publication date: 2-Dec-2020
https://doi.org/10.1007/978-3-030-58334-7_4
Gaou SFihri M(2020)A New Approach of a List of Items for Search Retrieval SystemsAdvanced Intelligent Systems for Sustainable Development (AI2SD’2019)10.1007/978-3-030-36674-2_6(53-66)Online publication date: 6-Feb-2020
https://doi.org/10.1007/978-3-030-36674-2_6
Show More Cited By

Index Terms

Summary of the NTCIR-10 INTENT-2 task: subtopic mining and search result diversification
1. Information systems
  1. Information retrieval

Recommendations

Report from the NTCIR-10 1CLICK-2 Japanese subtask: baselines, upperbounds and evaluation robustness
SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

The One Click Access Task (1CLICK) of NTCIR requires systems to return a concise multi-document summary of web pages in response to a query which is assumed to have been submitted in a mobile context. Systems are evaluated based on information units (or ...
The impact of intent selection on diversified search evaluation
SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

To construct a diversified search test collection, a set of possible subtopics (or intents) needs to be determined for each topic, in one way or another, and perintent relevance assessments need to be obtained. In the TREC Web Track Diversity Task, ...
Diversified search evaluation: lessons from the NTCIR-9 INTENT task
Abstract
The evaluation of diversified web search results is a relatively new research topic and is not as well-understood as the time-honoured evaluation methodology of traditional IR based on precision and recall. In diversity evaluation, one topic may ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

July 2013

1188 pages

ISBN:9781450320344

DOI:10.1145/2484028

General Chairs:
Gareth J.F. Jones
Dublin City University, Ireland
,
Páraic Sheridan
Dublin City University, Ireland
,
Program Chairs:
Diane Kelly
University of North Carolina, Chapel Hill, USA
,
Maarten de Rijke
University of Amsterdam, The Netherlands
,
Tetsuya Sakai
Microsoft Research Asia, China

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 July 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

SIGIR '13

Sponsor:

SIGIR

SIGIR '13: The 36th International ACM SIGIR conference on research and development in Information Retrieval

July 28 - August 1, 2013

Dublin, Ireland

Acceptance Rates

SIGIR '13 Paper Acceptance Rate 73 of 366 submissions, 20%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
255
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)1

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shajalal MAono M(2020)Coverage-based query subtopic diversification leveraging semantic relevanceKnowledge and Information Systems10.1007/s10115-020-01470-362:7(2873-2891)Online publication date: 27-Apr-2020
https://doi.org/10.1007/s10115-020-01470-3
Dou ZGuo J(2020)Query Intent UnderstandingQuery Understanding for Search Engines10.1007/978-3-030-58334-7_4(69-101)Online publication date: 2-Dec-2020
https://doi.org/10.1007/978-3-030-58334-7_4
Gaou SFihri M(2020)A New Approach of a List of Items for Search Retrieval SystemsAdvanced Intelligent Systems for Sustainable Development (AI2SD’2019)10.1007/978-3-030-36674-2_6(53-66)Online publication date: 6-Feb-2020
https://doi.org/10.1007/978-3-030-36674-2_6
Zouhair AGaou SEl Mokhtar ECastillo-Valdivieso PBertelle CLazaar MDuvallet CAl Achhab MMahboub OSilkan H(2019)A New Approach for Search Engine Optimization based Case Based Reasoning to detect user's intentProceedings of the 4th International Conference on Big Data and Internet of Things10.1145/3372938.3372975(1-8)Online publication date: 23-Oct-2019
https://dl.acm.org/doi/10.1145/3372938.3372975
(2018)Hierarchical classification of web search results to detect usersInternational Journal of Artificial Intelligence and Soft Computing10.5555/3309394.33093966:4(287-305)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.5555/3309394.3309396
Pothirattanachaikul SYamamoto TFujita STajima ATanaka KYoshikawa M(2018)Mining Alternative Actions from Community Q&A CorpusJournal of Information Processing10.2197/ipsjjip.26.42726(427-438)Online publication date: 2018
https://doi.org/10.2197/ipsjjip.26.427
Wu YLiu YChen FZhang MMa SSong DLiu TSun LBruza PMelucci MSebastiani FYang G(2018)Beyond Greedy SearchProceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3234944.3234967(99-106)Online publication date: 10-Sep-2018
https://dl.acm.org/doi/10.1145/3234944.3234967
Jiang ZDou ZZhao WNie JYue MWen J(2018)Supervised Search Result Diversification via Subtopic AttentionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2018.281087330:10(1971-1984)Online publication date: 1-Oct-2018
https://doi.org/10.1109/TKDE.2018.2810873
Umemoto KYamamoto TTanaka K(2017)Displaying the Amount of Missed Information in Recall-Oriented TasksTransactions of the Japanese Society for Artificial Intelligence10.1527/tjsai.WII-G32:1(WII-G_1-12)Online publication date: 2017
https://doi.org/10.1527/tjsai.WII-G
Pothirattanachaikul SYamamoto TFujita STajima ATanaka KSheth ANgonga AWang yChang EŚlęzak DFranczyk BAlt RTao X(2017)Mining alternative actions from community Q&A corpus for task-oriented web searchProceedings of the International Conference on Web Intelligence10.1145/3106426.3106461(607-614)Online publication date: 23-Aug-2017
https://dl.acm.org/doi/10.1145/3106426.3106461
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten