poster

Undergraduates' evaluations of assigned search topics

Authors:

SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval

Pages 812 - 813

https://doi.org/10.1145/1571941.1572141

Published: 19 July 2009 Publication History

Get Access

Abstract

This paper evaluates undergraduate students' knowledge, interests and experiences with 20 topics from the TREC Robust Track collection. The goal is to characterize these topics along several dimensions to help researchers make more informed decisions about which topics are most appropriate to use in experimental IIR evaluations with undergraduate student subjects.

References

[1]

Borlund, P. (2000). Experimental components for the evaluation of interactive information retrieval systems. Journal of Documentation, 56(1). 71--90.

Crossref

Google Scholar

[2]

Csikszentmihalyi, M.,&Larson, R. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease, 175(9), 526--536.

Crossref

Google Scholar

[3]

Järvelin, K.&Kekääläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM TOIS, 20, 422--446.

Digital Library

Google Scholar

[4]

Kelly, D.,&Fu, X. (2006). Elicitation of term relevance feedback: An investigation of term source and context. Proceedings of SIGIR '06, 453--460.

Digital Library

Google Scholar

[5]

Kelly, D., Gyllstrom, K.,&Bailey, E. W. (2009). A comparison of query and term suggestion features. Proceedings of SIGIR '09.

Digital Library

Google Scholar

[6]

Voorhees, E. M. (2006). Overview of the TREC 2005 Robust Retrieval Track. Proceedings of TREC-14.

Google Scholar

Cited By

View all

Chen ACapra RWu W(2015)An investigation of the effects of awareness and task orientation on collaborative searchProceedings of the American Society for Information Science and Technology10.1002/meet.2014.1450510106151:1(1-10)Online publication date: 24-Apr-2015
https://doi.org/10.1002/meet.2014.14505101061
Wildemuth BFreund LG. Toms E(2014)Untangling search task complexity and difficulty in the context of interactive information retrieval studiesJournal of Documentation10.1108/JD-03-2014-005670:6(1118-1140)Online publication date: 7-Oct-2014
https://doi.org/10.1108/JD-03-2014-0056
Capra RChen AHawthorne KArguello JShaw LMarchionini G(2013)Design and evaluation of a system to support collaborative searchProceedings of the American Society for Information Science and Technology10.1002/meet.1450490118149:1(1-10)Online publication date: 24-Jan-2013
https://doi.org/10.1002/meet.14504901181
Show More Cited By

Index Terms

Undergraduates' evaluations of assigned search topics
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Finding Semantically Valid and Relevant Topics by Association-Based Topic Selection Model
Regular Papers and Special Issue: Data-driven Intelligence for Wireless Networking

Topic modelling methods such as Latent Dirichlet Allocation (LDA) have been successfully applied to various fields, since these methods can effectively characterize document collections by using a mixture of semantically rich topics. So far, many models ...
Effectiveness Evaluation with a Subset of Topics: A Practical Approach
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

Several researchers have proposed to reduce the number of topics used in TREC-like initiatives. One research direction that has been pursued is what is the optimal topic subset of a given cardinality that evaluates the systems/runs in the most accurate ...
A few good topics: Experiments in topic set reduction for retrieval evaluation

We consider the issue of evaluating information retrieval systems on the basis of a limited number of topics. In contrast to statistically-based work on sample sizes, we hypothesize that some topics or topic sets are better than others at predicting ...

Comments

Information & Contributors

Information

Published In

SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval

July 2009

896 pages

ISBN:9781605584836

DOI:10.1145/1571941

General Chairs:
James Allan
University of Massachusetts Amherst, USA
,
Javed Aslam
Northeastern University, USA
,
Program Chairs:
Mark Sanderson
University of Sheffield, UK
,
ChengXiang Zhai
University of Illinois at Urbana-Champaign, USA
,
Justin Zobel
University of Melbourne, Australia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

SIGIR '09

Sponsor:

SIGIR '09: The 32nd International ACM SIGIR conference on research and development in Information Retrieval

July 19 - 23, 2009

MA, Boston, USA

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
311
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)1

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Chen ACapra RWu W(2015)An investigation of the effects of awareness and task orientation on collaborative searchProceedings of the American Society for Information Science and Technology10.1002/meet.2014.1450510106151:1(1-10)Online publication date: 24-Apr-2015
https://doi.org/10.1002/meet.2014.14505101061
Wildemuth BFreund LG. Toms E(2014)Untangling search task complexity and difficulty in the context of interactive information retrieval studiesJournal of Documentation10.1108/JD-03-2014-005670:6(1118-1140)Online publication date: 7-Oct-2014
https://doi.org/10.1108/JD-03-2014-0056
Capra RChen AHawthorne KArguello JShaw LMarchionini G(2013)Design and evaluation of a system to support collaborative searchProceedings of the American Society for Information Science and Technology10.1002/meet.1450490118149:1(1-10)Online publication date: 24-Jan-2013
https://doi.org/10.1002/meet.14504901181
Smucker MJethani CCrestani FMarchand-Maillet SChen HEfthimiadis ESavoy J(2010)Human performance and retrieval precision revisitedProceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval10.1145/1835449.1835549(595-602)Online publication date: 19-Jul-2010
https://dl.acm.org/doi/10.1145/1835449.1835549

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Finding Semantically Valid and Relevant Topics by Association-Based Topic Selection Model

Effectiveness Evaluation with a Subset of Topics: A Practical Approach

A few good topics: Experiments in topic set reduction for retrieval evaluation

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations