Skip to main content

Advertisement

Springer Nature Link
Log in

Evaluation of contextual information retrieval effectiveness: overview of issues and research

  • Survey Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

The increasing prominence of information arising from a wide range of sources delivered over electronic media has made traditional information retrieval systems less effective. Indeed, users are overwhelmed by the information delivered by such systems in response to their queries, particularly when the latter are ambiguous. In order to tackle this problem, the state-of-the-art reveals that there is a growing interest towards contextual information retrieval which relies on various sources of evidence issued from the user’s search background and environment like interests, preferences, time and location, in order to improve the retrieval accuracy. Contextual information retrieval systems are based on different definitions of the core concept of user’s context, various user’s context modeling approaches and several techniques of document relevance measurement, but all share the goal of providing the most useful information to the users in accordance with their context. However, the evaluation methodologies conceived in the past several years for traditional information retrieval and widely used in the evaluation campaigns have been challenged by the consideration of user’s context in the information retrieval process. Thus, we recognize that a critical review of existing evaluation methodologies in contextual information retrieval area is needed in order to design and develop standard evaluation frameworks. We present in this paper a comprehensive survey of contextual information retrieval evaluation methodologies and provide insights into how and why they are appropriate to measure the retrieval effectiveness. We also highlight some of the research challenges ahead that would constitute substantive research area for future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Agichtein E, Brill E, Dumais ST (2006) Improving web search ranking by incorporating user behavior information. In: Proceedings of the 29th international SIGIR conference on research and development in information retrieval, pp 19–26

  2. Allan J (2002) Challenges in information retrieval and langage modelling. In: Report of a workshop held at the Center for Intelligent Information Retrieval, University of Massachusetts, Amherst

  3. Allan J (2003) Hard track overview in trec 2003 high accuracy retrieval from documents. In: Proceedings of the 12th text retrieval conference (TREC-12), National Institute of Standards and Technology, NIST special publication, pp 24–37

  4. Anand SS, Mobasher B (2007) Introduction to intelligent techniques for web personalization. ACM Trans Internet Technol 7(4): 18

    Article  Google Scholar 

  5. Anderson C, Domingos P, Weld D (2001) Personalizing web sites for mobile users. In: Anderson CR, Domingos P, Weld DS (eds) Personalizing web sites for mobile users. Proceedings of the 10th international WWW conference, pp 565–575

  6. Baeza-Yates R, Ribeiro-Neto B (1999) Modern information retrieval. ACM Press, Addison-Wesley

    Google Scholar 

  7. Beard K, Sharma V (1997) Multidimensional ranking for data in digital spatial libraries. Int J Digit Libr 1(2): 153–160

    Article  Google Scholar 

  8. Belew RK, Hatton J (1996) Rave reviews: acquiring relevance assessments from multiple users. Working notes of the AAAI Spring symposium on Machine learning in information access, Standford, CA, USA

  9. Bierig R, Göker A (2006) Time, location and interest: an empirical and user-centred study. In: Proceedings of the 1st international conference on information interaction in context (IIiX). ACM, New York, pp 79–87

  10. Bilal D (2000) Children’s use of the yahooligans! web search engine: cognitive, physical, and affective behaviors on fact-based search tasks. J Am Soc Inf Sci 51(7): 646–665

    Article  Google Scholar 

  11. Borlund P (2003a) The concept of relevance in IR. J Am Soc Inf Sci Technol 54(10): 913–925

    Article  Google Scholar 

  12. Borlund P (2003b) The IIR evaluation model: a framework for evaluation of interactive information retrieval systems. J Inf Res 8(3): 152

    Google Scholar 

  13. Borlund P, Ingwersen P (1998) Measures of relative relevance and ranked half-life: performance indicators in interactive IR. In: Croft WB et al (eds) Proceedings of the 21st ACM SIGIR international conference on research and development, pp 324–331

  14. Bouvin NO, Christensen B, Gronbaeck K, Hansen FA (2003) Hycon: a framework for context-aware mobile hypermedia. Hypermedia 9(1): 59–88

    Article  Google Scholar 

  15. Buckley C, Voorhees EM (2004) Retrieval evaluation with incomplete information. In: Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval. ACM, New York, pp 25–32

  16. Budzik J, Hammond K (2000) User interactions with every day applications as context for just-in-time information access. In: Proceedings of the 5th international conference on intelligent user interfaces, pp 44–51

  17. Bylund M, Espinoza F (2002) Testing and demonstrating context-aware services with quake iii arena. Commun ACM 45(1): 46–48

    Article  Google Scholar 

  18. Bystrom K, Jarvelin K (1995) Task complexity affects information seeking and use. Inf Process Manag 31(2): 191–213

    Article  Google Scholar 

  19. Chandrasekaran P, Joshi A (2002) Mobileiq: a framework for mobile information access. In: Proceedings of the 3rd international conference on mobile data management. IEEE Computer Society, Washington, DC, USA, p 43

  20. Cheverst K, Davies N, Mitchell K, Friday A, Efstratiou C (2000) Developing a context-aware electronic tourist guide: some issues and experiences. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 17–24

  21. Chin DN (2001) Empirical evaluation of user models and user-adapted systems. User Model User-Adapted Interact 11(1-2): 181–194

    Article  MATH  Google Scholar 

  22. Chittaro, L. (eds) (2003) Human–computer interaction with mobile devices and services. Lecture Notes in Computer Science, vol 2795. Springer, Berlin

    Google Scholar 

  23. Clarcke C, Craswell N, Soboroff I (2004) Overview of the TREC 2004 terabyte track. In: The text retrieval conference. NIST

  24. Cleverdon C (1967) The cranfield test on index language devices. In: Aslib, pp 173–194

  25. Crestani F, Ruthven I (2007) Introduction to special issue on contextual information retrieval systems. Inf Retr 10: 829–847

    Article  MATH  Google Scholar 

  26. Daoud M, Tamine-Lechani L, Boughanem M (2008) Learning user interests for a session-based personalized search. In: Proceedings of the 2nd international IIiX symposium (IIiX’08), London, UK, pp 57–64

  27. Davies N, Mitchell K, Cheverest K, Blair G (1998) Developing a context sensitive tourist guide. In: First workshop on human computer interaction with mobile devices, GIST Technical Report G98-1

  28. Dervin B, Nilan M (1986) Information needs and uses. In: William ME (ed) ARIST, pp 3–33

  29. Dey AK (2001) Understanding and using context. Pers Ubiquitous Comput 5(1): 4–7

    Article  Google Scholar 

  30. Diaz A, Garcia A, Gervas P (2008) User-centred versus system-centred evaluation of a personalization system. Inf Process Manag 44(3): 1293–1307

    Article  Google Scholar 

  31. Ding C, Patra J (2007) User modeling for personalized web search with self-organizing map. J Am Soc Inf Sci Technol 58(4): 494–507

    Article  Google Scholar 

  32. Dumais S, Cuttrell E, Cadiz J, Jancke G, Sarin R, Robbins D (2003) Stuff i’ve seen: a system for a personal information retrieval and re-use. In: Proceedings of the 26th ACM SIGIR’, Toronto, pp 72–79

  33. Ellis D (1996) The dilemma of measurement in information retrieval research. J Am Soc Inf Sci 47(1): 23–36

    Article  Google Scholar 

  34. Frias-Martinez E, Chen SY, Macredie RD, Liu X (2007) The role of human factors in stereotyping behavior and perception of digital library users: a robust clustering approach. User Model User-Adapted Interact 17(3): 1391–1573

    Article  Google Scholar 

  35. Göker A, Myrhaug H (2008) Evaluation of a mobile information system in context. Inf Process Manag 44(1): 39–65

    Article  Google Scholar 

  36. Göker A, Myrhaug HI (2002) User context and personalisation. In: ECCBR workshop on case based reasoning and personalisation, Aberdeen

  37. Göker A, Watt S, Myrhaug H, Whitehead N, Yakici M, Bierig R, Kanth S, Cumming H (2004) An ambient, personalised, and context-sensitive information system for mobile users. In: Proceedings of the 2nd European union symposium on ambient intelligence (EUSAI), ACM, New York, NY, USA, pp 19–24

  38. Gowan JM (2003) A multiple model approach to personalised information access. Master Thesis in computer science, Faculty of science, University College Dublin

  39. Gwizdka J, Chignell M (1999) Towards information retrieval measures for evaluation of web search engines. Unpublished manuscript (1999). Available at http://www.imedia.mie.utoronto.ca/jacekg/pubs.html

  40. Harman D (1995) Overview of the 4th text retrieval conference (TREC-4). In: Proceedings of the 4th text retrieval conference (TREC-4). National Institute of Standards and Technology, NIST special publication, pp 1–24

  41. Harter S, Hert C (1997) Evaluation of information retrieval systems: approaches, issues, and methods. Ann Rev Inf Sci Technol 32(1): 3–94

    Google Scholar 

  42. Hattori S, Tezuka T, Tanaka K (2007) Context-aware query refinement for mobile web search. In: Proceedings of the 2007 international symposium on applications and the internet workshops, IEEE Computer Society, Washington, DC, USA, p 15

  43. Haveliwala T (2002) Topic-sensitive page rank. In: International ACM world wide web conference, pp 727–736

  44. Held A, Buchholz S, Schill A (2002) (n.d.) Modeling of context information for pervasive computing applications. In: Proceedings of the 6th world multiconference on systemics, cybernetics and informatics (SCI2002)

  45. Herlocker JL, Konstan JA, Terveen LG, Riedl JT (2004) Evaluating collaborative filtering recommender systems. ACM Trans Inf Syst 22(1): 5–53

    Article  Google Scholar 

  46. Hersh WR, Elliot DL, Hickam DH, Wolf SL, Molnar A (1995) Towards new measures of information retrieval evaluation. In: Proceedings of the 18th annual international ACM SIGIR conference on research and development in information retrieval, ACM, New York, NY, USA, pp 164–170

  47. Hupfer ME, Detlor B (2006) Gender and web information seeking: a self-concept orientation model. J Am Soc Inf Sci Technol 57(8): 1105–1115

    Article  Google Scholar 

  48. Ingwersen P (1994) Polyrepresentation of information needs and semantic entities: elements of a cognitive theory for information retrieval interaction. In: Proceedings of the 17th annual international ACM SIGIR conference on research and development in information retrieval. Springer, New York, pp 101–110

  49. Ingwersen P (1996) Cognitive perspectives of information retrieval interaction: Elements of a cognitive theory. J Doc 52(1): 3–50

    Article  Google Scholar 

  50. Ingwersen P, Jarvelin K (2005) The turn: integration of information seeking and information retrieval in context. Springer, Berlin

  51. Iqbal R, Sturm J, Kulyk O, Wang J, Terken J (2005) User-centred design and evaluation of ubiquitous services. In: Proceedings of the 23rd annual international conference on design of communication (SIGDOC ’05), ACM, New York, NY, USA, pp 138–145

  52. Jansen BJ, Booth DL, Spink A (2007a) Determining the user intent of web search engine queries. In: Proceedings of the 16th international conference on World Wide Web, ACM, New York, NY, USA, pp 1149–1150

  53. Jansen BJ, Booth DL, Spink A (2007b) Determining the user intent of web search engine queries. In: WWW ’07: proceedings of the 16th international conference on World Wide Web, ACM, New York, NY, USA, pp 1149–1150

  54. Jansen BJ, Booth DL, Spink A (2008) Determining the informational, navigational, and transactional intent of web queries. Inf Process Manag 44(3): 1251–1266

    Article  Google Scholar 

  55. Jarvelin K, Kekalainen J (2000) Ir Evaluation methods for highly relevant documents. In: Proceedings of the 23rd annual international ACM SIGIR conference on research and development in information retrieval, Belkin and al, pp 41–48

  56. Jarvelin K, Kekalainen J (2002) Cumulative gain-based evaluation of IR techniques. ACM Trans Inf Syst 20(4): 422–446

    Article  Google Scholar 

  57. Joachims T, Granka L, Hembrooke H, Radlinski F, Gay G (2007) Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Trans Inf Syst 25(2): 7

    Article  Google Scholar 

  58. John R, Mooney G (2001) Fuzzy user modelling for information retrieval on the world wide web. Knowl Inf Syst 3(1): 81–95

    Article  MATH  Google Scholar 

  59. Kekalainen J, Jarvelin K (2004) Evaluating information retrieval systems under the challenges of interaction and multidimensional dynamic relevance. In: Ingwersen P, Vakkari P (eds) Proceedings of the 4th CoLIS conference, pp 253–270

  60. Kelly D, Fu X (2007) Eliciting better information need descriptions from users of information search systems. Inf Process Manag 43(1): 30–46

    Article  Google Scholar 

  61. Kepler J (2005) Context-awarness in mobile tourism guides-a comprehensive survey. Technical Report, University. Linz, IFS/TK

  62. Kim K (2008) Effects of emotion control and task on web searching behavior. Inf Process Manag 44(1): 373–385

    Article  Google Scholar 

  63. Kim K, Allen B (2002) Cognitive and task influences on web searching behavior. J Am Soc Inf Sci Technol 53(2): 109–119

    Article  Google Scholar 

  64. Kjeldskov J, Graham C (2003) A review of mobile hci research methods. In: Proceedings mobile HCI, Lecture notes in computer science, pp 317–335

  65. Kraft R, Maghoul F, Chang C (2005) Y!q: contextual search at the point of inspiration. In: CIKM ’05: proceedings of the 14th ACM international conference on information and knowledge management, ACM Press, New York, NY, USA, pp 816–823

  66. Lang K (1995) NewsWeeder: learning to filter netnews. In: Proceedings of the 12th international conference on machine learning, Morgan Kaufmann, San Mateo, pp 331–339

  67. Law E, Klobucar T, Pipan M (2006) User effect in evaluating personalized information retrieval systems. In: EC-TEL, Springer, Berlin, pp 257–271

  68. Lee J, Hu X, Downie J (2005) Qa websites: rich research resources for contextualizing information retrieval behaviors. In: Proceedings of the 28th international SIGIR conference on research and development in information retrieval, Workshop on information retrieval in context, pp 33–366

  69. Leung CW, Chan SC, Chung F (2006) A collaborative filtering framework based on fuzzy association rules and multiple-level similarity. Knowl Inf Syst 10(3): 357–381

    Article  Google Scholar 

  70. Liu F, Yu C (2004) Personalized web search for improving retrieval effectiveness. IEEE Trans Knowl Data Eng 16(1): 28–40

    Article  Google Scholar 

  71. Marchionini G (1995) Information seeking in electronics environments. Cambridge university press, Cambridge Series on Human-Computer Interaction

  72. Micarelli A, Sicarrone F (2004) Anatomy and empirical evaluation of an daptive web-based information filtering system. User Model User-Adapted Interact 14(2-3): 159–200

    Article  Google Scholar 

  73. Mitchell T, Chen S, Macredie R (2005) Hypermedia learning and prior knowledge: domain expertise vs. system expertise. J Comput Assist Learn 21(12): 53–64

    Article  Google Scholar 

  74. Mizzaro S. (1998) How many relevances in information retrieval?. Interact Comput 10(3): 303–320

    Article  Google Scholar 

  75. Mobasher B (2007) Data mining for Web personalization. In: Brusilovsky P, Kobsa A, Nejdl W (eds) Lecture notes in computer science. Springer, Berlin

    Google Scholar 

  76. Mostafa J, Mukhopadhyay S, Palakal M (2003) Simulation studies of different dimensions of users’ interests and their impact on user modeling and information filtering. Inf Retr 6(2): 199–223

    Article  Google Scholar 

  77. Navarro-Prieto R, Scaife M, Rogers Y (2006) Cognitive strategies in web searching. In: 5th conference on human factors & the web

  78. Petrelli D (2008) On the role of user-centred evaluation in the advancement of interactive information retrieval. Inf Process Manag 44(1): 22–38

    Article  Google Scholar 

  79. Ramesh V, Glass L, Vessey I (2004) Research in computer science: an empirical study. J Syst Softw 70(1-2): 165–176

    Article  Google Scholar 

  80. Rigaux P (2002) Spatial database: with applications to GIS. Morgan Kauffmann, San Francisco

  81. Robertson S (2002) Comparing the performance of adaptive filtering and ranked output systems. Inf Retr 5(2-3): 257–268

    Article  Google Scholar 

  82. Robertson SE (1997) The probability ranking principle in IR. Readings in information retrieval, pp 281–286

  83. Robertson SE, Hancock-Beaulieu MM (1992) On the evaluation of IR systems. Inf Process Manag 28(4): 457–466

    Article  Google Scholar 

  84. Robertson S, Jones KS (1976) Relevance weighting for search terms. J Am Soc Inf Sci 27(3): 129–146

    Article  Google Scholar 

  85. Rocchio J (1971) Relevance feedback in information retrieval. In: Salton G (eds) The SMART retrieval system—experiments in automatic document processing, Prentice Hall, Englewood Cliffs

    Google Scholar 

  86. Ryan N, Pascoe J, Morse D (1997) Enhanced reality fieldwork: the context-aware archaeological assistant. In: Gaffney V, van Leusen M, Exxon S (eds) Computer Applications in Archeology

  87. Salton G (1971) The SMART information retrieval system. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  88. Schilit B, Adams N, Want R (1994) Context-aware computing applications. In: Proceedings of the workshop on mobile computing systems and applications, IEEE Computer Society, Santa Cruz, CA, pp 85–90

  89. Schilit BN, LaMarca A, Borriello G, Griswold WG, McDonald D, Lazowska E, Balachandran A, Hong J, Iverson V (2003) Ubiquitous location-aware computing and the place lab initiative challenge. In: WMASHE’03, The first ACM international workshop on wireless mobile applications and services on WLAN (WMASH 2003), San Diego, CA, September 19, 2003, ACM, New York, NY, USA

  90. Shamber L (1994) Relevance and information behaviour. In: William ME (ed) ARIST, pp 3–48

  91. Shen X, Tan B, Zhai C (2005a) Context-sensitive information retrieval using implicit feedback. In: Proceedings of the 28th annual international ACM SIGIR conference on research and development in information retrieval, ACM, New York, NY, USA, pp 43–50

  92. Shen X, Tan B, Zhai C (2005b) Implicit user modeling for personalized search. In: Proceedings of the 14th ACM international conference on Information and knowledge management, ACM, New York, NY, USA, pp 824–831

  93. Sieg A, Mobasher B, Burke R (2004) User’s information context: integrating user profiles and concept hierarchies. In: Proceedings of the 2004 meeting of the International Federation of Classification Societies

  94. Sieg A, Mobasher B, Burke R (2007) Web search personalization with ontological user profiles. In: Proceedings of the 16th ACM conference on conference on information and knowledge management, ACM, New York, NY, USA, pp 525–534

  95. Smyth B, Balfe E (2006) Anonymous personalization in collaborative web search. Inf Retr 9(2): 165–190

    Article  Google Scholar 

  96. Solomon P (1988) Children’s information retrieval behavior: a case analysis of opac. J Am Soc Inf Sci 44(5): 245

    Article  Google Scholar 

  97. Sonnenwald D, Pejtersen A (1994) Towards a framework to support information needs in design: a concurrent engineering example. Knowledge organization and mangement, pp 161–172

  98. Sormunen E (2002) Liberal relevance criteria of TREC-: counting on negligible documents? In: Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, ACM, New York, NY, USA, pp 324–330

  99. Sparck-Jones K, van Rijsbergen C (1976) Information retrieval test collections. J Doc 32(1): 59–72

    Article  Google Scholar 

  100. Speretta M, Gauch S (2005) Personalized search based on user search histories. In: Proceedings of the IEEE/WIC/ACM international conference on web intelligence, pp 622–628

  101. Su LT (1992) Evaluation measures for interactive information retrieval. Inf Process Manag 28(4): 503–516

    Article  Google Scholar 

  102. Tamine L, Boughanem M (2001) Applying heuristics to improve a genetic query optimisation process in information retrieval. In: 23 European colloquium on information retrieval, ECIR’2001, Darmstadt, Germany, 04/04/01-06/04/01, GMD-IPSI, pp 15–23

  103. Tamine L, Boughanem M, Zemirli WN (2008) Personalized document ranking: exploiting evidence from multiple user interests for profiling and retrieval. J Digit Inf Manag (in press)

  104. Tamine L, Chrisment C, Boughanem M (2003) Multiple query evaluation based on an enchanced genetic algorithm IP&M. Inf Process Manag 39(2):215–231

    Article  Google Scholar 

  105. Tan A, Ong H, Pan H, Ng J, Li Q (2004) Towards personalised web intelligence. Knowl Inf Syst 6(5): 595–616

    Article  Google Scholar 

  106. Tan B, Shen X, Zhai C (2006) Mining long-term search history to improve search accuracy. In: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, New York, NY, USA, pp 718–723

  107. Tao Y, Mamoulis N, Papadias D (2003) Validity information retrieval for spatio-temporal queries. In: SSTD, 2003

  108. Teevan J, Dumais S (2005) Personalizing search via automated analysis of interests and activities. In: Proceedings of the 28th international SIGIR conference on research and development in information retrieval, pp 449–456

  109. Tombros A, Ruthven I, Jose JM (2005) How users assess web pages for information seeking. J Am Soc Inf Sci Technol 56(4): 327–344

    Article  Google Scholar 

  110. Trajkova J, Gauch S (2004) Improving ontology-based user profiles. In: Proceedings of RIAO 2004, Vaucluse, France

  111. Turpin H, Hersh W (2001) Why batch and user evaluations do not give the same results. In: ACM conference on research and development in information retrieval, pp 225–231

  112. Uzuner O, Katz B, Yuret D (1999) Word sense disambiguation for information retrieval. In: AAAI ’99/IAAI ’99: proceedings of the sixteenth national conference on artificial intelligence and the eleventh innovative applications of artificial intelligence conference innovative applications of artificial intelligence, American Association for Artificial Intelligence, Menlo Park, CA, USA, p 985

  113. Vakkari P, Sormunen E (2004) The influence of relevance levels on the effectiveness of interactive information retrieval. J Am Soc Inf Sci Technol 55(11): 963–969

    Article  Google Scholar 

  114. Vieira V, Tedesco P, Salgado AC, Brézillon P (2007) Investigating the specifics of contextual elements management: the cemantika approach. In: CONTEXT, pp 493–506

  115. Voorhees E (2000) Variations in relevance judgments and the measurement of retrieval effectiveness. Inf Process Manag 36(1): 697–716

    Article  Google Scholar 

  116. Webb G, Pazzani M, Billsus D (2001) Machine learning for user modeling. User Model User-Adapted Interact 11(1-2): 19–29

    Article  MATH  Google Scholar 

  117. White R, Ruthven I, Jose J, van Rijsbergen C (2005) Evaluating implicit feedback models using searcher simulations. ACM Trans Inf Syst 23(3): 325–361

    Article  Google Scholar 

  118. Woodruff A, Plaunt C (1994) GIPSY: automated geographic indexing of text documents. J Am Soc Inf Sci 45(9): 645–655

    Article  Google Scholar 

  119. Wu X, Kumar V, Ross Quinlan J, Ghosh J, Yang Q, Motoda H, McLachlan GJ, Ng A, Liu B, Yu PS, Zhou Z-H, Steinbach M, Hand DJ, Steinberg D (2007) Top 10 algorithms in data mining. Knowl Inf Syst 14(1): 1–37

    Article  Google Scholar 

  120. Xie HI (2008) ‘Users’ evaluation of digital libraries (dls): their uses, their criteria, and their assessment. Inf Process Manag 44(3): 1346–1373

    Article  Google Scholar 

  121. Yau S, Huan L, Huang D, Yao Y (2003) Situation-aware personalized information retrieval for mobile internet. In: Proceedings of the 27th annual international computer software and applications conference (COMPSAC)’

  122. Yilmaz E, Aslam JA (2008) Estimating average precision when judgments are incomplete. Knowl Inf Syst 16(2): 173–211

    Article  Google Scholar 

  123. Zhou Y, Croft WB (2008) Measuring ranked list robustness for query performance prediction. Knowl Inf Syst 16(2): 155–171

    Article  Google Scholar 

  124. Zobel J (1998) How reliable are the results of large-scale information retrieval experiments?. In: In ACM conference on research and development in information retrieval, pp 307–314

Download references

Author information

Authors and Affiliations

  1. University of Paul Sabatier IRIT-SIG, 118 Route de Narbonne, 31062, Toulouse Cedex 9, France

    Lynda Tamine-Lechani, Mohand Boughanem & Mariam Daoud

Authors
  1. Lynda Tamine-Lechani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohand Boughanem
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mariam Daoud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lynda Tamine-Lechani.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tamine-Lechani, L., Boughanem, M. & Daoud, M. Evaluation of contextual information retrieval effectiveness: overview of issues and research. Knowl Inf Syst 24, 1–34 (2010). https://doi.org/10.1007/s10115-009-0231-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-009-0231-1

Keywords

Search

Navigation