Skip to main content
SpringerLink
Log in

What is the difference? A cognitive dissimilarity measure for information retrieval result sets

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Result rankings from context-aware information retrieval are inherently dynamic, as the same query can lead to significantly different outcomes in different contexts. For example, the search term Digital Camera will lead to different—albeit potentially overlapping—results in the contexts customer reviews and shops, respectively. The comparison of such result rankings can provide useful insights into the effects of context changes on the information retrieval results. In particular, the impact of single aspects of the context in complex applications can be analyzed to identify the most (and least) influential context parameters. While a multitude of methods exists for assessing the relevance of a result ranking with respect to a given query, the question how different two result rankings are from a user’s point of view has not been tackled so far. This paper introduces DIR, a cognitively plausible dissimilarity measure for information retrieval result sets that is based solely on the results and thus applicable independently of the retrieval method. Unlike statistical correlation measures, this dissimilarity measure reflects how human users quantify the changes in information retrieval result rankings. The DIR measure supports cognitive engineering tasks for information retrieval, such as work flow and interface design: using the measure, developers can identify which aspects of context heavily influence the outcome of the retrieval task and should therefore be in the focus of the user’s interaction with the system. The cognitive plausibility of DIR has been evaluated in two human participants tests, which demonstrate a strong correlation with user judgments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Agichtein E, Brill E, Dumais S, Ragno R (2006) Learning user interaction models for predicting web search result preferences. In: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM Press, New York, NY, USA, pp 3–10

  2. Albertoni R, De Martino M (2008) Asymmetric and context-dependent semantic similarity among ontology instances. J Data Semant Lect Notes Comput Sci 4900: 1–30

    Article  Google Scholar 

  3. Baeza-Yates R, Ribeiro-Neto B (1999) Modern information retrieval. Addison Wesley, Boston

    Google Scholar 

  4. Bazire M, Brézillon P (2005) Understanding context before using it. In: Dey AK, Kokinov B, Leake D, Turner R (eds) Modeling and using context—5th international and interdisciplinary conference (CONTEXT 2005), Paris, France. Lecture notes in computer science, vol 3554. Springer, Berlin, pp 29–40

    Google Scholar 

  5. Bikakis A, Antoniou G, Hasapis P (2010) Strategies for contextual reasoning with conflicts in ambient intelligence. Knowl Info Syst

  6. Brown PJ, Jones GJF (2001) Context-aware retrieval: exploring a new environment for information retrieval and information filtering. Pers Ubiquit Comput 5: 253–263

    Article  Google Scholar 

  7. Dey A (2001) Understanding and using Context. Pers Ubiquit Comput 5(1): 4–7

    Article  Google Scholar 

  8. Efron M (2009) Using multiple query aspects to build test collections without human relevance judgments. In: Boughanem M, Berrut C, Mothe J, Soule-Dupuy C (eds) ECIR ’09: proceedings of the 31th European conference on IR research on advances in information retrieval. Lecture notes in computer science, vol 5478. Springer, pp 276–287

  9. Finkelstein L, Gabrilovich E, Matias Y, Rivlin E, Solan Z, Wolfman G, Ruppin E (2001) Placing search in context: the concept revisited. In: WWW ’01: proceedings of the 10th international conference on World Wide Web. ACM Press, New York, NY, USA, pp 406–414

  10. Gärdenfors P (2000) Conceptual spaces: the geometry of thought. MIT Press, Cambridge

    Google Scholar 

  11. Goldstone R, Son J (2004) Similarity. In: Holyoak K, Morrison R (eds) Cambridge handbook of thinking and reasoning. Cambridge University Press, Cambridge

    Google Scholar 

  12. Harrison S (1995) A comparison of still, animated, or nonillustrated on-line help with written or spoken instructions in a graphical user interface. In: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM Press/Addison- Wesley Publishing Co., New York, NY, USA, pp 82–89

  13. Hu J, Chan P (2008) Personalized web search by using learned user profiles in re-ranking. In: Workshop on knowledge discovery on the web, KDD conference. pp 84–97

  14. Janowicz K (2008) Kinds of contexts and their impact on semantic similarity measurement. In: 5th IEEE workshop on context modeling and reasoning (CoMoRea) at the 6th IEEE international conference on pervasive computing and communication (PerComa’08)

  15. Janowicz K, Keßler C, Panov I, Wilkes M, Espeter M, Schwarz M (2008) A study on the cognitive plausibility of SIM-DL similarity rankings for geographic feature types. In: Bernard L, Friis-Christensen A, Pundt H (eds) The European information society—taking geoinformation science one step further (AGILE 2008 proceedings). Lecture notes in geoinformation and cartography. Springer, Berlin, pp 115–134

    Google Scholar 

  16. Janowicz K, Keßler C, Schwarz M, Wilkes M, Panov I, Espeter M, Bäumer B (2007) Algorithm, implementation and application of the SIM-DL similarity server. In: Fonseca F, Rodríguez M (eds) Second international conference on geoSpatial semantics, GeoS 2007. Lecture notes in computer science, vol 4853. Springer, Berlin, pp 128–145

  17. Kendall MG (1938) A new measure of rank correlation. Biometrika 30(12)

  18. Kent A, Berry MM, Fred J, Luehrs U, Perry JW (1955) Machine literature searching VIII. Operational criteria for designing information retrieval systems. Am Document 6(2): 93–101

    Article  Google Scholar 

  19. Keßler C (2007) Similarity measurement in context. In: Kokinov B, Richardson D, Roth-Berghofer T, Vieu L (eds) 6th international and interdisciplinary conference, CONTEXT 2007, Roskilde, Denmark. Lecture notes in artificial intelligence, vol 4635. Springer, Berlin, pp 277–290

    Google Scholar 

  20. Keßler C, Raubal M, Janowicz K (2007) The effect of context on semantic similarity measurement. In: Meersman R, Tari Z, Herrero P (eds) On the move—OTM 2007 workshops, part II. Lecture notes in computer science, vol 4806. Springer, Berlin, pp 1274–1284

    Google Scholar 

  21. Keßler C, Raubal M, Wosniok C (2009) Semantic rules for context-aware geographical information retrieval. In: Barnaghi P, Moessner K, Presser M, Meissner S (eds) Smart sensing and context, 4th European conference, EuroSSC 2009, Guildford, UK, September 2009. Lecture notes in computer science, vol 5741. Springer-Verlag, Berlin, pp 77–92

  22. Kim HR, Chan PK (2008) Learning implicit user interest hierarchy for context in personalization. Appl Intell 28(2): 153–166

    Article  Google Scholar 

  23. Kokinov B, Richardson D, Roth-Berghofer T, Vieu L (eds) (2007) Modeling and using context 6th international and interdisciplinary conference CONTEXT 2007, Roskilde, Denmark, 20–24, August, 2007, proceedings. Lecture notes in artificial intelligence, vol 4635. Springer, Berlin

  24. Kraft R, Chang CC, Maghoul F, Kumar R (2006) Searching with context. In: WWW ’06: proceedings of the 15th international conference on World Wide Web. ACM Press, New York, NY, USA, pp 477–486

  25. Leonidis A, Baryannis G, Fafoutis X, Korozi M, Gazoni N, Dimitriou M, Koutsogiannaki M, Boutsika A, Papadakis M, Papagiannakis H, Tesseris G, Voskakis E, Bikakis A, Antoniou G (2009) Alertme: a semantics-based context-aware notification system. In: 33rd annual IEEE international computer software and applications conference, pp 200–205

  26. Manjunath BS, Ohm JR, Vasudevan VV, Yamada A (2001) Color and texture descriptors. IEEE Trans Circuit Syst Video Technol 11(6): 703–715

    Article  Google Scholar 

  27. Marchionini G (2006) Toward human-computer information retrieval. June/July 2006 bulletin of the American society for information science, available online at http://www.asis.org/Bulletin/Jun-06/marchionini.html

  28. Medin D, Goldstone R, Gentner D (1993) Respects for similarity. Psychol Rev 100(2): 254–278

    Article  Google Scholar 

  29. Melucci M (2008) A basis for information retrieval in context. ACM Trans Info Syst 26(3): 1–41

    Article  Google Scholar 

  30. Melucci M, Pretto M (2007) PageRank: when order changes. Lecture notes in computer science. Springer, Berlin, pp, pp 581–588

    Google Scholar 

  31. Meza BA, Halaschek C, Arpinar BI, Sheth A (2003) Context-aware semantic association ranking. In: Semantic web and databases workshop proceedings. Berlin, Germany, pp 33–50

  32. Miller GA (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 63: 81–97

    Article  Google Scholar 

  33. Nedas KA, Egenhofer MJ (2008) Integral vs. separable attributes in spatial similarity assessments. In: Proceedings of the international conference on spatial cognition VI. Springer, Berlin, pp 295–310

  34. Page L, Brin S, Motwani R, Winograd T (1998) The PageRank citation ranking: bringing order to the web. Technical report. Digital Library Technologies Project, Stanford

  35. Pfitzner D, Leibbrandt R, Powers D (2009) Characterization and evaluation of similarity measures for pairs of clusterings. Knowl Info Syst 19(3): 361–394

    Article  Google Scholar 

  36. Raubal M (2004) Formalizing conceptual spaces. In: Vieu LVA (eds) Formal ontology in information systems, proceedings of the 3rd international conference (FOIS 2004), Frontiers in artificial intelligence and applications. IOS Press, Amsterdam, NL, pp 153–164

  37. Rissland E (2006) AI and similarity. IEEE Intell Syst 21(3): 39–49

    Article  Google Scholar 

  38. Robertson SE (1997) The probability ranking principle in IR. pp 281–286

  39. Rodgers JL, Nicewanderer WA (1988) Thirteen ways to look at the correlation coefficient. Am Stat 42(1): 59–66

    Article  Google Scholar 

  40. Rodríguez A, Egenhofer MJ (2004) Comparing geospatial entity classes: an asymmetric and context-dependent similarity measure. Int J Geo Info Sci 18(3): 229–256

    Article  Google Scholar 

  41. Rose DE, Levinson D (2004) Understanding user goals in web search. In: Feldman S, Uretsky M (eds) WWW ’04: Proceedings of the 13th international conference on World Wide Web. ACM Press, pp 13–19

  42. Rosset S, Perlich C, Zadrozny B (2007) Ranking-based evaluation of regression models. Knowl Info Syst 12(3): 331–353

    Article  Google Scholar 

  43. Schwering A (2008) Approaches to semantic similarity measurement for Geo-spatial data—a survey. Trans GIS 12(1): 5–12

    Article  Google Scholar 

  44. Spearman C (1904) The proof and measurement of association between two things. Am J psychol 15: 72–101

    Article  Google Scholar 

  45. Spink, A, Cole, C (eds) (2005) New directions in cognitive information retrieval. Springer, Netherlands

    MATH  Google Scholar 

  46. Strang T, Linnhoff-Popien C (2004) A context modeling survey. In: First international workshop on advanced context modelling, reasoning and management at UbiComp 2004, Nottingham, England, 7 September, 2004

  47. Strube G (1992) The role of cognitive science in knowledge engineering. In: Proceedings of the first joint workshop on contemporary knowledge engineering and cognition. Lecture notes in computer science, vol 622. Springer, pp 161–174

  48. Tamine-Lechani L, Boughanem M, Daoud M (2009) Evaluation of contextual information retrieval effectiveness: overview of issues and research. Knowl Info Syst

  49. Ukkonen A, Castillo C, Donato D, Gionis A (2008) Searching the wikipedia with contextual information. In: CIKM ’08: proceeding of the 17th ACM conference on information and knowledge mining. ACM press, New York, NY, USA, pp 1351–1352

  50. van Rijsbergen CJ (1979) Information retrieval, 2 edn. Butterworth.

  51. Wang D, Tse Q, Zhou Y (2009) A decentralized search engine for dynamic web communities. Knowl Info Syst

  52. Wang J (2009) Mean-Variance Analysis: A New Document Ranking Theory in Information Retrieval. In: Boughanem M, Berrut C, Mothe J, Soule-Dupuy C (eds) ECIR ’09: Proceedings of the 31th European conference on IR research on advances in information retrieval. Lecture notes in computer science, vol 5478. Springer, pp 4–16

  53. Weerkamp W, Balog K, de Rijke M (2009) Using contextual information to improve search in email archives. In: Advances in information retrieval. 31st European conference on information retrieval conference (ECIR 2009). pp 400–411

  54. Wilkes M (2008) A graph-based alignment approach to context-sensitive similarity between climbing routes. Diploma thesis, Institute for Geoinformatics, University of Münster, Germany

  55. Wu G, Chang EY, Panda N (2005) Formulating context-dependent similarity functions. In: MULTIMEDIA ’05: proceedings of the 13th annual ACM international conference on multimedia. ACM press, New York, NY, USA, pp 725–734

  56. Yang J, Cheung W, Chen X (2009) Learning element similarity matrix for semi-structured document analysis. Knowl Info Syst 19(1): 53–78

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Geoinformatics, University of Münster, Münster, Germany

    Carsten Keßler

Authors
  1. Carsten Keßler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carsten Keßler.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Keßler, C. What is the difference? A cognitive dissimilarity measure for information retrieval result sets. Knowl Inf Syst 30, 319–340 (2012). https://doi.org/10.1007/s10115-011-0382-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-011-0382-8

Keywords

Search

Navigation