Skip to main content
SpringerLink
Log in
Book cover

Information Retrieval Evaluation in a Changing World pp 565–582Cite as

Visual Analytics and IR Experimental Evaluation

  • Chapter
  • First Online:

  • 687 Accesses

Part of the book series: The Information Retrieval Series ((INRE,volume 41))

Abstract

We investigate the application of Visual Analytics (VA) techniques to the exploration and interpretation of Information Retrieval (IR) experimental data. We first briefly introduce the main concepts about VA and then we present some relevant examples of VA prototypes developed for better investigating IR evaluation data. Finally, we conclude with an discussion of the current trends and future challenges on this topic.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   99.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Andrienko G, Andrienko N, Jankowski P, Keim DA, Kraak MJ, MacEachren A, Wrobel S (2007) Geovisual analytics for spatial decision support: setting the research agenda. Int J Geogr Inf Sci 21(8):839–858

    Article  Google Scholar 

  • Angelini M, Ferro N, Santucci G, Silvello G (2012) Visual interactive failure analysis: supporting users in information retrieval evaluation. In: Kamps J, Kraaij W, Fuhr N (eds) Proceedings of 4th symposium on information interaction in context (IIIX 2012). ACM Press, New York, pp 195–203

    Google Scholar 

  • Angelini M, Ferro N, Santucci G, Silvello G (2014) VIRTUE: a visual tool for information retrieval performance evaluation and failure analysis. J Vis Lang Comput 25(4):394–413

    Article  Google Scholar 

  • Angelini M, Ferro N, Santucci G, Silvello G (2016a) A visual analytics approach for what-if analysis of information retrieval systems. In: Perego R, Sebastiani F, Aslam J, Ruthven I, Zobel J (eds) Proceedings of 39th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2016). ACM Press, New York, pp 1081–1084

    Google Scholar 

  • Angelini M, Ferro N, Santucci G, Silvello G (2016b) What-if analysis: a visual analytics approach to information retrieval evaluation. In: Di Nunzio GM, Nardini FM, Orlando S (eds) Proceedings of 7th Italian information retrieval workshop (IIR 2016). CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1653/

  • Angelini M, Ferro N, Santucci G, Silvello G (2017) Visual analytics for information retrieval evaluation campaigns. In: Sedlmair M, Tominski C (eds) Proceedings of 8th international workshop on visual analytics (EuroVA 2017). Eurographics Association, Goslar, pp 25–29

    Google Scholar 

  • Angelini M, Fazzini V, Ferro N, Santucci G, Silvello G (2018) CLAIRE: a combinatorial visual analytics system for information retrieval evaluation. Inf Process Manag 54(6), 1077–1100

    Article  Google Scholar 

  • Banks D, Over P, Zhang NF (1999) Blind men and elephants: six approaches to TREC data. Inf Retriev 1(1–2):7–34

    Article  Google Scholar 

  • Behrisch M, Davey J, Simon S, Schreck T, Keim D, Kohlhammer J (2013) Visual comparison of orderings and rankings. In: Pohl M, Schumann H (eds) Proceedings of 4th international workshop on visual analytics (EuroVA 2013). Eurographics Association, Goslar

    Google Scholar 

  • Buckley C, Voorhees EM (2005) Retrieval system evaluation. In: Harman DK, Voorhees EM (eds) TREC. experiment and evaluation in information retrieval. MIT Press, Cambridge, pp 53–78

    Google Scholar 

  • Card SK, Mackinlay JD, Shneiderman B (1999) Readings in information visualization: using vision to think. Morgan Kaufmann Publishers, San Francisco, CA

    Google Scholar 

  • Chen C (2004) Information visualization - beyond the horizon. Springer, London

    Google Scholar 

  • Cleverdon CW (1967) The cranfield tests on index languages devices. Aslib Proc 19(6):173–194

    Article  Google Scholar 

  • Crestani F, Vegas J, de la Fuente P (2004) A graphical user interface for the retrieval of hierarchically structured documents. Inf Process Manag 40(2):269–289

    Article  Google Scholar 

  • Derthick M, Christel MG, Hauptmann AG, Wactlar HD (2003) Constant density displays using diversity sampling. In: Munzner T, North S (eds) Proceedings of 9th IEEE symposium on information visualization (INFOVIS 2003). IEEE Computer Society, Los Alamitos, pp 137–144

    Google Scholar 

  • Ferro N, Harman D (2010) CLEF 2009: Grid@CLEF pilot track overview. In: Peters C, Di Nunzio GM, Kurimo M, Mandl T, Mostefa D, Peñas A, Roda G (eds) Multilingual information access evaluation vol. I text retrieval experiments – tenth workshop of the cross–language evaluation forum (CLEF 2009). Revised selected papers. Lecture notes in computer science (LNCS), vol 6241. Springer, Heidelberg, pp 552–565

    Google Scholar 

  • Ferro N, Silvello G (2016) A general linear mixed models approach to study system component effects. In: Perego R, Sebastiani F, Aslam J, Ruthven I, Zobel J (eds) Proceedings of 39th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2016). ACM Press, New York, pp 25–34

    Google Scholar 

  • Fowler RH, Lawrence-Fowler WA, Wilson BA (1991) Integrating query, thesaurus, and documents through a common visual representation. In: Fox EA (ed) Proceedings of 14th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 1991). ACM Press, New York, pp 142–151

    Google Scholar 

  • Hearst MA (2009) Search user interfaces, 1st edn. Cambridge University Press, New York

    Book  Google Scholar 

  • Hearst MA (2011) “Natural” search user interfaces. Commun ACM 54(11):60–67

    Article  Google Scholar 

  • Inselberg A (2009) Parallel coordinates. Visual multidimensional geometry and its applications. Springer, New York

    MATH  Google Scholar 

  • Ioannakis G, Koutsoudis A, Pratikakis I, Chamzas C (2018) Retrieval–an online performance evaluation tool for information retrieval methods. IEEE Trans Multimedia 20(1):119–127

    Article  Google Scholar 

  • Järvelin K, Kekäläinen J (2002) Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst 20(4):422–446

    Article  Google Scholar 

  • Keim DA (2001) Visual exploration of large data sets. Commun ACM 44(8):38–44

    Article  Google Scholar 

  • Keim DA, Mansmann F, Schneidewind J, Ziegler H (2006) Challenges in visual data analysis. In: Banissi E (ed) Proceedings of the 10th international conference on information visualization (IV 2006). IEEE Computer Society, Los Alamitos, pp 9–16

    Google Scholar 

  • Keim DA, Kohlhammer J, Ellis G, Mansmann F (eds) (2010) Mastering the information age – solving problems with visual analytics. Eurographics Association, Goslar

    Google Scholar 

  • Koshman S (2005) Testing user interaction with a prototype visualization-based information retrieval system. J Am Soc Inf Sci Technol 56(8):824–833

    Article  Google Scholar 

  • Lipani A, Lupu M, Hanbury A (2017) Visual pool: a tool to visualize and interact with the pooling method. In: Kando N, Sakai T, Joho H, Li H, de Vries AP, White RW (eds) Proceedings of 40th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2017). ACM Press, New York, pp 1321–1324

    Google Scholar 

  • McGill R, Tukey JW, Larsen WA (1978) Variations of box plots. Am Stat 32(1):12–16

    Google Scholar 

  • Moffat A, Zobel J (2008) Rank-biased precision for measurement of retrieval effectiveness. ACM Trans Inf Syst (TOIS) 27(1):2:1–2:27

    Article  Google Scholar 

  • Morse EL, Lewis M, Olsen KA (2002) Testing visual information retrieval methodologies case study: comparative analysis of textual, icon, graphical, and spring displays. J Am Soc Inf Sci Technol 53(1):28–40

    Article  Google Scholar 

  • Rocco G, Silvello G (2019) An InfoVis tool for interactive component-based evaluation. arXivorg, information retrieval (csIR) arXiv:1901.11372

    Google Scholar 

  • Sankey HR (1898) Introductory note on the thermal efficiency of steam-engines. Report of the committee appointed on the 31st March, 1896, to consider and report to the council upon the subject of the definition of a standard or standards of thermal efficiency for steam-engines: with an introductory note. Minutes of proceedings of the institution of civil engineers, vol 134, pp 278–283 including Plate 5

    Google Scholar 

  • Schmidt M (2008) The sankey diagram in energy and material flow management. J Ind Ecol 12(1):82–94

    Article  Google Scholar 

  • Seo J, Shneiderman B (2005) A rank-by-feature framework for interactive exploration of multidimensional data. Inf Vis 4(2):96–113

    Article  Google Scholar 

  • Sormunen E, Hokkanen S, Kangaslampi P, Pyy P, Sepponen B (2002) Query performance analyser – a web-based tool for IR research and instruction. In: Järvelin K, Beaulieu M, Baeza-Yates R, Hyon Myaeng S (eds) Proceedings of 25th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2002). ACM Press, New York, p 450

    Google Scholar 

  • Spence R (2007) Information visualization: design for interaction, 2nd edn. Pearson Education Limited, London

    Google Scholar 

  • Ware C (2012) Information visualization - perception for design, 3rd edn. Morgan Kaufmann Publishers, San Francisco

    Google Scholar 

  • Wong PC, Thomas JJ (2004) Visual analytics - guest editors’ introduction. IEEE Comput Graph Appl 24(5):20–21

    Article  Google Scholar 

  • Zhang J (2001) TOFIR: a tool of facilitating information retrieval - introduce a visual retrieval model. Inf Process Manag 37(4):639–657

    Article  Google Scholar 

  • Zhang J (2008) Visualization for information retrieval. Springer, Heidelberg

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Engineering, University of Padua, Padova, Italy

    Nicola Ferro

  2. Department of Computer, Control, and Management Engineering “Antonio Ruberti”, Sapienza University of Rome, Rome, Italy

    Giuseppe Santucci

Authors
  1. Nicola Ferro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giuseppe Santucci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicola Ferro .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, Università degli Studi di Padova , Padova, Italy

    Nicola Ferro

  2. Consiglio Nazionale delle Ricerche, Istituto di Scienza e Tecnologie dell’Informazione, Pisa, Italy

    Carol Peters

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Ferro, N., Santucci, G. (2019). Visual Analytics and IR Experimental Evaluation. In: Ferro, N., Peters, C. (eds) Information Retrieval Evaluation in a Changing World. The Information Retrieval Series, vol 41. Springer, Cham. https://doi.org/10.1007/978-3-030-22948-1_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-22948-1_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-22947-4

  • Online ISBN: 978-3-030-22948-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation