Abstract
The performance of a Content-Based Image Retrieval (CBIR) system presented in the form of Precision-Recall or Precision-Scope graphs offers an incomplete overview of the system under study: the influence of the irrelevant items is obscured. In this paper, we propose a comprehensive and well normalized description of the ranking performance compared to the performance of an Ideal Retrieval System defined by ground-truth for a large number of predefined queries. We advocate normalization with respect to relevant class size and restriction to specific normalized scope values. We also propose new performance graphs for total recall studies in a range of embeddings.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Salton, G.: The SMART retrieval system. Prentice-Hall, Englewood Cliffs (1971)
van Rijsbergen, C.: Information Retrieval. Butterworths, London (1979)
Voorhees, E.M., Harman, D.: Proc. TREC (1999)
MPEG-7. Special Issue of IEEE Trans. Circuits and Systems for Video Technology 11 (2001)
Porkaew, K., Chakrabarti, K., Mehrotra, S.: Query refinement for multimedia similarity retrieval in MARS. ACM Multimedia, 235–238 (1999)
Vasconcelos, N., Lippman, A.: A probabilistic architecture for content-based image retrieval. In: CVPR, pp. 216–221 (2000)
Baumgarten, C.: A probabilistic solution to the selection and fusion problem in distributed information retrieval. In: SIGIR, pp. 246–253 (1999)
Müller, H., Müller, W., Squire, D.M., Marchand-Maillet, S., Pun, T.: Performance evaluation in content-base image retrieval: Overview and proposals. Pattern Recog. Letters 22, 593–601 (2001)
Müller, H., Marchand-Maillet, S., Pun, T.: The truth about Corel - Evaluation in image retrieval. In: Lew, M., Sebe, N., Eakins, J.P. (eds.) CIVR 2002. LNCS, vol. 2383, pp. 38–49. Springer, Heidelberg (2002)
Huijsmans, D., Sebe, N.: Extended performance graphs for cluster retrieval. In: CVPR, pp. 26–31 (2001)
Huijsman, D.P., Sebe, N.: How to complete performance graphs in content-based image retrieval: Add generality and normalize scope. IEEE Trans. on PAMI (2004) (to appear)
Gokhale, D., Kullback, S.: The Information in Contingency Tables. In: Dekker, M, ed. (1978)
van Bemmel, J.H., Musen, M.A.: Handbook of Medical Informatics. Springer, Heidelberg (1997)
Huijsmans, D., Sebe, N.: Content-based indexing performance: A class size normalized precision, recall, generality evaluation. In: ICIP, pp. 733–736 (2003)
Tague-Sutcliffe, J.: The pragmatics of information retrieval experimentation, revisited. Information Processing and Management 28, 467–490 (1992)
Raghavan, V., Bollmann, P., Jung, G.: A critical investigation of recall and precision as measures of retrieval system performance. ACM Trans. Information Systems 7, 205–229 (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sebe, N., Huijsmans, D.P., Tian, Q., Gevers, T. (2004). Complete Performance Graphs in Probabilistic Information Retrieval. In: Aizawa, K., Nakamura, Y., Satoh, S. (eds) Advances in Multimedia Information Processing - PCM 2004. PCM 2004. Lecture Notes in Computer Science, vol 3332. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30542-2_29
Download citation
DOI: https://doi.org/10.1007/978-3-540-30542-2_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23977-2
Online ISBN: 978-3-540-30542-2
eBook Packages: Computer ScienceComputer Science (R0)