Abstract
Automatic methods for archiving, indexing and retrieving multimedia content become more and more important through the steadily increasing amount of digital data in the web and at home. THESEUS, a German research program, focuses on developing sophisticated algorithms and evaluation strategies for the automated processing of digital data. In this paper we present how evaluation is performed in THESEUS and introduce a generic framework for the evaluation of various video and image analysis algorithms. Besides, evaluation campaigns like the Cross Evaluation Language Forum (CLEF) and subprojects like ImageCLEF deal with the evaluation of such algorithms and provide an objective comparison of their performance. We relate the THESEUS tasks to the work done in ImageCLEF and propose a new task for ImageCLEF 2009.
This work has been supported by the German THESEUS program.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Müller, H., Geissbuhler, A., Marchand-Maillet, S., Clough, P.: Benchmarking image retrieval applications. In: Proc. of the 10th Intern. Conf. Distributed Multimedia Systems, Workshop on Visual Information Systems, San Francisco (2004)
Smeaton, A.F., Over, P., Kraaij, W.: Evaluation campaigns and TRECVid. In: MIR 2006: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, pp. 321–330. ACM Press, New York (2006)
Müller, H., Marchand-Maillet, S., Pun, T.: The Truth about Corel-Evaluation in Image Retrieval. In: Lew, M., Sebe, N., Eakins, J.P. (eds.) CIVR 2002. LNCS, vol. 2383, pp. 38–49. Springer, Heidelberg (2002)
Datta, R., Joshi, D., Li, J., Wang, J., Surveys, A.: Image Retrieval: Ideas, Influences, and Trends of the New Age. ACM Computing Surveys 40(2) (2008)
Huiskes, M.J., Lew, M.S.: The MIR Flickr Retrieval Evaluation. In: MIR 2008: Proceedings of the 2008 ACM International Conference on Multimedia Information Retrieval. ACM, New York (2008)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. In: Proc. 8th Int. Conf. Computer Vision, vol. 2, pp. 416–423 (2001)
Huang, Q., Dom, B.: Quantitative methods of evaluating image segmentation. In: IEEE International Conference on Image Processing, vol. 3, pp. 53–56 (1995)
Deselaers, T., Hanbury, A.: The Visual Concept Detection Task in ImageCLEF 2008. In: Peters, C., et al. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 531–538. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nowak, S., Dunker, P., Paduschek, R. (2009). THESEUS Meets ImageCLEF: Combining Evaluation Strategies for a New Visual Concept Detection Task 2009. In: Peters, C., et al. Evaluating Systems for Multilingual and Multimodal Information Access. CLEF 2008. Lecture Notes in Computer Science, vol 5706. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04447-2_82
Download citation
DOI: https://doi.org/10.1007/978-3-642-04447-2_82
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04446-5
Online ISBN: 978-3-642-04447-2
eBook Packages: Computer ScienceComputer Science (R0)