Skip to main content
Springer Nature Link
Log in

Robust content-based image indexing using contextual clues and automatic pseudofeedback

  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract.

In this paper we present a robust information integration approach to identifying images of persons in large collections such as the Web. The underlying system relies on combining content analysis, which involves face detection and recognition, with context analysis, which involves extraction of text or HTML features. Two aspects are explored to test the robustness of this approach: sensitivity of the retrieval performance to the context analysis parameters and automatic construction of a facial image database via automatic pseudofeedback. For the sensitivity testing, we reevaluate system performance while varying context analysis parameters. This is compared with a learning approach where association rules among textual feature values and image relevance are learned via the CN2 algorithm. A face database is constructed by clustering after an initial retrieval relying on face detection and context analysis alone. Experimental results indicate that the approach is robust for identifying and indexing person images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alp Aslandogan Y, Yu C (2000a) Experiments in using visual and textual clues for image hunting on the Web. In: Proceedings of VISUAL 2000, Lyon, France, November 2000, pp 108-119

  2. Alp Aslandogan Y, Yu C (2000b) Multiple evidence combination in image retrieval: Diogenes searches for people on the Web. In: Proceedings of ACM SIGIR 2000, Athens, Greece, July 2000, pp 88-95

  3. Brill E (1994) Some advances in transformation-based part of speech tagging. In: Proceedings of the 12th national conference on artificial intelligence, 1994, pp 722-727

  4. Cox I, Miller M, Omohundro S, Yianilos P (1996) Pichunter: Bayesian relevance feedback for image retrieval. In: Proceedings of the international conference on pattern recognition, Vienna, Austria, 25-30 August 1996, 3:361-369

  5. Dorai C, Venkatesh S (2001) Bridging the semantic gap in content management systems: computational media aesthetics. In: Proceedings of COSIGN 2001: computational semiotics for games and new media, Amsterdam, 10-12 September 2001, pp 33-52

  6. Faloutsos C, Barber R, Flickner M, Hafner J, Niblack W Petkovic D, Equitz W (1994) Efficient and effective querying by image content. J Intell Inf Sys 3(1):231-262

    Google Scholar 

  7. Golomb BA, Lawrence DT, Sejnowski TJ (1991) Sexnet: a neural network identifies sex from human faces. In: Lippman RP, Moody J, Touretzky DS (eds) Advances in neural information processing systems, vol 3. Morgan Kaufmann, San Mateo, CA, pp 572-577

  8. Gose E, Johnsonbaugh R, Jost S (1996) Pattern recognition and image analysis. Artech House/Prentice-Hall, Upper Saddle River, NJ

  9. Grosky WI, Zaho R (2001) Negotiating the semantic gap: from feature maps to semantic landscapes. In: Proceedings of the conference on current trends in theory and practice of informatics, location, date 2001, pp 33-52. http://citeseer.nj.nec.com/466769.html

  10. Hall DL (1992) Mathematical techniques in multisensor data fusion. Artech House, Norwood, MA

  11. Jain R, Hampapur A (1994) Metadata in video databases. SIGMOD Rec 23(4):27-33

    MATH  Google Scholar 

  12. Jose JM, Furner J, Harper DJ (1998) Spatial querying for image retrieval: a user oriented evaluation. In: Proceedings of ACM SIGIR, Melbourne, Australia, 24-28 August 1998, pp 232-240

  13. Katz A, Thrift P (1993) Hybrid neural network classifiers for automatic target detection Expert Sys 10(4):243-251

    Google Scholar 

  14. LaCascia M, Sethi S, Sclaroff S (1998) Combining textual and visual cues for content-based image retrieval on the World Wide Web. In: Proceedings of the IEEE workshop on content-based access of image and video libraries, Santa Barbara, CA, June 1998, pp 24-29

  15. Meilhac C, Nastar C (1999) Relevance feedback and category search in image databases. In: Proceedings of the IEEE international conference on multimedia computing and systems, Florence, Italy, June 1999, 1:512-517

  16. Mitchell TM (1996) Machine learning. McGraw-Hill, New York

  17. Moghaddam B, Pentland A (1994) Face recognition using view-based and modular eigenspaces. In: Proceedings of the conference on automatic systems for the identification and inspection of humans, SPIE, San Diego, July 1994, 2277:12-21

  18. Mu X, Artiklar M, Artiklar M, Hassoun M, Watta P (2001) Training algorithms for robust face recognition using a template-matching approach. In: Proceedings of the international joint conference on neural networks (IJCNN’01), Washington, DC, July 2001

  19. Mukherjea S, Hirata K, Hara Y (1999) AMORE: A World Wide Web image retrieval engine. World Wide Web 2(3):115-132

    Article  Google Scholar 

  20. Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Patt Anal Mach Intell 20(1):23-38

    Article  Google Scholar 

  21. Rui Y, Huang T, Mehrotra S (1997) Content-based image retrieval with relevance feedback in mars. In: Proceedings of the IEEE international conference on image processing, San Jose, CA, October 1997, pp 815-818

  22. Rui Y, Huang TS, Mehrotra S (1998) Relevance feedback techniques in interactive content-based image retrieval. In: Proceedings of the conference on storage and retrieval for image and video databases (SPIE), San Jose, CA, 1998, pp 25-36

  23. Santini S (2000) The integration of textual and visual search in image databases. In: Proceedings of the 1st international workshop on intelligent multimedia computing and networking, Atlantic City, NJ, 27 February-3 March 2000

  24. Santini S (2000) Emergent semantics through interaction in image databases. Knowl Data Eng 13(3):337-351

    Article  Google Scholar 

  25. Shafer G (1976) A mathematical theory of evidence. Princeton University Press, Princeton, NJ

  26. Smeaton AF, Qigley I (1996) Experiments on using semantic distances between words in image caption retrieval. In: Proceedings of ACM SIGIR, Zurich, Switzerland, August 1996, pp 174-180

  27. Smets Ph, Kennes R (1994) The transferable belief model. Artif Intell 66:191-234

    Article  MathSciNet  MATH  Google Scholar 

  28. Smets Ph, Kennes R (1998) The transferable belief model for quantified belief representation. In: Gabbay DM, Smets P (eds) Handbook on defeasible reasoning and uncertainty management systems, vol 1. Kluwer, Dordrecht, 1998, pp 267-301

  29. Smith JR, Chang SF (1997) Visually searching the Web for content. IEEE Multimedia 4(3):12-20

    Article  Google Scholar 

  30. Taycher L, LaCascia M, Sclaroff S (1997) Image digestion and relevance feedback in the ImageRover WWW search engine. In: Proceedings of SPIE Visual’97, San Diego, 15-17 December 1997, pp 85-92

  31. Turk M, Pentland A (1991) Eigenfaces for recognition. Cognit Neurosci 3(1):71-86

    Google Scholar 

  32. Wang JZ, Li J, Wiederhold G (2001) Simplicity: semantics-sensitive integrated matching for picture libraries. IEEE Trans Patt Anal Mach Intell 23(9):947-963

    Article  Google Scholar 

  33. Wood MEJ, Campbell NW, Thomas BT (1998) Iterative refinement by relevance feedback in content-based digital image retrieval. In: Proceedings of the 6th ACM international multimedia conference, Bristol, UK, September 1998, pp 13-20

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, The University of Texas at Arlington, 416 Yates St., 306 Nedderman Hall, TX 76019-0015, Arlington, USA

    Y. Alp Aslandogan, Clement T. Yu, Ravishankar Mysore & Bo Liu

Authors
  1. Y. Alp Aslandogan
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Clement T. Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ravishankar Mysore
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Bo Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Y. Alp Aslandogan.

Additional information

Y. Alp Aslandogan: Correspondence to:

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aslandogan, Y.A., Yu, C.T., Mysore, R. et al. Robust content-based image indexing using contextual clues and automatic pseudofeedback. Multimedia Systems 9, 548–560 (2004). https://doi.org/10.1007/s00530-003-0127-y

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-003-0127-y

Keywords: