Dhiraj Joshi
ABSTRACT
In this paper, we present an approach towards automated story picturing based on mutual reinforcement principle. Story picturing refers to the process of illustrating a story with suitable pictures. In our approach, semantic keywords are extracted from the story text and an annotated image database is searched to form an initial picture pool. Thereafter, a novel image ranking scheme automatically determines the importance of each image. Both lexical annotations and visual content of an image play a role in determining its rank. Annotations are processed using the Wordnet to derive a lexical signature for each image. An integrated region based similarity is also calculated between each pair of images. An overall similarity measure is formed using lexical and visual features. In the end, a mutual reinforcement based rank is calculated for each image using the image similarity matrix. We also present a human behavior model based on a discrete state Markov process which captures the intuition for our technique. Experimental results have demonstrated the effectiveness of our scheme
- M. Agosti, F. Crestani and G. Pasi, Lectures on Information Retrieval, Lecture Notes in Computer Science, vol. 1980, Springer-Verlag, Germany, 2000. Google Scholar
Digital Library
- K. Barnard and D. Forsyth, "Learning the Semantics of Words and Pictures " Proc. Int. Conf. on Computer Vision, pp. 408--415, July 2001.Google ScholarCross Ref
- K. Barnard, P. Duygulu, D. Forsyth, N. -de. Freitas D. M. Blei, and M. I. Jordan, "Matching Words and Pictures " Journal of Machine Learning Research, vol. 3, pp. 1107--1135, 2003. Google ScholarDigital Library
- S. Brin and L. Page, "The Anatomy of a Large-Scale Hypertextual Web Search Engine " Proc. Seventh Int. World Wide Web Conf., pp 107--117, April 1998. Google ScholarDigital Library
- D. C. Brown and B. Chandrasekaran, "Design Considerations for Picture Production in a Natural Language Graphics System " ACM SIGGRAPH Computer Graphics., vol. 15, no. 2, pp. 174--207, 1981. Google ScholarDigital Library
- A. Budanitsky and G. Hirst, "Semantic Distance in Wordnet: An Experimental, Application-Oriented Evaluation of Five Measures " Workshop on WordNet and Other Lexical Resources, NAACL, June 2001.Google Scholar
- C. Carson, S. Belongie, H. Greenspan, and J. Malik, "Blobworld: Color and Texture-Based Image Segmentation using EM and its Application to Image Querying and Classification " IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 8, pp. 1026--1038, 2002. Google ScholarDigital Library
- C.-c. Chen, H. Wactlar, J. Z. Wang, and K. Kiernan, "Digital Imagery for Significant Cultural and Historical Materials - An Emerging Research Field Bridging People, Culture, and Technologies " International Journal on Digital Libraries, 2004, accepted.Google Scholar
- Y. Chen, J. Z. Wang, and R. Krovetz, "CLUE: Cluster-based Retrieval of Images by Unsupervised Learning " IEEE Transactions on Image Processing, vol. 13, no. 15, pp. 2004, accepted.Google Scholar
- S. R. Clay and J. Wilhelms, "Put: Language-Based Interactive Manipulation of Objects ", IEEE Computer Graphics and Applications, vol. 16, no. 2, pp 31--39, 1996. Google ScholarDigital Library
- B. Coyne and R. Sproat, WordsEye: An Automatic Text-to-Scene Conversion System ", Proc. 28th Annual Conf. on Computer Graphics and Interactive Techniques, pp 487--496, August 2001. Google ScholarDigital Library
- C. Fellbaum, WordNet - An electronic lexical database, MIT Press, Cambridge, Massachusetts and London, England, 1998.Google Scholar
- E. Garfield, "Citation Analysis as a Tool in Journal Evaluation " Science, vol. 178, pp. 471--479, 1972.Google ScholarCross Ref
- J. M. Kleinberg, "Authoritative Sources in a Hyperlinked Environment " Journal of the ACM, vol. 46, no. 5, pp. 604--632, 1999. Google ScholarDigital Library
- J. Li and J. Z. Wang, "Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach "IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1075--1088, 2003. Google ScholarDigital Library
- J. Li and J. Z. Wang, "Studying Digital Imagery of Ancient Paintings by Mixtures of Stochastic Models " IEEE Transactions on Image Processing, vol. 13 no. 3, pp. 340--353, 2004. Google ScholarDigital Library
- L. Li, Y. Shang, and W. Zhang, "Improvement of HITS-based Algorithms on Web Documents " Proc. Eleventh Int. World Wide Web Conf., pp 527--535, 2002. Google ScholarDigital Library
- R. Lu and S. Zhang, Automatic Generation of Computer Animation, Lecture Notes in Artificial Intelligence, vol. 2160, Springer-Verlag, Germany, 2002. Google ScholarDigital Library
- W. Y. Ma and B. S. Manjunath, "NeTra: A Toolbox for Navigating Large Image Databases " Multimedia Systems, vol. 7, no. 3, pp. 184--198, 1999. Google ScholarDigital Library
- G. Miller, R. Beckwith, C. Fellbaum, D. Gross, and K. Miller, "Introduction to WordNet: An On-line Lexical Database " Journal of Lexicography, vol. 3, no. 4, pp. 235--244, 1990.Google ScholarCross Ref
- G. Pinski and F. Narin, "Citation In uence for Journal Aggregates of Scientific Publications: Theory, with Application to the Literature of Physics " Information Processing and Management, vol. 12, pp. 297--312, 1976.Google ScholarCross Ref
- R. Simmons and G. Novak, "Semantically Analyzing an English Subset for the CLOWNS Microworld " American Journal of Computational Linguistics, Microfiche 18. 1975.Google Scholar
- A. W. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, "Content-Based Image Retrieval at the End of the Early Years " IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349--1380, 2000. Google ScholarDigital Library
- J. Z. Wang, J. Li, and G. Wiederhold, "SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture Libraries " IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 9, pp. 947--963, 2001. Google ScholarDigital Library
Index Terms
- The story picturing engine: finding elite images to illustrate a story using mutual reinforcement
Recommendations
The Story Picturing Engine---a system for automatic text illustration
We present an unsupervised approach to automated story picturing. Semantic keywords are extracted from the story, an annotated image database is searched. Thereafter, a novel image ranking scheme automatically determines the importance of each image. ...
Abstract Images Have Different Levels of Retrievability Per Reverse Image Search Engine
Computer Vision – ECCV 2022 WorkshopsAbstractMuch computer vision research has focused on natural images, but technical documents typically consist of abstract images, such as charts, drawings, diagrams, and schematics. How well do general web search engines discover abstract images? Recent ...
Measuring search engine quality in image queries in 10 non-English languages: an exploratory study
iNEWS '08: Proceedings of the 2nd ACM workshop on Improving non english web searchingThe main aim of this short paper is to measure the relevance of the retrieved images in a number of queries in different non-English languages. The reasoning behind retrieval failures is investigated and suggestions for improving these problems are ...
Comments