Meng Yang
Gary Marchionini
ABSTRACT
How do people make sense of a video based on viewing a few frames of that video? What elements constitute the "visual gist" in their minds? Answers to these questions will give implications to both content-based video retrieval and the interface design (e.g., key-frame selection) of digital video libraries. A preliminary study was conducted to unravel the issues and 45 subjects participated in the study. After viewing a fast forward surrogate, the subjects were asked to choose pictures which they thought would "belong to" the video. And they were also asked to think aloud during their selection processes. Nine visual gist attributes (e.g., people, objects and actions) were generated using the grounded theory method and their frequencies were also compared and analyzed.
- Ding, W., Marchionini, G., Soergel, D. Multimodal Surrogates for Video Browsing. In: Proc. of Digital Libraries '99: 85--93 Google Scholar
Digital Library
- Dufaux, F. Key frame selection to represent a video. ICIP 2000. Vol, II. p. 275--278Google Scholar
- Grodal, T. Emotions, Cognitions, and Narrative Patterns in Film. In Passionate views : film, cognition, and emotion, edited by Plantinga, C. & Smith, G. M. 1999, 127--145Google Scholar
- Jorgensen, C. Image attributes in describing tasks: an investigation. Information Processing & Management, 34(2/3), 1998, 161--174 Google ScholarDigital Library
- Levin, D. T. & Simons, D. J. Failure to detect changes to attended objects in motion pictures. Psychological Bulletin, 4, 1997, 501--506Google ScholarCross Ref
- Lieberman, L. R., & Culpepper, J. T. Words versus objects: comparison of free verbal recall. Pscychol. Rep. 17,1965, 983--988Google ScholarCross Ref
- Mandler, J & Ritchey G. H. Long term memory for pictures. Journal of Experimental Psychology {Human learning and memory}, 3, 1977. 386--396Google Scholar
- Markkula, M. and Sormumen, E. Searching for photos: journalists' practices in pictorial IR, The Challenge of Image Retrieval Research Workshop, 1998.Google ScholarCross Ref
- Massey, M.; Bender, W. Salient stills: process and practice. IBM Systems Journal, 35(3-4), 1996, 557--573. Google ScholarDigital Library
- Paivio, A. & Csapo, K. Picture superiority in free recall: Imagery or dual coding? Cognitive Psychology, 5, 1973, 176--206Google ScholarCross Ref
- Ponceleon, D., Srinivasan, S., Amir, A., Petkovic, D., & Diklic, D. Key to effective video retrieval: Effective cataloguing and browsing. In Proc. ACM Multimedia, 1998, 99--107. Google ScholarDigital Library
- Shepard, R. N. Recognition memory for words, sentences, and pictures. Journal of Verbal Learning and Verbal Behavior, 6, 1967, 156--163Google ScholarCross Ref
- Simons, D. J. & Levin, D. T., Change blindness. Trends Cognitive Science, 1, 1997, 261--267Google Scholar
- Wildemuth, B. M., Marchionini, G., Wilkens, T., Yang, M., Geisler, G., Fowler, B., Hughes, A., & Mu, X. Alternative surrogates for video objects in a digital library: users' perspectives on their relative usability. Proc., the European Conference on Digital Libraries (ECDL), 2002, 493--507 Google ScholarDigital Library
- Wildemuth, B. M., Marchionini, G., Yang, M., Geisler, G., Wilkens, T., Hughes, A., & Gruss, R. How fast is too fast? Evaluating fast forward surrogates for digital video. Proc., Joint Conference on Digital Libraries, 2003, 221--230 Google ScholarDigital Library
- Wolfe, J.M. Visual memory: what do you know about what you saw? Current Biology, 8(9), 1998, 303--304.Google ScholarCross Ref
Index Terms
- Deciphering visual gist and its implications for video retrieval and interface design
Recommendations
Content-based video retrieval: does video's semantic visual feature matter?
SIGIR '06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrievalA new shot level video browsing method based on semantic visual features (e.g., car, mountain, and fire) is proposed to facilitate content-based retrieval. The video's binary semantic feature vector is utilized to calculate the score of similarity ...
The evolution of visual information retrieval
This paper seeks to provide a brief overview of those developments which have taken the theory and practice of image and video retrieval into the digital age. Drawing on a voluminous literature, the context in which visual information retrieval takes ...
Supporting semantic visual feature browsing in contentbased video retrieval
SIGIR '06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrievalA new shot level video retrieval system that supports semantic visual features (e.g., car, mountain, and fire) browsing is developed to facilitate content-based retrieval. The video's binary semantic feature vector is utilized to calculate the score of ...
Comments