Abstract
To retrieve Web video intuitively, the concept of “impression” is of great importance, because many users consider feelings and moods to be one of the most significant factors motivating them to watch videos. In this paper, we propose an impression-aware video stream retrieval system for querying the visual impression of video streams by analyzing the temporal change in sentiments. As a metric of visual impression, we construct a 180-dimensional vector space called as color-impression space; each dimension corresponds to a specific adjective representing humans’ color perception. The main feature of this system is a context-dependent query processing mechanism to generate a ranking by considering the temporal transition of each video’s visual impressions on viewers’ emotion. We design an impression-aware noise reduction mechanism that dynamically reduces the number on non-zero features for each item mapped in the high-dimensional color-impression space by extracting the dominant salient impression features from a video stream. This system allows users to retrieve videos by submitting emotional queries such as “Find videos whose overall impression is happy and which have several sad and cool scenes”. Through this query processing mechanism, users can effectively retrieve videos without requiring detailed information about them.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cisco: Cisco Visual Networking Index: Forecast and Methodology, 2009-2014 (2010), http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-481360.pdf
Cunningham, S.J., Nichols, D.M.: How people find videos. In: Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 201–210 (2008)
Kiyoki, Y., Kitagawa, T., Hayama, T.: A metadatabase system for semantic image search by a mathematical model of meaning. ACM SIGMOD Record 23(4), 34–41 (1994)
Kiyoki, Y., Kitagawa, T., Hitomi, Y.: A fundamental framework for realizing semantic interoperability in a multidatabase environment. Journal of Integrated Computer-Aided Engineering 2(1), 3–20 (1995)
Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia information retrieval: State of the art and challenges. ACM TOMCCAP 2(1), 1–19 (2006)
Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)
Smeaton, A.F.: Techniques used and open challenges to the analysis, indexing and retrieval of digital video. Information Systems 32(4), 545–559 (2007)
Hou, X., Zhang, L.: Color conceptualization. In: Proceedings of the 15th International Conference on Multimedia, pp. 265–268. ACM (2007)
Corridoni, J.M., Del Bimbo, A., Pala, P.: Image retrieval by color semantics. Multimedia Systems 7(3), 175–183 (1999)
Valdez, P., Mehrabian, A.: Effects of color on emotions. Journal of Experimental Psychology: General 123(4), 394–409 (1994)
Kobayashi, S.: The aim and method of the color image scale. Color Research & Application 6(2), 93–107 (1981)
Kobayashi, S.: Color Image Scale. Oxford University Press (1992)
Nakamura, S., Tanaka, K.: Video Search by Impression Extracted from Social Annotation. In: Vossen, G., Long, D.D.E., Yu, J.X. (eds.) WISE 2009. LNCS, vol. 5802, pp. 401–414. Springer, Heidelberg (2009)
Lehane, B., O’Connor, N.E., Lee, H., Smeaton, A.F.: Indexing of Fictional Video Content for Event Detection and Summarisation. EURASIP Journal on Image and Video Processing, Article ID 14615, 15 pages (2007)
Russell, J.A., Mehrabian, A.: Evidence for a three-factor theory of emotions. Journal of Research in Personality 11, 273–294 (1977)
Arifin, S., Cheung, P.Y.K.: A computation method for video segmentation utilizing the pleasure-arousal-dominance emotional information. In: Proceedings of the 15th ACM International Conference on Multimedia, pp. 68–77 (2007)
Kurabayashi, S., Ueno, T., Kiyoki, Y.: A Context-Based Whole Video Retrieval System with Dynamic Video Stream Analysis Mechanisms. In: Proceedings of the 11th IEEE International Symposium on Multimedia (ISM 2009), pp. 505–510 (2009)
Kurabayashi, S., Kiyoki, Y.: MediaMatrix: A Video Stream Retrieval System with Mechanisms for Mining Contexts of Query Examples. In: Kitagawa, H., Ishikawa, Y., Li, Q., Watanabe, C. (eds.) DASFAA 2010, Part II. LNCS, vol. 5982, pp. 452–455. Springer, Heidelberg (2010)
Newhall, S.M., Nickerson, D., Judd, D.B.: Final Report of the O.S.A. Subcommittee on the Spacing of the Munsell Colors. Journal of the Optical Society of America 33(7), 385–411 (1943)
Godlove, I.H.: Improved Color-Difference Formula, with Applications to the Perceptibility and Acceptability of Fadings. Journal of the Optical Society of America 41(11), 760–770 (1951)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kurabayashi, S., Kiyoki, Y. (2012). Impression-Aware Video Stream Retrieval System with Temporal Color-Sentiment Analysis and Visualization. In: Liddle, S.W., Schewe, KD., Tjoa, A.M., Zhou, X. (eds) Database and Expert Systems Applications. DEXA 2012. Lecture Notes in Computer Science, vol 7447. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32597-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-32597-7_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32596-0
Online ISBN: 978-3-642-32597-7
eBook Packages: Computer ScienceComputer Science (R0)