Jana Machajdik
Allan Hanbury
ABSTRACT
Images can affect people on an emotional level. Since the emotions that arise in the viewer of an image are highly subjective, they are rarely indexed. However there are situations when it would be helpful if images could be retrieved based on their emotional content. We investigate and develop methods to extract and combine low-level features that represent the emotional content of an image, and use these for image emotion classification. Specifically, we exploit theoretical and empirical concepts from psychology and art theory to extract image features that are specific to the domain of artworks with emotional expression. For testing and training, we use three data sets: the International Affective Picture System (IAPS); a set of artistic photography from a photo sharing site (to investigate whether the conscious use of colors and textures displayed by the artists improves the classification); and a set of peer rated abstract paintings to investigate the influence of the features and ratings on pictures without contextual content. Improved classification results are obtained on the International Affective Picture System (IAPS), compared to state of the art work.
- deviantart. www.deviantart.com.Google Scholar
- R. Arnheim. Art and Visual Perception: A Psychology of the Creative Eye. University of California Press, 2004.Google Scholar
- N. Bianchi-Berthouze. K-dime: an affective image filtering system. Multimedia, IEEE, Volume 10(Issue :103 -- 106, 2003. Google ScholarDigital Library
- S.-B. Cho. Emotional image and musical information retrieval with interactive genetic algorithm. Proc. of the IEEE, 92(4):702{711, 2004.Google ScholarCross Ref
- C. Colombo, A. Del Bimbo, and P. Pala. Semantics in visual information retrieval. Multimedia, IEEE, 6(3):38{53, 1999. Google ScholarDigital Library
- J. M. Corridoni, A. Del Bimbo, and P. Pala. Image retrieval by color semantics. Multimedia Syst., 7(3):175{183, 1999. Google ScholarDigital Library
- R. Datta, D. Joshi, J. Li, and J. Z. Wang. Studying aesthetics in photographic images using a computational approach. In ECCV (3), pages 288--301, 2006. Google ScholarDigital Library
- I. Daubechies. Ten Lectures on Wavelets. Regional Conf. Series in Applied Mathematics. Soc for Industrial & Applied Math, December 1992. Google ScholarDigital Library
- P. Ekman, W. V. Friesen, M. O'Sullivan, A. Chan, I. Diacoyanni-Tarlatzis, K. Heider, R. Krause, W. A. LeCompte, T. Pitcairn, P. E. Ricci-Bitti, K. Scherer, M. Tomita, and A. Tzavaras. Universals and cultural di erences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology, 53(Issue 4):712--717, Oct 1987.Google ScholarCross Ref
- A. Hanbury. Constructing cylindrical coordinate colour spaces. Pat. Rec. Lett., 29(4):494{500, 2008. Google ScholarDigital Library
- A. Hanjalic. Extracting moods from pictures and sounds: towards truly personalized TV. Signal Processing Magazine, IEEE, 23(2):90--100, 2006.Google ScholarCross Ref
- A. Hanjalic and L.-Q. Xu. A ective video content representation and modeling. IEEE Transactions on Multimedia, 7(1):143--154, 2005. Google ScholarDigital Library
- R. Haralick and L. Shapiro. Computer and Robot Vision. Addison-Wesley Longman, 1992. Google ScholarDigital Library
- T. Hayashi and M. Hagiwara. Image query by impression words-the IQI system. Consumer Electronics, IEEE Transactions on, 44(2):347--352, May 1998. Google ScholarDigital Library
- J. Itten. The art of color : the subjective experience and objective rationale of color. John Wiley, New York, 1973.Google Scholar
- P. Lang, M. Bradley, and B. Cuthbert. International a ective picture system (IAPS): A ective ratings of pictures and instruction manual. Technical report, Univ. Florida, Gainesville, 2008.Google Scholar
- M. S. Lew, N. Sebe, C. Djeraba, and R. Jain. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimedia Comput. Commun. Appl., 2(1):1--19, 2006. Google ScholarDigital Library
- C. Liensberger, J. Stottinger, and M. Kampel. Color-based and context-aware skin detection for online video annotation. In Proc. IEEE 2009 Int. Workshop on Multimedia Signal Processing, 2009.Google ScholarCross Ref
- B. Marcotegui and S. Beucher. Fast implementation of waterfall based on graphs. In C. Ronse, L. Najman, and E. Decenciere, editors, Mathematical Morphology: 40 Years on, volume 30 of Computational Imaging and Vision, pages 177--186. Springer-Verlag, Dordrecht, 2005.Google Scholar
- K. V. Mardia and P. E. Jupp. Directional Statistics. Wiley, 1972.Google Scholar
- J. A. Mikels, B. L. Fredrickson, G. R. Larkin, C. M. Lindberg, S. J. Maglio, and P. A. Reuter-Lorenz. Emotional category data on images from the international affective picture system. Behavior Research Methods, 37(4):626--630, 2005.Google ScholarCross Ref
- C. E. Osgood, G. Suci, and P. Tannenbaum. The measurement of meaning. University of Illinois Press, Urbana, IL, 1957.Google Scholar
- L.-C. Ou, M. R. Luo, A. Woodcock, and A. Wright. A study of colour emotion and colour preference. Part I: Colour emotions for single colours. Color Research and Application, 29(Issue 3):232 -- 240, June 2004.Google Scholar
- J. Stottinger, J. Banova, T. Ponitz, A. Hanbury, and N. Sebe. Translating journalists' requirements into features for image search. Int. Conf. on Virtual Systems and Multimedia, 2009. Google ScholarDigital Library
- H. Tamura, S. Mori, and T. Yamawaki. Textural features corresponding to visual perception. IEEE Transactions on Systems, Man and Cybernetics, 8(Issue 6):460--473, June 1978.Google ScholarCross Ref
- P. Valdez and A. Mehrabian. Effects of color on emotions. Journal of Experimental Psychology: General, 123(4):394{409, 1994.Google ScholarCross Ref
- J. van de Weijer, C. Schmid, and J. Verbeek. Learning color names from real-world images. IEEE CVPR, pages 1{8, 2007.Google ScholarCross Ref
- P. Viola and M. Jones. Robust real-time face detection. Int. Journal of Computer Vision, 57(2):137--154, 2004. Google ScholarDigital Library
- H. L. Wang and L.-F. Cheong. Affective understanding in film. Circuits and Systems for Video Technology, IEEE Transactions on, 16(6):689--704, 2006. Google ScholarDigital Library
- W. Wang and Q. He. A survey on emotional semantic image retrieval. 15th IEEE Int. Conf. on Image Processing, pages 117--120, 2008.Google ScholarCross Ref
- W.-N. Wang and Y.-L. Yu. Image emotional semantic query based on color semantic description. In Machine Learning and Cybernetics. Proc. 2005 Int. Conf. on, volume 7, pages 4571--4576, 2005.Google ScholarCross Ref
- W. Wei-ning, Y. Ying-lin, and J. Sheng-ming. Image retrieval by emotional semantics: A study of emotional space and feature extraction. IEEE Int. Conf. on Systems, Man and Cybernetics, 4(Issue 8--11):3534 -- 3539, Oct. 2006.Google Scholar
- Q. Wu, C. Zhou, and C. Wang. Content-based affective image classification and retrieval using support vector machines. A ective Computing and Intelligent Interaction, 3784:239--247, 2005. Google ScholarDigital Library
- V. Yanulevskaya, J. C. van Gemert, K. Roth, A. K. Herbold, N. Sebe, and J. M. Geusebroek. Emotional valence categorization using holistic image features. In IEEE Int. Conf. on Image Processing, 2008.Google ScholarCross Ref
Index Terms
- Affective image classification using features inspired by psychology and art theory
Recommendations
Exploring Principles-of-Art Features For Image Emotion Recognition
MM '14: Proceedings of the 22nd ACM international conference on MultimediaEmotions can be evoked in humans by images. Most previous works on image emotion analysis mainly used the elements-of-art-based low-level visual features. However, these features are vulnerable and not invariant to the different arrangements of ...
Emotion based classification of natural images
DETECT '11: Proceedings of the 2011 international workshop on DETecting and Exploiting Cultural diversiTy on the social webImages convey opinions and emotional messages in the communication process. With the increasing use of images in various scenarios, the area of opinion mining and sentiment analysis has recently received a huge burst of interest. In particular, in the ...
Analyzing emotional semantics of abstract art using low-level image features
IDA'11: Proceedings of the 10th international conference on Advances in intelligent data analysis XIn this work, we study people's emotions evoked by viewing abstract art images based on traditional low-level image features within a binary classification framework. Abstract art is used here instead of artistic or photographic images because those ...
Comments