Image Emotion Analysis Based on the Distance Relation of Emotion Categories via Deep Metric Learning

Abstract

Existing deep learning-based image emotion analysis methods regard image emotion classification as a usual classification task in which the semantics of categories are clear. Nevertheless, the semantics of emotion categories are fuzzy, leading to that people are ambiguous between emotions of similar semantic distance when observing images. Considering the semantic distance of emotion categories, that is, far or near distance relations between them, we design a similarity decline rule to first pre-process the similarities of sample pairs making them comparable. Then, image emotion analysis is performed through deep metric learning. For key issues in deep metric learning, that is, sampling and weighting, we design adaptive decision boundaries for sampling and a double-weighted mechanism for sampled pairs which is integrated in our proposed emotion constraint loss, which learns more information contributing to update model by boasting the weights. Therefore, more expressive embedding features are learned from embedding space. Thus, the similarity of pairs from adjacent categories is larger than that from far away ones. The experimental results demonstrate that our proposed method outperforms the state-of-the-art methods. In addition, the ablation experiments show that it is necessary to consider the semantic distance of emotion categories in image emotion analysis.