Abstract
Can you imagine that two people who have different native languages and cannot understand other’s language are able to communicate with each other without professional interpreter? In this paper, a visualized communication system is designed to facilitate such people chatting with each other via visual information. Differing from the online instant message tools such as MSN, Google talk and ICQ, which are mostly based on textual information, the visualized communication system resorts to the vivid images which are relevant to the conversation context aside from text to jump the language obstacle. The multi-phase visual concept detection strategy is applied to associate the text with the corresponding web images. Then, a re-ranking algorithm attempts to return the most related and highest quality images at top positions. In addition, sentiment analysis is performed to help people understand the emotion of each other to further reduce the language obstacle. A number of daily conversation scenes are implemented in the experiments and the performance is evaluated by user study. The experimental results show that the visualized communication system is able to effectively help people with language obstacle to better understand each other.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Adams, P.H., Martell, C.H.: Topic Detection and Extraction in Chat. In: 2008 IEEE International Conference on Semantic Computing, pp. 581–588 (2008)
Dong, H., Hui, S.C., He, Y.: Structural analysis of chat messages for topic detection. Online Information Review, 496–516 (2006)
Wang, L., Jia, Y., Han, W.: Instant message clustering based on extended vector space model. In: Proceedings of the 2nd International Conference on Advances in Computation and Intelligence, pp. 435–443 (2007)
Jiang, Y.-G., Yang, J., Ngo, C.-W., Hauptmann, A.G.: Representations of Keypoint-Based Semantic Concept Detection: A Comprehensive Study. IEEE Transitions on Multimedia, 42–53 (2009)
Jiang, Y.G., Ngo, C.W., Chang, S.F.: Semantic context transfer across heterogeneous sources for domain adaptive video search. In: Proceedings of the Seventeen ACM International Conference on Multimedia, pp. 155–164 (2009)
Snoek, C.G.M., Huurnink, B., Hollink, L., de Rijke, M., Schreiber, G., Worring, M.: Adding semantics to detectors for video retrieval. IEEE Transaction on Multimedia 9(5), 975–986 (2007)
Snoek, C.G.M., Worring, M., Van Gemert, J.C., Geusebroek, J.M., Smeulders, A.W.M.: The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proceedings of the 14th Annual ACM International Conference on Multimedia, p. 430 (2006)
Yanagawa, A., Chang, S.-F., Kennedy, L., Hsu, W.: Columbia university’s baseline detectors for 374 lscom semantic visual concepts. In: Columbia University ADVENT Technical Report #222-2006-8 (2007)
Jiang, Y.-G., Ngo, C.-W., Yang, J.: Towards optimal bag-of-features for object categorization and semantic video retrieval. In: Proceedings of the 6th ACM International Conference on Image and Video Retrieval, p. 501 (2007)
Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: CVPR 2006 (2006)
Natsev, A., Haubold, A., Tesic, J., Xie, L., Yan, R.: Semantic concept-based query expansion and re-ranking for multimedia retrieval. In: ACM Multimedia, p. 1000 (2007)
Yao, T., Mei, T., Ngo, C.W.: Co-reranking by Mutual Reinforcement for Image Search. In: Proceedings of the 6th ACM International Conference on Image and Video Retrieval (2010)
Shih, J.-L., Chen, L.-H.: Color image retrieval based on primitives of color moments. In: IEE Proceedings-Vision, Image, and Signal Processing, p. 370 (2002)
Van de Wouwer, G., Scheunders, P., Dyck, D.V.: Statistical Texture Characterization from Discrete Wavelet Representations. IEEE Transactions on Image Processing, 592-598 (1999)
Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object categories: A comprehensive study. IJCV, 213–238 (2007)
Keshtkar, F., Inkpen, D.: Using Sentiment Orientation Features for Mood Classification in Blogs. IEEE, Los Alamitos (2009)
Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up?: sentiment classification using machine learning techniques. In: Proceedings of the ACL 2002 Conference on Empirical Methods in Natural Language Processing, vol. 10 (2002)
Zha, Z.-J., Yang, L., Mei, T., Wang, M., Wang, Z.: Visual query suggestion. In: Proceedings of ACM International Conference on Multimedia, pp. 15–24 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, X., Liu, Y., Liang, C., Xu, C. (2011). A Visualized Communication System Using Cross-Media Semantic Association. In: Lee, KT., Tsai, WH., Liao, HY.M., Chen, T., Hsieh, JW., Tseng, CC. (eds) Advances in Multimedia Modeling. MMM 2011. Lecture Notes in Computer Science, vol 6524. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17829-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-17829-0_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17828-3
Online ISBN: 978-3-642-17829-0
eBook Packages: Computer ScienceComputer Science (R0)