tutorial

Social Multimedia Sentiment Analysis

Authors:

Quanzeng YouAuthors Info & Claims

MM '17: Proceedings of the 25th ACM international conference on Multimedia

Pages 1953 - 1954

https://doi.org/10.1145/3123266.3130143

Published: 23 October 2017 Publication History

Abstract

Social multimedia refers to the multimedia content (text, images, and videos) generated by social network users for social interactions. The increasing popularity of online social networks leads to a significant amount of multimedia content generated by online social network users. Researchers from both the industrial and academic have been working on a broad range of projects related to the analyzing and understanding the online multimedia content, including real world activity prediction and content recommendation. Particularly, understanding online users' opinions or sentiments is a fundamental task that can benefit many applications, such as political campaigning and commercial marketing. We present a few recent advances in social multimedia sentiment analysis. Specifically, this tutorial consists of three parts. The first part is on visual sentiment analysis. We will introduce the task of visual sentiment, its main challenges, and the state-of-the-art approaches. We will include several representative approaches to manually designing visual features for this task as well as some approaches using deep neural networks. The second part is on building multimedia sentiment analysis datasets. We will introduce the challenges, the solutions in the construction of different large-scale datasets for sentiment analysis. The final part is mainly on multimodality model for sentiment analysis. We will introduce some recent research projects on multimodality designing and learning. In addition, we will also share some applications of sentiment analysis, as well as thoughts on current challenges and future directions.

References

[1]

Mohammad Al-Naser, Seyyed Saleh Mozafari Chanijani, Syed Saqib Bukhari, Damian Borth, and Andreas Dengel. 2015. What Makes a Beautiful Landscape Beautiful: Adjective Noun Pairs Attention by Eye-Tracking and Gaze Analysis. In Proceedings of the 1st International Workshop on Affect & Sentiment in Multimedia. ACM, 51--56.

Digital Library

[2]

Philipp Blandfort, Tushar Karayil, Damian Borth, and Andreas Dengel. 2016. Introducing concept and syntax transition networks for image captioning ICMR. ACM, 385--388.

Digital Library

[3]

Damian Borth. 2014. Visual Learning of Semantic Concepts in Social Multimedia. KI-Künstliche Intelligenz Vol. 28, 4 (2014), 333--336.

[4]

Damian Borth, Tao Chen, Rongrong Ji, and Shih-Fu Chang. 2013 a. Sentibank: large-scale ontology and classifiers for detecting sentiment and emotions in visual content. In ACM MM. ACM, 459--460.

Digital Library

[5]

Damian Borth, Rongrong Ji, Tao Chen, Thomas Breuel, and Shih-Fu Chang. 2013 b. Large-scale visual sentiment ontology and detectors using adjective noun pairs ACM MM. ACM, 223--232.

Digital Library

[6]

Tushar Karayil, Philipp Blandfort, Damian Borth, and Andreas Dengel. 2016. Generating Affective Captions using Concept And Syntax Transition Networks ACM MM. ACM, 1111--1115.

Digital Library

[7]

Takuya Narihira, Damian Borth, Stella X Yu, Karl Ni, and Trevor Darrell. 2015. Mapping Images to Sentiment Adjective Noun Pairs with Factorized Neural Nets. arXiv preprint arXiv:1511.06838 (2015).

[8]

Sebastian Sager, Damian Borth, Benjamin Elizalde, Christian Schulze, Bhiksha Raj, Ian Lane, and Andreas Dengel. 2016. Audiosentibank: Large-scale semantic ontology of acoustic concepts for audio content analysis. arXiv preprint arXiv:1607.03766 (2016).

[9]

Christian Schulze, Dominik Henter, Damian Borth, and Andreas Dengel. 2014. Automatic detection of CSA media by multi-modal feature fusion for law enforcement support lCMR. ACM, 353.

Digital Library

[10]

Marco Stricker, Syed Saqib Bukhari, Mohammad Al-Naser, Seyyed Saleh Mozafari Chanijani, Damian Borth, and Andreas Dengel. 2017. Which Saliency Detection Method is the Best to Estimate the Human Attention for Adjective Noun Concepts?. In ICAARTI. 185--195.

[11]

Quanzeng You. 2016. Sentiment and Emotion Analysis for Social Multimedia: Methodologies and Applications ACM MM. 1445--1449.

Digital Library

[12]

Quanzeng You, Liangliang Cao, Hailin Jin, and Jiebo Luo. 2016 a. Robust Visual-Textual Sentiment Analysis: When Attention meets Tree-structured Recursive Neural Networks. In ACM MM. 1008--1017.

Digital Library

[13]

Quanzeng You, Hailin Jin, and Jiebo Luo. 2017. Visual Sentiment Analysis by Attending on Local Image Regions AAAI. 231--237.

[14]

Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. 2016 b. Image captioning with semantic attention. In CVPR. 4651--4659.

[15]

Quanzeng You and Jiebo Luo. 2013. Towards social imagematics: sentiment analysis in social multimedia Proceedings of the Thirteenth International Workshop on Multimedia Data Mining. ACM, 3.

Digital Library

[16]

Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang. 2015 a. Joint visual-textual sentiment analysis with deep neural networks ACM MM. ACM, 1071--1074.

Digital Library

[17]

Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang. 2015 b. Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks. AAAI. 381--388.

Digital Library

[18]

Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang. 2016 c. Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark. AAAI. 308--314.

Digital Library

[19]

Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang. 2016 d. Cross-modality consistent regression for joint visual-textual sentiment analysis of social multimedia. In WSDM. ACM, 13--22.

Digital Library

[20]

Jianbo Yuan, Sean Mcdonough, Quanzeng You, and Jiebo Luo. 2013. Sentribute: image sentiment analysis from a mid-level perspective International Workshop on Issues of Sentiment Discovery and Opinion Mining. ACM, 10.

Digital Library

Cited By

Sun WZhou JDong LTian JLiu J(2021)Optimal Pre-Filtering for Improving Facebook Shared ImagesIEEE Transactions on Image Processing10.1109/TIP.2021.309379430(6292-6306)Online publication date: 2021
https://doi.org/10.1109/TIP.2021.3093794
Chen DS. D(2020)Social Multimedia Security and Suspicious Activity Detection in SDN using Hybrid Deep Learning TechniqueJournal of Information Technology and Digital World10.36548/jitdw.2020.2.0042:2(108-115)Online publication date: 27-May-2020
https://doi.org/10.36548/jitdw.2020.2.004
Ragusa ECambria EZunino RGastaldo P(2019)A Survey on Deep Learning in Image Polarity Detection: Balancing Generalization Performances and Computational CostsElectronics10.3390/electronics80707838:7(783)Online publication date: 12-Jul-2019
https://doi.org/10.3390/electronics8070783
Show More Cited By

Index Terms

Social Multimedia Sentiment Analysis

Recommendations

Social sentiment sensor: a visualization system for topic detection and topic sentiment analysis on microblog

As a new form of social media, microblogging provides platform sharing, wherein users can share their feelings and ideas on certain topics. Bursty topics from microblogs are the results of the emerging issues that instantly attract more followers and ...
Towards social imagematics: sentiment analysis in social multimedia
MDMKDD '13: Proceedings of the Thirteenth International Workshop on Multimedia Data Mining

Online social networks have attracted attention of people from both the academia and real world. In particular, the rich multimedia information accumulated in recent years provides an easy and convenient way for more active communication between people. ...
A survey on sentiment analysis and opinion mining for social multimedia

Social media sentiment analysis (also known as opinion mining) which aims to extract people's opinions, attitudes and emotions from social networks has become a research hotspot. Conventional sentiment analysis concentrates primarily on the textual ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '17: Proceedings of the 25th ACM international conference on Multimedia

October 2017

2028 pages

ISBN:9781450349062

DOI:10.1145/3123266

General Chairs:
Qiong Liu
FXPAL, USA
,
Rainer Lienhart
Universität Augsburg, Germany
,
Haohong Wang
TCL America, USA
,
Program Chairs:
Sheng-Wei "Kuan-Ta" Chen
Academia Sinica, Taiwan
,
Susanne Boll
University of Oldenburg, Germany
,
Phoebe Chen
La Trobe University, Australia
,
Gerald Friedland
Lawrence Livermore National Lab, USA
,
Jia Li
Google, USA
,
Shuicheng Yan
Qihoo 360, China

Copyright © 2017 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2017

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

MM '17

Sponsor:

SIGMM

MM '17: ACM Multimedia Conference

October 23 - 27, 2017

California, Mountain View, USA

Acceptance Rates

MM '17 Paper Acceptance Rate 189 of 684 submissions, 28%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
508
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sun WZhou JDong LTian JLiu J(2021)Optimal Pre-Filtering for Improving Facebook Shared ImagesIEEE Transactions on Image Processing10.1109/TIP.2021.309379430(6292-6306)Online publication date: 2021
https://doi.org/10.1109/TIP.2021.3093794
Chen DS. D(2020)Social Multimedia Security and Suspicious Activity Detection in SDN using Hybrid Deep Learning TechniqueJournal of Information Technology and Digital World10.36548/jitdw.2020.2.0042:2(108-115)Online publication date: 27-May-2020
https://doi.org/10.36548/jitdw.2020.2.004
Ragusa ECambria EZunino RGastaldo P(2019)A Survey on Deep Learning in Image Polarity Detection: Balancing Generalization Performances and Computational CostsElectronics10.3390/electronics80707838:7(783)Online publication date: 12-Jul-2019
https://doi.org/10.3390/electronics8070783
Ragusa EGastaldo PZunino R(2019)Fast Transfer Learning for Image Polarity Detection10.1007/978-3-030-16841-4_4(27-37)Online publication date: 3-Apr-2019
https://doi.org/10.1007/978-3-030-16841-4_4
Zhang WXu MJiang Q(2018)Opinion Mining and Sentiment Analysis in Social Media: Challenges and ApplicationsHCI in Business, Government, and Organizations10.1007/978-3-319-91716-0_43(536-548)Online publication date: 5-Jun-2018
https://doi.org/10.1007/978-3-319-91716-0_43

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten