research-article

CrowdStory: Fine-Grained Event Storyline Generation by Fusion of Multi-Modal Crowdsourced Data

Authors:

Yu WangAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 1, Issue 3

Article No.: 55, Pages 1 - 19

https://doi.org/10.1145/3130920

Published: 11 September 2017 Publication History

Abstract

Event summarization based on crowdsourced microblog data is a promising research area, and several researchers have recently focused on this field. However, these previous works fail to characterize the fine-grained evolution of an event and the rich correlations among posts. The semantic associations among the multi-modal data in posts are also not investigated as a means to enhance the summarization performance. To address these issues, this study presents CrowdStory, which aims to characterize an event as a fine-grained, evolutionary, and correlation-rich storyline. A crowd-powered event model and a generic event storyline generation framework are first proposed, based on which a multi-clue--based approach to fine-grained event summarization is presented. The implicit human intelligence (HI) extracted from visual contents and community interactions is then used to identify inter-clue associations. Finally, a cross-media mining approach to selective visual story presentation is proposed. The experiment results indicate that, compared with the state-of-the-art methods, CrowdStory enables fine-grained event summarization (e.g., dynamic evolution) and correctly identifies up to 60% strong correlations (e.g., causality) of clues. The cross-media approach shows diversity and relevancy in visual data selection.

References

[1]

J. Bian, Y. Yang, H. Zhang, and T. Chua, “Multimedia Summarization for Social Events in Microblog Stream,” IEEE Transactions on multimedia, vol. 17, no. 2, pp. 216--228, 2015.

Digital Library

[2]

D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” Journal of machine Learning research, vol. 3, no.1, pp. 993-1022, 2003.

[3]

F. Buckley and M. Lewinter, “A friendly introduction to graph theory,” Prentice Hall, 2003.

[4]

D. Chakrabarti and K. Punera, “Event Summarization using Tweets,” in Proc. of ICWSM’11, AAAI, 2011, pp. 66--73.

[5]

Y. Chen, et al., “Event detection using customer care calls,” in Proc. of INFOCOM’13, 2013, pp. 1690-1698.

[6]

D. Corney, C. Martin, and A. Göker, “Two sides to every story: Subjective event summarization of sports events using Twitter,” in Proc. of the SoMuS ICMR 2014 Workshop, ACM, 2014.

[7]

A. Cui, M. Zhang, Y. Liu, et al., “Discover breaking events with popular hashtags in twitter,” in Proc. of CIKM’12, ACM, 2012, pp. 1794-1798.

Digital Library

[8]

N. Dehghani, M. Asadpour, “Graph-based Method for Summarized Storyline Generation in Twitter,” arXiv preprint arXiv:1504.07361, 2015.

[9]

J.L. Fleiss, B. Levin, C.P. Myunghee, “The measurement of interrater agreement. Statistical methods for rates and proportions,” no. 2, pp. 212-236, 1981.

[10]

B. Guo, H. Chen, Z. Yu, X. Xie, S. Huangfu, D. Zhang, “FlierMeet: A Mobile Crowdsensing System for Cross-Space Public Information Reposting, Tagging, and Sharing,” IEEE Transactions on Mobile Computing, vol. 14, no. 10, 2015, pp. 2020-2033.

Digital Library

[11]

M. Halkidi, Y. Batistakis, and M. Vazirgiannis, “On clustering validation techniques,” Journal of intelligent information systems, vol. 17, no. 2-3, pp. 107-145, 2001.

Digital Library

[12]

T. Hua, X. Zhang, W. Wang, et al., “Automatical Storyline Generation with Help from Twitter,” in Proc. of CIKM’16, ACM, 2016, pp. 2383-2388.

Digital Library

[13]

L. Huang and L. Huang, “Optimized Event Storyline Generation based on Mixture-Event-Aspect Model,” in Proc. of EMNLP’13, ACL, 2013, pp. 726--735.

[14]

L. Hubert and P. Arabie, “Comparing partitions,” Journal of classification, vol. 2, no. 1, pp. 193-218, 1985.

[15]

J. Kim, A. Monroy-Hernandez, “Storia: Summarizing social media content based on narrative theory using crowdsourcing,” in Proc. of CSCW’16. ACM, 2016, pp. 1018-1027.

Digital Library

[16]

J. Kleinberg, D. Easley, Networks, Crowds, and Markets, Cambridge University Press, 2010.

[17]

P. Lee, et al., “CAST: A Context-Aware Story-Teller for Streaming Social Content,” in Proc. of CIKM’14, ACM, 2014, pp. 789-798.

Digital Library

[18]

C. Lin, et al., “Generating event storylines from microblogs,” in Proc. of CIKM’15, ACM, 2012, pp. 175-184.

Digital Library

[19]

C. Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in Proc. of ACL’04 workshop, 2004, pp. 1-8.

[20]

H. Lin and J. Bilmes, “Multi-document summarization via budgeted maximization of submodular functions,” in Proc. of ACL’10, 2010, pp. 912-920.

Digital Library

[21]

D. G. Lowe, “Object recognition from local scale-invariant features,” in Proc. of ICCV’99, 1999, pp. 1150-1157.

Digital Library

[22]

M. Mathioudakis, N. Koudas, “TwitterMonitor: trend detection over the Twitter stream,” in Proc. Of SIGMOD’10, ACM, 2010, pp. 1155-1158.

Digital Library

[23]

A. J. McMinn, Y. Moshfeghi, and J. M. Jose, “Building a large-scale corpus for evaluating event detection on twitter,” in Proc. of CIKM’13, ACM, 2013, pp. 409-418

Digital Library

[24]

P. Meladianos, et al., “Degeneracy-based real-time sub-event detection in twitter stream,” in Proc. of ICWSM’15, 2015, pp. 248-257.

[25]

J. Nichols, J. Mahmud, and C. Drews, “Summarizing sporting events using twitter,” in Proc. of IUI’12, ACM, 2012, pp. 189-198.

Digital Library

[26]

J. Pearl, “Causality,” Cambridge university press, 2009.

[27]

K. Rudra, et al., “Extracting Situational Information from Microblogs during Disaster Events: a Classification-Summarization Approach,” in Proc. of CIKM’15, ACM, 2015, pp. 583-592.

Digital Library

[28]

T. Sakaki, M. Okazaki, and Y. Matsuo, “Earthquake shakes Twitter users: real-time event detection by social sensors”, in Proc. of WWW’10, ACM, 2010, pp. 851-860.

Digital Library

[29]

M. Schinas, S. Papadopoulos, Y. Kompatsiaris, et al., “Visual event summarization on social media using topic modelling and graph-based ranking algorithms,” in Proc. of ICMR’15, ACM, 2015, pp. 203-210.

Digital Library

[30]

M. Schinas, S. Papadopoulos, Y. Kompatsiaris, et al., “StreamGrid: Summarization of Large Scale Events using Topic Modelling and Temporal Analysis,” SoMuS@ ICMR, 2014.

[31]

X. Shang, H. Zhang, T S. Chua, “Deep learning generic features for cross-media retrieval,” International Conference on Multimedia Modeling. Springer International Publishing, 2016, pp. 264-275.

Digital Library

[32]

C. Shen, F. Liu, F. Weng, et al., “A Participant-based Approach for Event Summarization Using Twitter Streams,” HLT-NAACL, 2013, pp. 1152-1162.

[33]

J. Song J, Y. Yang, Y. Yang, et al., “Inter-media hashing for large-scale retrieval from heterogeneous data sources,” in Proc. of SIGMOD’13. ACM, 2013, pp. 785-796.

Digital Library

[34]

S. Tang, F. Wu, S. Li, et al., “Sketch the Storyline with CHARCOAL: A Non-Parametric Approach,” IJCAI, 2015, pp. 3841-3848.

Digital Library

[35]

D. Wang, T. Li, and M. Ogihara, “Generating Pictorial Storylines Via Minimum-Weight Connected Dominating Set Approximation in Multi-View Graphs”, in Proc. of AAAI’12, 2012.

Digital Library

[36]

W. Wang, et al., “Effective deep learning-based multi-modal retrieval,” The VLDB Journal, vol. 25, no. 1, pp. 79-101, 2016.

Digital Library

[37]

Y. Wei, Y. Zhao, Z. Zhu, et al., “Modality-dependent cross-media retrieval,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 7, no. 4, pp. 57, 2016.

Digital Library

[38]

A. Weiler, M H. Scholl, F. Wanner, et al., “Event identification for local areas using social media streaming data,” in Proc. of the ACM SIGMOD Workshop on Databases and Social Networks. ACM, 2013, pp. 1-6.

Digital Library

[39]

A. Witayangkurn, T. Horanont, Y. Sekimoto, et al., “Anomalous event detection on large-scale gps data from mobile phones using hidden markov model and cloud platform,” in Proc. of UbiComp’13. ACM, 2013, pp. 1219-1228.

Digital Library

[40]

F. Wu, X. Lu, Z. Zhang, et al., “Cross-media semantic representation via bi-directional learning to rank,” in Proc. of MM’13, ACM, 2013, pp. 877-886.

Digital Library

[41]

D. Wu, Q. Liu, Y. Li, et al., “Adaptive Lookup of Open WiFi Using Crowdsensing,” IEEE/ACM Transactions on Networking, vol. 24, no. 6, pp. 3634-3647, 2016.

Digital Library

[42]

J. Xu, T C. Lu, “Seeing the big picture from microblogs: Harnessing social signals for visual event summarization,” in Proc. of IUI’15. ACM, 2015, pp. 62-66.

Digital Library

[43]

Z. Yu, F. Wu, Y. Yang, et al., “Discriminative coupled dictionary hashing for fast cross-media retrieval,” in Proc. of SIGIR’14, ACM, 2014, pp. 395-404.

Digital Library

[44]

D. Zhang and W.J. Li, “Large-Scale Supervised Multimodal Hashing with Semantic Correlation Maximization,” in Proc. of AAAI’14, 2014.

Digital Library

[45]

Q. Zhang, B. Goncalves, “Topical differences between Chinese language Twitter and Sina Weibo,” in Proc. of WWW’16, 2016.

Digital Library

[46]

J. Zhou, G. Ding, Y. Guo, “Latent semantic sparse hashing for cross-modal similarity search,” in Proc. of SIGIR’14, 2014, pp. 415-424.

Digital Library

[47]

W. Zhou, et al., “Generating textual storyline to improve situation awareness in disaster management,” in Proc. IRI’14, 2014, pp. 585-592.

[48]

L. J. Griffin, “Narrative, event-structure analysis, and causal interpretation in historical sociology,” American journal of Sociology, vol. 98, no. 5, pp. 1094-1133, 1993.

Cited By

Xie WJiang NMa YWai Ming Lee ELi XYu H(2025)Simulating Pedestrian Flow on Slopes via Transfer Learning Approach: From Single-File to CrowdIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.351258326:3(3873-3884)Online publication date: Mar-2025
https://doi.org/10.1109/TITS.2024.3512583
Ouyang JYu MYu WQin ZRegan AWu D(2024)TPGraph: A Spatial-Temporal Graph Learning Framework for Accurate Traffic Prediction on Arterial RoadsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.333455825:5(3911-3926)Online publication date: May-2024
https://doi.org/10.1109/TITS.2023.3334558
Roy Chowdhury SBasu SMaulik U(2024)Disastrous Event and Sub-Event Detection From Microblog Posts Using Bi-Clustering MethodIEEE Transactions on Computational Social Systems10.1109/TCSS.2022.321379411:1(161-170)Online publication date: Feb-2024
https://doi.org/10.1109/TCSS.2022.3213794
Show More Cited By

Recommendations

CrowdStory: multi-layered event storyline generation with mobile crowdsourced data
UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

Online social media, like Twitter, can be viewed as a kind of source of mobile crowdsourced data. It can be utilized to understand real-world social events. Recently, many researchers have worked on event summarization from this kind of data. However, ...
FBSN: A hybrid fine-grained neural network for biomedical event trigger identification
Abstract
Biomedical event extraction is one of the fundamental tasks in medical research and disease prevention. Event trigger usually signifies the occurrence of a biomedical event by adopting a word or a phrase. Meanwhile, the task of ...
Sentiment evaluation of forex news
CF '19: Proceedings of the 16th ACM International Conference on Computing Frontiers

Sentiment analysis is significant for excavating text opinion. There are two issues in the foreign exchange (Forex) field. 1) In sentiment orientation, most researches focus on product reviews, lack fine-grained sentiment analysis for Forex news. 2) In ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 1, Issue 3

September 2017

2023 pages

EISSN:2474-9567

DOI:10.1145/3139486

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 September 2017

Accepted: 01 July 2017

Revised: 01 May 2017

Received: 01 February 2017

Published in IMWUT Volume 1, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
790
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xie WJiang NMa YWai Ming Lee ELi XYu H(2025)Simulating Pedestrian Flow on Slopes via Transfer Learning Approach: From Single-File to CrowdIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.351258326:3(3873-3884)Online publication date: Mar-2025
https://doi.org/10.1109/TITS.2024.3512583
Ouyang JYu MYu WQin ZRegan AWu D(2024)TPGraph: A Spatial-Temporal Graph Learning Framework for Accurate Traffic Prediction on Arterial RoadsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.333455825:5(3911-3926)Online publication date: May-2024
https://doi.org/10.1109/TITS.2023.3334558
Roy Chowdhury SBasu SMaulik U(2024)Disastrous Event and Sub-Event Detection From Microblog Posts Using Bi-Clustering MethodIEEE Transactions on Computational Social Systems10.1109/TCSS.2022.321379411:1(161-170)Online publication date: Feb-2024
https://doi.org/10.1109/TCSS.2022.3213794
Zhou TCai ZLiu FSu J(2023)In Pursuit of Beauty: Aesthetic-Aware and Context-Adaptive Photo Selection in CrowdsensingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.323796935:9(9364-9377)Online publication date: 1-Sep-2023
https://doi.org/10.1109/TKDE.2023.3237969
Zhao XWang JJin PWang CYang CLi BZhang H(2023)Automatically Generating Storylines from Microblogging PlatformsNeural Information Processing10.1007/978-981-99-1648-1_4(38-50)Online publication date: 15-Apr-2023
https://doi.org/10.1007/978-981-99-1648-1_4
Aloufi SSaddik A(2022)MMSUM Digital Twins: A Multi-view Multi-modality Summarization Framework for Sporting EventsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/346277718:1(1-25)Online publication date: 27-Jan-2022
https://dl.acm.org/doi/10.1145/3462777
Jepsen TJensen CNielsen T(2022)Relational Fusion Networks: Graph Convolutional Networks for Road NetworksIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2020.301179923:1(418-429)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1109/TITS.2020.3011799
Chowdhury SBasu SMaulik U(2022)A survey on event and subevent detection from microblog data towards crisis managementInternational Journal of Data Science and Analytics10.1007/s41060-022-00335-y14:4(319-349)Online publication date: 10-Jun-2022
https://doi.org/10.1007/s41060-022-00335-y
Zhou TCai ZLiu F(2021)The Crowd Wisdom for Location Privacy of Crowdsensing PhotosProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34781065:3(1-23)Online publication date: 14-Sep-2021
https://dl.acm.org/doi/10.1145/3478106
Unankard SNadee W(2020)Sub-Events Tracking from Social Network based on the Relationships between Topics2020 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT & NCON)10.1109/ECTIDAMTNCON48261.2020.9090732(1-6)Online publication date: Mar-2020
https://doi.org/10.1109/ECTIDAMTNCON48261.2020.9090732
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents