Abstract
User generated content, available in massive amounts on the Internet, is receiving increased attention due to its many potential applications. One of such applications is the representation of events using multimedia data. In this paper, an event-based cross media question answering system, which retrieves and summarizes events on a given topic is proposed. In other words, we present a framework for leveraging social media data to extract and illustrate social events automatically on any given query. The system is built in three steps. First, the input query is parsed semantically to identify the topic, location, and time information related to the News of interest. Then, we use the parsed information to mine the latest and hottest related News from social news web services. Third, to identify a unique event, we model the News content by latent Dirichlet Allocation and cluster the News using the DBSCAN algorithm. In the end, for each event, we retrieve both textual and visual content of News that refer the same event. The resulting documents are shown within a vivid interface featuring both event description, tag cloud and photo collage.
Similar content being viewed by others
References
Allan J, Carbonell J, Doddington G, Yamron J, Yang Y (1998) Topic detection and tracking pilot study: final report. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, Lansdowne, VA, USA, Feb. 007, pp 194–218
Arthur D, Vassilvitskii S (2007) k-means++: the advantages of careful seeding. In SODA’07: Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, pages 1027–1035, Philadelphia, PA, USA. Society for Industrial and Applied Mathematics
Bao B-K, Min W, Sang J, Xu C (2012) Multimedia news digger on emerging topics from social streams. In Proceedings of the 20th ACM international conference on Multimedia, MM’12, pp 1357–1358
Becker H, Iter D, Naaman M, Gravano L (2012) Identifying content for planned events across social media sites. In ACM conference on WSDM
Chen L, Roy A (2009) Event detection from flickr data through wavelet-based spatial analysis. In ACM conference on CIKM
Chengjie S, Yi G (2004) A statistical approach for content extraction from web page. J Chin Inf Process 18(5):17–22
Delgado D, Magalhães JA, Correia N (2010) Assisted News Reading with Automated Illustrations. In ACM conference on Multimedia, pp 1647–1650
Firan CS, Georgescu M, Nejdl W, Paiu R (2010) Bringing order to your photos: Event-Driven Classification of Flickr Images Based on Social Knowledge. In Proceedings of the 19th ACM international conference on Information and knowledge management, New York, USA, pp 189
Gao Y, Wang M, Zha Z-J, Shen J, Li X, Wu X (2013) Visual-textual joint relevance learning for tag-based social image search. IEEE Trans Image Process 22(1):363–376
Hong R, Wang M, Li G, Nie L, Zha Z-J, Chua T-S (2012) Multimedia question answering. MultiMedia IEEE 19(4):72–78
Joshi D, Wang JZ, Li J (2006) The story picturing engine—a system for automatic text illustration. ACM Trans Multimed Comput Commun Appl 2(1):68–89
Li G, Ming Z, Li H, Chua T-S (2009) Video reference: question answering on youtube. In Proceedings of the 17th ACM international conference on Multimedia, pp 773–776
Li H, Tang J, Wang Y, Liu B (2012) Looking into the world on google maps with view direction estimated photos. Neurocomput 95:72–77
Liu X, Huet B, Troncy R (2011) Eurecom@ mediaeval 2011 social event detection task. In MediaEval
Liu X, Huet B, Troncy R (2011) Eurecom@ MediaEval 2011 social event detection task. In Proceedings of the MediaEval 2011 Workshop
Liu X, Troncy R, Huet B (2011) Finding Media Illustrating Events. In ACM International Conference on ICMR, Trento, Italy
Manning CD, Raghavan P, Schütze H (2008) Introduction to Information Retrieval. 1 edition, July
Mei T, Yang B, Yang S-Q, Hua X-S (2009) Video collage: presenting a video sequence using a single image. Visual Comput 39–51
Nie L, Wang M, Gao Y, Zha Z-J, Chua T-S (2013) Beyond text qa: multimedia answer generation by harvesting web information. IEEE Trans Multimedia 15(2):426–441
Pan C-C, Mitra P (2011) Event detection with spatial latent Dirichlet allocation. In Proceeding of the 11th annual international ACM/IEEE joint conference on Digital libraries, page 349, New York, USA, June.
Quack T, Leibe B, Van Gool L (2008) World-scale mining of objects and events from community photo collections. In Proceedings of the 2008 international conference on Content-based image and video retrieval, page 47, New York, USA, July
Sakaki T, Okazaki M, Matsuo Y (2010) Earthquake shakes twitter users: real-time event detection by social sensors. In International conference on World Wide Web, Raleigh, North Carolina, USA
Shi J, Malik J (2000) Normalized cuts and image segmentation. Pattern Anal Mach Intell IEEE Trans 22(8):888–905
Wang M, Hong R, Li G, Zha Z-J, Yan S, Chua T-S (2012) Event driven web video summarization by tag localization and key-shot identification. IEEE Trans Multimedia 14(4):975–985
Wang M, Li G, Lu Z, Gao Y, Chua T-S (2013) When amazon meets google: product visualization by exploring multiple web sources. ACM Trans Internet Technol 12(4):12:1–12:17
Weng J, Lee F (2011) Event detection in twitter. In AAAI conference on Weblogs and Social Media, Barcelona, Spain
Yahiaoui I, Mérialdo B, Huet B (2003) Comparison of multi-episode video summarization algorithms. EURASIP Journal on applied signal processing special issue on multimedia signal processing - Volume 2003 N°1, January 2003, 01
Zha Z-J, Yang L, Mei T, Wang M, Wang Z, Chua T-S, Hua X-S (2010) Visual query suggestion: towards capturing user intent in internet image search. ACM Trans Multimedia Comput Commun Appl 6(3):13:1–13:19
Zha Z-J, Zhang H, Wang M, Luan H, Chua T-S (2013) Detecting group activities with multi-camera context. IEEE Trans Circ Syst Video Technol 23(5):856–869
Zhu X, Goldberg AB, Eldawy M, Dyer CR, Strock B (2007) A text-to-picture synthesis system for augmenting communication. In Proceedings of the 22nd national conference on Artificial intelligence, number 2, p 1590–1595
Acknowledgments
This work was supported by the 973 Program of China (No. 2013CB329604), and the European Commission under contracts FP7-287911 LinkedTV and FP7-318101 MediaMixer, as well as.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, X., Huet, B. Event-based cross media question answering. Multimed Tools Appl 75, 1495–1508 (2016). https://doi.org/10.1007/s11042-014-2085-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-014-2085-0