Skip to main content
Springer Nature Link
Log in

A multiple kernel learning based fusion for earthquake detection from multimedia twitter data

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

An efficient way of extracting useful information from multiple sources of data is to use data fusion technology. This paper introduces a data fusion approach in multimedia data for earthquake detection in twitter by using kernel fusion. The fusion method applies to fuse two types of data. The first type is features extracted from text by using bag-of-words method which is based on the calculation of the term frequency-inverse document frequency. The second type is the visual features extracted from images by applying scale-invariant feature transform. A multiple kernel fusion is applied in order to fuse the information from these two sources. Our experiments have indicated that comparing to the approaches using single data source, the proposed approach of using multiple kernel learning algorithm as early fusion increased the accuracy for earthquake detection. Experimental results for the proposed method achieved a high accuracy of 0.94, comparing to accuracy of 0.89 with texts only, and accuracy of 0.83 with images only.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Alqhtani SM, Luo S, Regan B (2015) Fusing text and image for event detection in twitter. arXiv preprint arXiv:1503.03920

  2. Atefeh F, Khreich W (2015) A survey of techniques for event detection in twitter. Comput Intell 31(1):132–164

    Article  MathSciNet  Google Scholar 

  3. Brocher TM et al (2015) The Mw 6.0 24 August 2014 South Napa earthquake. Seismol Res Lett 86(2A):309–326

    Article  Google Scholar 

  4. Burks L, Miller M, Zadeh R (2014) Rapid estimate of ground shaking intensity by combining simple earthquake characteristics with tweets. In: 10th US Nat. Conf. Earthquake Eng., front. Earthquake Eng., anchorage, Jul. 21Y25

  5. Consortium C. C. (2012) Computing for Disasters: A Report from the Community Workshop. In: Visioning Workshop on Computing Disaster Management, Washington, DC. Retrieved from http://archive2.cra.org/ccc/files/docs/init/computingfordisasters pdf

  6. Crooks A, Croitoru A, Stefanidis A, Radzikowski J (2013) #Earthquake: Twitter as a distributed sensor system. Trans GIS 17(1):124–147

    Article  Google Scholar 

  7. Daneshvar S, Ghassemian H (2010) MRI and PET image fusion by combining IHS and retina-inspired models. Information Fusion 11(2):114–123

    Article  Google Scholar 

  8. Earle PS, Bowden DC, Guy M (2012) Twitter earthquake detection: earthquake monitoring in a social world. Ann Geophys 54(6):708–715

  9. Gray RM (1984) Vector quantization. ASSP Magazine, IEEE 1(2):4–29

    Article  Google Scholar 

  10. He M et al (2010) Performance evaluation of score level fusion in multimodal biometric systems. Pattern Recogn 43(5):1789–1800

    Article  MATH  Google Scholar 

  11. Isson J-P, Harriott J (2012) Win with advanced business analytics: creating business value from your data. John Wiley & Sons, Hoboken

    Book  Google Scholar 

  12. Iwanaga ISM, Nguyen T-M, Kawamura T, Nakagawa H, Tahara Y, Ohsuga A (2011) Building an earthquake evacuation ontology from twitter. In: 2011 I.E. international conference on granular Computing (GrC), pp 306–311. IEEE

  13. Kirsch S (2010) Sustainable Mining, (in English). Dialect Anthropol 34(1):87–93

    Article  MathSciNet  Google Scholar 

  14. Klein LA (2004) Sensor and data fusion: a tool for information assessment and decision making. Spie Press Bellingham, eWA WA

  15. Kompatsiaris Y, Hobson P (2008) Semantic multimedia and ontologies. Springer, Berlin

    Book  Google Scholar 

  16. Lan Z-z, Bao L, Yu S-I, Liu W, Hauptmann A (2014) Multimedia classification and event detection using double fusion, (in English). Multimed Tools Appl 71(1):333–347

    Article  Google Scholar 

  17. Li Y, Crandall DJ, Huttenlocher DP (2009) Landmark classification in large-scale image collections. In: Computer vision, 2009 I.E. 12th international conference on, pp 1957–1964. IEEE

  18. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  19. Ma A, Flenner A, Needell D, Percus AG (2014) Improving image clustering using sparse text and the wisdom of the crowds. arXiv preprint arXiv:1405.2102

  20. Mao J (2012) Multimodal data fusion as a Predictior of missing information in social networks. Tempe, Arizona State University

    Google Scholar 

  21. Mendoza M, Poblete B, Castillo C (2010) Twitter under crisis: can we trust what we RT? In: Proceedings of the first workshop on social media analytics, pp 71–79. ACM

  22. Mishra N, Silakari DS (2012) Image mining in the context of content based image retrieval: a perspective. IJCSI International Journal of Computer Science Issues 9(4):98–107

    Google Scholar 

  23. Muhammad K, Ahmad J, Sajjad M, Baik SW (2016) Visual saliency models for summarization of diagnostic hysteroscopy videos in healthcare systems. SpringerPlus 5(1):1495

    Article  Google Scholar 

  24. Nguyen T-V, Dao M-S, Mattivi R, Sansone E, De Natale FG, Boato G (2013) Event clustering and classification from social media: watershed-based and kernel methods. In: Working notes proceedings of the MediaEval 2013 workshop, CEUR-WS.org, Barcelona

  25. Ozdikis O, Senkul P, Oguztuzun H (2012) Semantic expansion of hashtags for enhanced event detection in twitter. In Proceedings of the 1st international workshop on online social systems. Citeseer

  26. Petkos G. et al (2014) Social event detection at MediaEval: a three-year retrospect of tasks and results. In: ACM ICMR 2014 workshop on social events in web multimedia (SEWM), Glasgow, pp 27–34

  27. Sakaki T, Okazaki M, Matsuo Y (2010) Earthquake shakes twitter users: real-time event detection by social sensors. In: Proceedings of the 19th international conference on world wide web, pp. 851–860. ACM

  28. Salton G, Buckley C (1988) Term-weighting approaches in automatic text retrieval. Inf Process Manag 24(5):513–523

    Article  Google Scholar 

  29. Scherp A, Jain R, Kankanhalli M, Mezaris V (2010) Modeling, detecting, and processing events in multimedia. In: Proceedings of the 18th ACM international conference on multimedia, pp 1739–1740. ACM

  30. Snoek CG, Worring M, Smeulders AW (2005) Early versus late fusion in semantic video analysis. In: Proceedings of the 13th annual ACM international conference on multimedia, pp 399–402. ACM

  31. Sokolova M, Lapalme G (2009) A systematic analysis of performance measures for classification tasks. Inf Process Manag 45(4):427–437

    Article  Google Scholar 

  32. Ting IH (2012) T., Social Network Mining, Analysis, and Research Trends: A Phenomenal Analysis. Cengage Learning, Boston

    Book  Google Scholar 

  33. Twitter Developer Documentation. Available: https://dev.twitter.com/docs

  34. USGS (2015) M 8.3-48km W of Illapel, Chile. Science for a Changing World, Available: https://earthquake.usgs.gov/earthquakes/eventpage/us20003k7a#executive

  35. Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemom Intell Lab Syst 2(1):37–52

    Article  Google Scholar 

  36. Xie Z, Guan L (2013) Multimodal information fusion of audiovisual emotion recognition using novel information theoretic tools. In IEEE international conference on multimedia and expo (ICME), pp 1–6. IEEE.

  37. Xu Z et al (2016) Crowdsourcing based description of urban emergency events using social media big data. IEEE Trans on Cloud Computing, PP(99):1–1

  38. Yang J, Jiang Y-G, Hauptmann AG, Ngo C-W (2007) Evaluating bag-of-visual-words representations in scene classification. In: Proceedings of the international workshop on multimedia information retrieval, pp 197–206. ACM

  39. Yardi S, Boyd D (2010) Tweeting from the Town Square: measuring geographic local networks. ICWSM, In, pp 194–201

    Google Scholar 

  40. Zafarani R, Abbasi MA, Liu H (2014) Social media mining: an introduction. Cambridge University Press, Cambridge

    Book  Google Scholar 

  41. Zhang Z, Zhang R (2010) Multimedia data mining: a systematic introduction to concepts and theory. Boca Raton, CRC Press

    MATH  Google Scholar 

  42. Zhang X, Chen X, Chen Y, Wang S, Li Z, Xia J (2015) Event detection and popularity prediction in microblogging. Neurocomputing 149:1469–1480

    Article  Google Scholar 

  43. Zhou, S. Leung, H. and Yao F. (2013) Multimedia Data Fusion. Mathematical Problems in Engineering, 2013: 3. Art. no. 586259.

Download references

Acknowledgements

The corresponding author and related research is sponsored by Najran University in Saudi Arabia.

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computing, The University of Newcastle, Callaghan, NSW, 2308, Australia

    Samar M. Alqhtani, Suhuai Luo & Brian Regan

Authors
  1. Samar M. Alqhtani
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Suhuai Luo
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Brian Regan
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Samar M. Alqhtani.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alqhtani, S.M., Luo, S. & Regan, B. A multiple kernel learning based fusion for earthquake detection from multimedia twitter data. Multimed Tools Appl 77, 12519–12532 (2018). https://doi.org/10.1007/s11042-017-4901-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4901-9

Keywords