Skip to main content
SpringerLink
Log in

Visual attention-based comparative study on disaster detection from social media images

  • Original Article
  • Published:
Innovations in Systems and Software Engineering Aims and scope Submit manuscript

Abstract

The availability of images of events almost in real-time on social media has a prospect in many application developments. A humanitarian technology for disaster type and level assessment can be developed using the images and video available on social media. In this paper, we investigate the potential use of various available deep learning techniques to develop such an application. For our research, based on the use of publicly available image data, we have started collecting disaster images from various sources from South Asia. We created the South Asia Disaster (SAD) image dataset containing 493 images from various online news portals. Using the Keras as our framework to run our models: Visual Geometry Group (VGG-16 and VGG-19), Inception-V3 and Inception-ResNet-V2 (ResNet: Residual Network). However, to boost up the training speed, we dropped the fully connected layer and added a small, fully connected model. To identify the five different disasters: fire disaster, flood disaster, human disaster, infrastructure disaster, natural disaster; our proposed method with VGG-16 model’s recognition accuracy was 84.51%, which is the highest accuracy on the SAD dataset. After performing the testing, we calculate the VGG-16 classifier’s attention to visualize which part of the disaster images VGG-16 pays attention.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ahmad K, Conci N, Natale F (2018) A comparative study of global and deep features for the analysis of user-generated natural disaster related images (accepted for presentation in IVMSP 2018)

  2. Al Jazeera ZHC, Media Network, aljazeera. In: viewed 20 Jun 2019. https://www.aljazeera.com/indepth/inpictures/pictures-bangladesh-front-lines-climate-crisis-191203102447517.html

  3. Alam F, Imran M, Ofli F (2017) Image4act: online social media image processing for disaster response. In: Proceedings of the 2017 IEEE/ACM international conference on advances in social networks analysis and mining 2017, ASONAM ’17. Association for Computing Machinery, New York, NY, USA, pp 601–604. https://doi.org/10.1145/3110025.3110164

  4. Amin MA, Mohammed MK (2015) Overview of the imageclef 2015 medical clustering task. In: CLEF

  5. Anam M (1991) The daily star. In: viewed 20 Jun 2019. https://www.thedailystar.net/

  6. Avgerinakis K, Moumtzidou A, Andreadis S, Michail E, Gialampoukidis I, Vrochidis S, Kompatsiaris I (2017) Visual and textual analysis of social media and satellite images for flood detection @ multimedia satellite task mediaeval 2017

  7. Bose J, The Times of India. In: viewed 20 Jun 2019. https://timesofindia.indiatimes.com/

  8. Giannakeris P, Avgerinakis K, Karakostas A, Vrochidis S, Kompatsiaris I (2018) People and vehicles in danger—a fire and flood detection system in social media. In: 2018 IEEE 13th image, video, and multidimensional signal processing workshop (IVMSP), pp 1–5

  9. Jetley S, Lord NA, Lee N, Torr PHS (2018) Learn to pay attention. CoRR. arXiv:1804.02391

  10. Mark Getty JK, gettyimages. In: viewed 20 Jun 2019, https://www.gettyimages.com/photos/india-fire?mediatype=photography&page=3&phrase=india

  11. Mark Getty JK, gettyimages. In: viewed 20 Jun 2019

  12. Mark Getty JK, gettyimages. In: viewed 20 Jun 2019. https://www.gettyimages.com/photos/bangladesh-flood?mediatype=photography&phrase=bangladesh

  13. Mouzannar H, Rizk Y, Awad M (2018) Damage identification in social media posts using multimodal deep learning. In: ISCRAM

  14. Muhammad K, Ahmad J, Baik SW (2018) Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing 288(C):30–42. https://doi.org/10.1016/j.neucom.2017.04.083

    Article  Google Scholar 

  15. Network AJM, aljazeera. In: viewed 20 Jun 2019. https://www.aljazeera.com/

  16. Nguyen DT, Ofli F, Imran M, Mitra P (2017) Damage assessment from social media imagery data during disasters. In: Proceedings of the 2017 IEEE/ACM international conference on advances in social networks analysis and mining 2017, ASONAM ’17, Association for Computing Machinery, New York, NY, USA, pp 569–576. https://doi.org/10.1145/3110025.3110109

  17. Rizk Y, Jomaa HS, Awad M, Castillo C (2019) A computationally efficient multi-modal classification approach of disaster-related twitter images. In: Proceedings of the 34th ACM/SIGAPP symposium on applied computing, SAC ’19, Association for Computing Machinery, New York, NY, USA, pp 2050–2059. https://doi.org/10.1145/3297280.3297481

  18. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the thirty-first AAAI conference on artificial intelligence, AAAI’17. AAAI Press, pp 4278–4284

  19. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2818–2826

  20. Zafar Sobhan KAA (2013) Dhakatribune. In: viewed 20 Jun 2019. https://www.dhakatribune.com/

  21. Zhang X, Zou J, He K, Sun J (2016) Accelerating very deep convolutional networks for classification and detection. IEEE Trans Pattern Anal Mach Intell 38(10):1943–1955. https://doi.org/10.1109/TPAMI.2015.2502579

    Article  Google Scholar 

  22. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2921–2929

Download references

Acknowledgements

This project is supported by a grant from the Bangladesh Information and Communication Technology Ministries ICT Division, and Independent University, Bangladesh (IUB). We would also like to thank Dr. Moinul Islam Zaber, Associate Professor, CSE, Dhaka University, Bangladesh, for his valuable suggestions during this project.

Author information

Authors and Affiliations

  1. Artificial Intelligence and Cybernetics Lab, CSE, Independent University, Bangladesh, Dhaka, Bangladesh

    Arif, M. Ashraful Amin, Amin Ahsan Ali & A. K. M. Mahhubur Rahman

Authors
  1. Arif
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Ashraful Amin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amin Ahsan Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. K. M. Mahhubur Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. K. M. Mahhubur Rahman.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arif, Amin, M.A., Ali, A.A. et al. Visual attention-based comparative study on disaster detection from social media images. Innovations Syst Softw Eng 16, 309–319 (2020). https://doi.org/10.1007/s11334-020-00368-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11334-020-00368-1

Keywords

Search

Navigation