Skip to main content
SpringerLink
Log in

Assessing flood severity from crowdsourced social media photos with deep neural networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The use of social media data in disaster and crisis management is increasing rapidly. Particularly in connection to flooding events, geo-referenced images shared by citizens can provide situational awareness to emergency responders, as well as assistance to financial loss assessment, giving information that would otherwise be very hard to collect through conventional sensors or remote sensing. Moreover, recent advances in computer vision and deep learning can perhaps support the automated analysis of these data. In this paper, focusing on ground-level images taken by humans during flooding events, we evaluate the use of deep convolutional neural networks for (a) discriminating images showing direct evidence of a flood, and (b) estimating the severity of the flooding event. Considering distinct datasets (i.e., the European Flood 2013 dataset, and data from different editions of the MediaEval Multimedia Satellite Task), we specifically evaluated models based on the DenseNet and EfficientNet neural network architectures, concluding that these classification models can achieve a very high accuracy on this task, thus having a clear potential to complement other sources of information (e.g., satellite imagery) related to flooding events.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. http://www.efas.eu

  2. http://www.inf-cv.uni-jena.de/Research/Datasets/European+Flood+2013.html

  3. http://www.github.com/jorgemspereira/Classifying-Geo-Referenced-Photos

  4. http://www.eawag.ch/en/department/sww/projects/floodvision/

  5. http://www.figure-eight.com

  6. http://www.keras.io/applications/#densenet

  7. http://www.github.com/titu1994/keras-efficientnets

References

  1. Agarap AF (2018) Deep learning using rectified linear units (ReLU). arXiv:1803.0837

  2. Ahmad K, Pogorelov K, Riegler M, Conci N, Halvorsen P (2017) CNN and GAN based satellite and social media data fusion for disaster detection. In: Proceedings of the MediaEval workshop

  3. Ahmad S, Ahmad K, Ahmad N, Conci N (2017) Convolutional neural networks for disaster images retrieval. In: Proceedings of the MediaEval workshop

  4. Avgerinakis K, Moumtzidou A, Andreadis S, Michail E, Gialampoukidis I, Vrochidis S, Kompatsiaris I (2017) Visual and textual analysis of social media and satellite images for flood detection. In: Proceedings of the MediaEval workshop

  5. Bischke B, Bhardwaj P, Gautam A, Helber P, Borth D, Dengel A (2017) Detection of flooding events in social multimedia and satellite imagery using deep neural networks. In: Proceedings of the MediaEval workshop

  6. Bischke B, Helber P, Brugman S, Basar E, Zhao Z, Larson M, Pogorelov K (2019) The multimedia satellite task at MediaEval 2019. In: Proceedings of the MediaEval workshop

  7. Bischke B, Helber P, Schulze C, Srinivasan V, Dengel A, Borth D (2017) The multimedia satellite task at MediaEval 2017. In: Proceedings of the MediaEval workshop

  8. Bischke B, Helber P, Zhengyu Z, Bruijn J, Borth D (2018) The multimedia satellite task at MediaEval 2018. In: Proceedings of the MediaEval workshop

  9. Chaudhary P, D’Aronco S, Moy de Vitry M, Leitão J, Wegner J (2019) Flood-water level estimation from social media images. ISPRS Annals of the Photogrammetry Remote Sensing and Spatial Information Sciences 4(2/W5)

  10. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  11. Cian F, Marconcini M, Ceccato P (2018) Normalized difference flood index for rapid flood mapping: taking advantage of EO big data. Remote Sensing of Environment 209

  12. Dao MS, Pham QNM, Dang-Nguyen DT (2017) A domain-based late-fusion for disaster image retrieval from social media. In: Proceedings of the MediaEval workshop

  13. Deng J, Dong W, Socher R, Li L, Li K, Fei-fei L (2009) ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  14. Fu X, Bin Y, Peng L, Zhou J, Yang Y, Shen H (2017) BMC at MediaEval 2017 multimedia satellite task via regression random forest. In: Proceedings of the MediaEval workshop

  15. Geetha M, Manoj M, Sarika AS, Mohan M, Rao SN (2017) Detection and estimation of the extent of flood from crowd sourced images. In: Proceedings of the international conference on communication and signal processing

  16. Giannakeris P, Avgerinakis K, Karakostas A, Vrochidis S, Kompatsiaris I (2018) People and vehicles in danger-a fire and flood detection system in social media. In: Proceedings of the IEEE image, video, and multidimensional signal processing workshop

  17. Gong J, Ji S (2018) Photogrammetry and deep learning. Cehui Xuebao/Acta Geodaetica et Cartographica Sinica 47(1)

  18. Guan Q, Huang Y, Zhong Z, Zheng Z, Zheng L, Yang Y (2018) Diagnose like a radiologist: attention guided convolutional neural network for thorax disease classification. arXiv:1801.09927

  19. Hanif M, Tahir M, Mahrukh K, Rafi M (2017) Flood detection using social media data and spectral regression based kernel discriminant analysis. In: Proceedings of the MediaEval workshop

  20. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  21. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  22. Huang G, Liu Z, Maaten VDL, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  23. Katharopoulos A, Fleuret F (2019) Processing megapixel images with deep attention-sampling models. arXiv:1905.03711

  24. Khan S, Rahmani H, Shah SAA, Bennamoun M (2018) A guide to convolutional neural networks for computer vision. Morgan & Claypool Synthesis Lectures on Computer Vision 8(1)

  25. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. In: Proceedings of the international conference on learning representations

  26. Kornblith S, Shlens J, Le QV (2019) Do better ImageNet models transfer better?. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  27. Li L, Chen Y, Yu X, Liu R, Huang C (2015) Sub-pixel flood inundation mapping from multispectral remotely sensed images based on discrete particle swarm optimization. ISPRS Journal of Photogrammetry and Remote Sensing 101

  28. Li Y, Martinis S, Wieland M (2019) Urban flood mapping with an active self-learning convolutional neural network based on terraSAR-x intensity and interferometric coherence. ISPRS, Journal of Photogrammetry and Remote Sensing 152

  29. Liu L, Liu Y, Wang X, Yu D, Liu K, Huang H, Hu G (2015) Developing an effective 2-D urban flood inundation model for city emergency management based on cellular automata. Nat Hazards Earth Syst Sci 15

  30. Lo SW, Wu JH, Lin FP, Hsu CH (2015) Visual sensing for urban flood monitoring. Sensors 15(8)

  31. Lopez-Fuentes L, Rossi C, Skinnemoen H (2017) River segmentation for flood monitoring. In: Proceedings of the IEEE international conference on big data

  32. Lopez-Fuentes L, van de Weijer J, Bolanos M, Skinnemoen H (2017) Multi-modal deep learning approach for flood detection. In: Proceedings of the MediaEval workshop

  33. Lorini V, Castillo C, Dottori F, Kalas M, Nappo D, Salamon P (2019) Integrating social media into a pan-european flood awareness system: a multilingual approach. In: Proceedings of the international conference on information systems for crisis response and management

  34. Malinowski R, Groom G, Schwanghart W, Heckrath G (2015) Detection and delineation of localized flooding from WorldView-2 multispectral data. Remote Sensing 7(11)

  35. Mouratidis A, Ampatzidis D (2019) European digital elevation model validation against extensive global navigation satellite systems data and comparison with SRTM DEM and ASTER GDEM in central Macedonia (Greece). International Journal of Geo-Information 8

  36. Tkachenko N, Zubiaga A, Procter RN (2017) WISC at MediaEval 2017: multimedia satellite task. In: Proceedings of the MediaEval workshop

  37. Narayanan R, Lekshmy VM, Rao S, Sasidhar K (2014) A novel approach to urban flood monitoring using computer vision. In: Proceedings of the international conference on computing, communication and networking technologies

  38. Nogueira K, Fadel SG, Dourado IC, Werneck R, Muñoz JAV, Penatti OAB, Calumby RT, Li LT, Santos JA, Torres RS (2017) Data-driven flood detection using neural networks. In: Proceedings of the MediaEval workshop

  39. Nogueira K, Fadel SG, Dourado IC, Werneck R, Muñoz JAV, Penatti OAB, Calumby RT, Li LT, Santos JA, Torres RS (2018) Exploiting ConvNet diversity for flooding identification. IEEE Geoscience and Remote Sensing Letters 15(9)

  40. Pereira J, Dias M, Monteiro J, Estima J, Silva J, Pires JM, Martins B (2019) A dense U-Net model leveraging multiple remote sensing data sources for flood extent mapping. Unpublished Technical Report

  41. Petrasova A, Mitasova H, Petras V, Jeziorska J (2017) Fusion of high-resolution DEMs for water flow modeling. Open Geospatial Data Software and Standards 2(1)

  42. Ren S, He K, Girshick R, Sun J (2015) Faster r-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the conference on neural information processing systems

  43. Reuter HIA, Nelson AJ (2007) An evaluation of void filling interpolation methods for SRTM data. International Journal of Geographic Information Science 21(9)

  44. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  45. See LM (2019) A review of citizen science and crowdsourcing in applications of pluvial flooding. Frontiers in Earth Science 7:44

    Article  Google Scholar 

  46. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision

  47. Shen X, Wang D, Mao K, Anagnostou E, Hong Y (2019) Inundation extent mapping by synthetic aperture radar: a review. Remote Sensing 11(7)

  48. Smith LN (2017) Cyclical learning rates for training neural networks. In: Proceedings of the IEEE winter conference on applications of computer vision

  49. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15(1)

  50. Tadono T, Ishida H, Oda F, Naito S, Minakawa K, Iwamoto H (2014) Precise global dem generation by ALOS PRISM. ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences 2(4)

  51. Tan M, Chen B, Pang R, Vasudevan V, Sandler M, Howard A, Le QV (2019) Mnasnet: platform-aware neural architecture search for mobile. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  52. Tan M, Le QV (2019) EfficientNet: rethinking model scaling for convolutional neural networks. arXiv:1905.11946

  53. Witherow MA, Sazara C, Winter-Arboleda IM, Elbakary MI, Cetin M, Iftekharuddin KM (2018) Floodwater detection on roadways from crowdsourced images. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization 0(0)

  54. Xie J, He T, Zhang Z, Zhang H, Zhang Z, Li M (2018) Bag of tricks for image classification with convolutional neural networks. arXiv:1812.01187

  55. Zhang L, Schaeffer H (2018) Forward stability of ResNet, and its variants. arXiv:1811.09885

  56. Zhao Z, Larson M (2017) Retrieving social flooding images based on multimodal information. In: Proceedings of the MediaEval workshop

Download references

Acknowledgements

This research was supported through Fundação para a Ciência e Tecnologia (FCT), namely through the project grants PTDC/EEI-SCR/1743/2014 (Saturn), PTDC/CTA-OHR/29360/2017 (RiverCure), and PTDC/CCI-CIF/32607/2017 (MIMU), as well as through the INESC-ID multi-annual funding from the PIDDAC programme with reference UIDB/50021/2020. We also gratefully acknowledge the support of NVIDIA Corporation, with the donation of the two Titan Xp GPUs used in our experiments.

Author information

Authors and Affiliations

  1. Universidade de Lisboa, IST / INESC-ID, Lisboa, Portugal

    Jorge Pereira & Bruno Martins

  2. Universidade de Lisboa, IST / INESC-ID, Porto Salvo, Portugal

    João Monteiro

  3. INESC-ID, Lisboa, Portugal

    Joel Silva & Jacinto Estima

  4. Universidade Europeia, Lisboa, Portugal

    Jacinto Estima

Authors
  1. Jorge Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João Monteiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joel Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jacinto Estima
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bruno Martins
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jorge Pereira.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pereira, J., Monteiro, J., Silva, J. et al. Assessing flood severity from crowdsourced social media photos with deep neural networks. Multimed Tools Appl 79, 26197–26223 (2020). https://doi.org/10.1007/s11042-020-09196-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09196-8

Keywords

Search

Navigation