Abstract
Traditional post-disaster assessment of damage heavily relies on expensive geographic information system (GIS) data, especially remote sensing image data. In recent years, social media have become a rich source of disaster information that may be useful in assessing damage at a lower cost. Such information includes text (e.g., tweets) or images posted by eyewitnesses of a disaster. Most of the existing research explores the use of text in identifying situational awareness information useful for disaster response teams. The use of social media images to assess disaster damage is limited. We have recently proposed a novel approach, based on convolutional neural networks and class activation mapping, to locate building damage in a disaster image and to quantify the degree of the damage. In this paper, we study the usefulness of the proposed approach for other categories of infrastructure damage, specifically bridge and road damage, and compare two-class activation mapping approaches in this context. Experimental results show that our proposed approach enables the use of social network images for post-disaster infrastructure damage assessment and provides an inexpensive and feasible alternative to the more expensive GIS approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alam F, Imran M, Ofli F (2017) Image4act: online social media image processing for disaster response. In: Proceedings of the 2017 IEEE/ACM international conference on advances in social networks analysis and mining 2017. ACM, pp 601–604
Ashktorab Z, Brown C, Nandi M, Culotta A (2014) Tweedr: mining twitter to inform disaster response. In: Proceedings of ISCRAM
Attari N, Ofli F, Awad M, Lucas J, Chawla S (2017) Nazr-CNN: fine-grained classification of UAV imagery for damage assessment. In: 2017 IEEE international conference on data science and advanced analytics (DSAA). IEEE, pp 50–59
Bica M, Palen L, Bopp C (2017) Visual representations of disaster. In: Proceedings of the 2017 ACM CSCW. ACM, NY, pp. 1262–1276
Caragea C, Silvescu A, Tapia AH (2016) Identifying informative messages in disasters using convolutional neural networks. In: Proceedings of the ISCRAM, Brazil
Castillo C (2016) Big crisis data: social media in disasters and time-critical situations. Cambridge University Press, Cambridge
Cha YJ, Choi W, Büyüköztürk O (2017) Deep learning-based crack damage detection using convolutional neural networks. Comput Aided Civ Infrastruct Eng 32(5):361–378
Chattopadhyay A, Sarkar A, Howlader P, Balasubramanian VN (2017) Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks. CoRR arXiv:abs/1710.11063
Enenkel M, Saenz SM, Dookie DS, Braman L, Obradovich N, Kryvasheyeu Y (2018) Social media data analysis and feedback for advanced disaster risk management. CoRR arXiv:abs/1802.02631
Everingham M, Gool L, Williams CK, Winn J, Zisserman A (2010) The Pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338
Fan Y, Wen Q, Wang W, Wang P, Li L, Zhang P (2017) Quantifying disaster physical damage using remote sensing data—a technical work flow and case study of the 2014 Ludian Earthquake in china. Int J Disaster Risk Sci 8(4):471–488
Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning, vol 1. MIT Press, Cambridge
Guan X, Chen C (2014) Using social media data to understand and assess disasters. Nat Hazards 74(2):837–850
Gueguen L, Hamid R (2015) Large-scale damage detection using satellite imagery. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1321–1328
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Huang Q, Xiao Y (2015) Geographic situational awareness: mining tweets for disaster preparedness, emergency response, impact, and recovery. ISPRS Int J Geo Inf 4(3):1549–1568
Imran M, Castillo C, Diaz F, Vieweg S (2015) Processing social media messages in mass emergency: a survey. ACM Comput Surv (CSUR) 47(4):67
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Kryvasheyeu Y, Chen H, Obradovich N, Moro E, Van Hentenryck P, Fowler J, Cebrian M (2016) Rapid assessment of disaster damage using social media activity. Sci Adv 2(3):e1500779
LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436
Li H, Caragea D, Caragea C, Herndon N (2017) Disaster response aided by tweet classification with a domain adaptation approach. J Conting Crisis Manag (JCCM) 26(1):16–27
Liu W, Anguelov D, Erhan D, Szegedy C, Reed SE, Fu C, Berg AC (2016) SSD: single shot multibox detector. In: In European conference on computer vision. Springer, pp 21–37
Li X, Zhang H, Caragea D, Imran M (2018) Localizing and quantifying damage in social media images. In: IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), pp 194–201
Maeda H, Sekimoto Y, Seto T, Kashiyama T, Omata H (2018) Road damage detection using deep neural networks with images captured through a smartphone. arXiv preprint arXiv:1801.09454
Meier P (2015) Digital humanitarians: how big data is changing the face of humanitarian response. CRC Press Inc, Boca Raton
Nguyen DT, Al-Mannai K, Joty SR, Sajjad H, Imran M, Mitra P (2016) Rapid classification of crisis-related data on social networks using convolutional neural networks. CoRR arXiv:abs/1608.03902
Nguyen DT, Ofli F, Imran M, Mitra P (2017) Damage assessment from social media imagery data during disasters. In: Proceedings of the 2017 IEEE/ACM international conference on advances in social networks analysis and mining 2017. ACM, pp 569–576
Nia KR, Mori G (2017) Building damage assessment using deep learning and ground-level image data. In: 14th conference on computer and robot vision (CRV). IEEE, pp 95–102
Palen L, Anderson KM (2016) Crisis informatics-new data for extraordinary times. Science 353(6296):224–225
Resch B, Usländer F, Havas C (2017) Combining machine-learning topic models and spatiotemporal analysis of social media data for disaster footprint and damage assessment. In: Cartography and geographic information science, pp 1–15
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: Visual explanations from deep networks via gradient-based localization. In: IEEE international conference on computer vision (ICCV)
Sen A, Rudra K, Ghosh S (2015) Extracting situational awareness from microblogs during disaster events. In: 7th international conference on communication systems and networks (COMSNETS). IEEE, pp 1–6
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Computer vision and pattern recognition (CVPR)
Vetrivel A, Gerke M, Kerle N, Nex F, Vosselman G (2018) Disaster damage detection through synergistic use of deep learning and 3D point cloud features derived from very high resolution oblique aerial images, and multiple-kernel-learning. ISPRS J Photogramm Remote Sens 140:45–59
Xie S, Duan J, Liu S, Dai Q, Liu W, Ma Y, Guo R, Ma C (2016) Crowdsourcing rapid assessment of collapsed buildings early after the earthquake based on aerial remote sensing image: a case study of Yushu earthquake. Remote Sens 8(9):759
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Advances in neural information processing systems, pp 3320–3328
Yuan F, Liu R (2018) Feasibility study of using crowdsourcing to identify critical affected areas for rapid damage assessment: Hurricane Matthew case study. Int J Disaster Risk Reduct 28:758–767
Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2921–2929
Acknowledgements
The computing for this project was performed using Amazon Web Services (AWS). We thank the National Science Foundation and Amazon Web Services for support from Grant IIS-1741345, which enabled the research and the computation in this study. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either express or implied, of the National Science Foundation or AWS.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, X., Caragea, D., Zhang, H. et al. Localizing and quantifying infrastructure damage using class activation mapping approaches. Soc. Netw. Anal. Min. 9, 44 (2019). https://doi.org/10.1007/s13278-019-0588-4
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s13278-019-0588-4