skip to main content
10.1145/3487351.3488318acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article
Public Access

A deep contrastive learning approach to extremely-sparse disaster damage assessment in social sensing

Published: 19 January 2022 Publication History

Abstract

Social sensing has emerged as a pervasive and scalable sensing paradigm to obtain timely information of the physical world from "human sensors". In this paper, we study a new extremely-sparse disaster damage assessment (DBA) problem in social sensing. The objective is to automatically assess the damage severity of affected areas in a disaster event by leveraging the imagery data reported on online social media with extremely sparse training data (e.g., only 1% of the data samples have labels). Our problem is motivated by the limitation of current DDA solutions that often require a significant amount of high-quality training data to learn an effective DDA model. We identify two critical challenges in solving our problem: i) it remains to be a fundamental challenge on how to effectively train a reliable DDA model given the lack of sufficient damage severity labels; ii) it is a difficult task to capture the excessive and fine-grained damage-related features in each image for accurate damage assessment. In this paper, we propose ContrastDDA, a deep contrastive learning approach to address the extremely-sparse DDA problem by designing an integrated contrastive and augmentative neural network architecture for accurate disaster damage assessment using the extremely sparse training samples. The evaluation results on two real-world DDA applications demonstrate that ContrastDDA clearly outperforms state-of-the-art deep learning and semi-supervised learning baselines with the highest DDA accuracy under different application scenarios.

References

[1]
D. Wang, T. Abdelzaher, and L. Kaplan, Social sensing: building reliable systems on unreliable data. Morgan Kaufmann, 2015.
[2]
Y. Zhang, Y. Lu, D. Zhang, L. Shang, and D. Wang, "Risksens: A multi-view learning approach to identifying risky traffic locations in intelligent transportation systems using social and remote sensing," in 2018 IEEE International Conference on Big Data (Big Data). IEEE, 2018, pp. 1544--1553.
[3]
T. H. Nazer, G. Xue, Y. Ji, and H. Liu, "Intelligent disaster response via social media analysis a survey," ACM SIGKDD Explorations Newsletter, vol. 19, no. 1, pp. 46--59, 2017.
[4]
Y. Mejova, I. Weber, and L. Fernandez-Luque, "Online health monitoring using facebook advertisement audience estimates in the united states: evaluation study," JMIR public health and surveillance, vol. 4, no. 1, p. e30, 2018.
[5]
D. Zhang, Y. Zhang, Q. Li, T. Plummer, and D. Wang, "Crowdlearn: A crowd-ai hybrid system for deep learning-based damage assessment applications," in 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS). IEEE, 2019, pp. 1221--1232.
[6]
D. T. Nguyen, F. Ofli, M. Imran, and P. Mitra, "Damage assessment from social media imagery data during disasters," in Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017, 2017, pp. 569--576.
[7]
X. Li, D. Caragea, H. Zhang, and M. Imran, "Localizing and quantifying damage in social media images," in 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 2018, pp. 194--201.
[8]
P. Kumar, F. Ofli, M. Imran, and C. Castillo, "Detection of disaster-affected cultural heritage sites from social media images using deep learning techniques," Journal on Computing and Cultural Heritage (JOCCH), vol. 13, no. 3, pp. 1--31, 2020.
[9]
Y. Zhang, R. Zong, and D. Wang, "A hybrid transfer learning approach to migratable disaster assessment in social media sensing," in 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 2020, pp. 131--138.
[10]
F. Alam, F. Ofli, M. Imran, and M. Aupetit, "A twitter tale of three hurricanes: Harvey, irma, and maria," arXiv preprint arXiv:1805.05144, 2018.
[11]
X. Li, D. Caragea, C. Caragea, M. Imran, and F. Ofli, "Identifying disaster damage images using a domain adaptation approach," in Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management, 2019.
[12]
C. Schaffer, "Overfitting avoidance as bias," Machine learning, vol. 10, no. 2, pp. 153--178, 1993.
[13]
F. Alam, S. Joty, and M. Imran, "Graph based semi-supervised learning with convolution neural networks to classify crisis related tweets," in Proceedings of the International AAAI Conference on Web and Social Media, vol. 12, no. 1, 2018.
[14]
Y. Zhang, R. Zong, Z. Kou, L. Shang, and D. Wang, "Collablearn: An uncertainty-aware crowd-ai collaboration system for cultural heritage damage assessment," IEEE Transactions on Computational Social Systems, 2021.
[15]
A. Rasmus, H. Valpola, M. Honkala, M. Berglund, and T. Raiko, "Semi-supervised learning with ladder networks," arXiv preprint arXiv:1507.02672, 2015.
[16]
D. Wang, B. K. Szymanski, T. Abdelzaher, H. Ji, and L. Kaplan, "The age of social sensing," Computer, vol. 52, no. 1, pp. 36--45, 2019.
[17]
D. Wang, L. Kaplan, T. Abdelzaher, and C. C. Aggarwal, "On credibility estimation tradeoffs in assured social sensing," IEEE Journal on Selected Areas in Communications, vol. 31, no. 6, pp. 1026--1037, 2013.
[18]
D. Wang, M. T. Amin, S. Li, T. Abdelzaher, L. Kaplan, S. Gu, C. Pan, H. Liu, C. C. Aggarwal, R. Ganti et al., "Using humans as sensors: an estimation-theoretic perspective," in IPSN-14 proceedings of the 13th international symposium on information processing in sensor networks. IEEE, 2014, pp. 35--46.
[19]
J. Zhang and D. Wang, "Duplicate report detection in urban crowdsensing applications for smart city," in 2015 IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity). IEEE, 2015, pp. 101--107.
[20]
D. Wang, L. Kaplan, H. Le, and T. Abdelzaher, "On truth discovery in social sensing: A maximum likelihood estimation approach," in Information Processing in Sensor Networks (IPSN), 2012 ACM/IEEE 11th International Conference on. IEEE, 2012, pp. 233--244.
[21]
H. Garcia-Molina, M. Joglekar, A. Marcus, A. Parameswaran, and V. Verroios, "Challenges in data crowdsourcing," IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 4, pp. 901--911, 2016.
[22]
D. Y. Zhang, Y. Huang, Y. Zhang, and D. Wang, "Crowd-assisted disaster scene assessment with human-ai interactive attention," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 03, 2020, pp. 2717--2724.
[23]
D. Wang, T. Abdelzaher, L. Kaplan, and C. C. Aggarwal, "Recursive fact-finding: A streaming approach to truth estimation in crowdsourcing applications," in 2013 IEEE 33rd international conference on distributed computing systems. IEEE, 2013, pp. 530--539.
[24]
F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H. Torr, and T. M. Hospedales, "Learning to compare: Relation network for few-shot learning," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1199--1208.
[25]
Q. Sun, Y. Liu, T.-S. Chua, and B. Schiele, "Meta-transfer learning for few-shot learning," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 403--412.
[26]
I. Z. Yalniz, H. Jégou, K. Chen, M. Paluri, and D. Mahajan, "Billion-scale semi-supervised learning for image classification," arXiv preprint arXiv:1905.00546, 2019.
[27]
T. Miyato, S.-i. Maeda, M. Koyama, and S. Ishii, "Virtual adversarial training: a regularization method for supervised and semi-supervised learning," IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 8, pp. 1979--1993, 2018.
[28]
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, "A simple framework for contrastive learning of visual representations," in International conference on machine learning. PMLR, 2020, pp. 1597--1607.
[29]
J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. D. Guo, M. G. Azar et al., "Bootstrap your own latent: A new approach to self-supervised learning," arXiv preprint arXiv:2006.07733, 2020.
[30]
W. Liu, Y. Wen, Z. Yu, and M. Yang, "Large-margin softmax loss for convolutional neural networks." in ICML, vol. 2, no. 3, 2016, p. 7.
[31]
D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
[32]
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017.
[33]
N. Sharma, V. Jain, and A. Mishra, "An analysis of convolutional neural networks for image classification," Procedia computer science, vol. 132, pp. 377--384, 2018.
[34]
S. Targ, D. Almeida, and K. Lyman, "Resnet in resnet: Generalizing residual architectures," arXiv preprint arXiv:1603.08029, 2016.
[35]
M. Ghifary, W. B. Kleijn, M. Zhang, D. Balduzzi, and W. Li, "Deep reconstruction-classification networks for unsupervised domain adaptation," in European Conference on Computer Vision. Springer, 2016, pp. 597--613.
[36]
D. Chicco and G. Jurman, "The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation," BMC genomics, vol. 21, no. 1, p. 6, 2020.

Cited By

View all
  • (2025)DisasterRes-Net: A framework for analyzing social media images in disaster responseInternational Journal of Disaster Risk Reduction10.1016/j.ijdrr.2024.105119116(105119)Online publication date: Jan-2025
  • (2024)PhD Forum: Learning at the Time of Disasters2024 IEEE International Conference on Smart Computing (SMARTCOMP)10.1109/SMARTCOMP61445.2024.00061(256-257)Online publication date: 29-Jun-2024
  • (2023)On optimizing model generality in AI-based disaster damage assessmentProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/701(6317-6325)Online publication date: 19-Aug-2023
  • Show More Cited By

Index Terms

  1. A deep contrastive learning approach to extremely-sparse disaster damage assessment in social sensing
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        ASONAM '21: Proceedings of the 2021 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining
        November 2021
        693 pages
        ISBN:9781450391283
        DOI:10.1145/3487351
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        In-Cooperation

        • IEEE CS

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 19 January 2022

        Permissions

        Request permissions for this article.

        Check for updates

        Qualifiers

        • Research-article

        Funding Sources

        Conference

        ASONAM '21
        Sponsor:

        Acceptance Rates

        ASONAM '21 Paper Acceptance Rate 22 of 118 submissions, 19%;
        Overall Acceptance Rate 116 of 549 submissions, 21%

        Upcoming Conference

        KDD '25

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)83
        • Downloads (Last 6 weeks)13
        Reflects downloads up to 27 Jan 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2025)DisasterRes-Net: A framework for analyzing social media images in disaster responseInternational Journal of Disaster Risk Reduction10.1016/j.ijdrr.2024.105119116(105119)Online publication date: Jan-2025
        • (2024)PhD Forum: Learning at the Time of Disasters2024 IEEE International Conference on Smart Computing (SMARTCOMP)10.1109/SMARTCOMP61445.2024.00061(256-257)Online publication date: 29-Jun-2024
        • (2023)On optimizing model generality in AI-based disaster damage assessmentProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/701(6317-6325)Online publication date: 19-Aug-2023
        • (2023)ContrastFaux: Sparse Semi-supervised Fauxtography Detection on the Web using Multi-view Contrastive LearningProceedings of the ACM Web Conference 202310.1145/3543507.3583869(3994-4003)Online publication date: 30-Apr-2023
        • (2023)A Crowdsourced Learning Framework to Optimize Cross-Event QoS in AI-powered Social Sensing2023 20th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON)10.1109/SECON58729.2023.10287448(429-437)Online publication date: 11-Sep-2023
        • (2023)CrowdWaterSensPervasive and Mobile Computing10.1016/j.pmcj.2023.10178892:COnline publication date: 1-May-2023
        • (2022)SmartWaterSens: A Crowdsensing-based Approach to Groundwater Contamination Estimation2022 IEEE International Conference on Smart Computing (SMARTCOMP)10.1109/SMARTCOMP55677.2022.00022(48-55)Online publication date: Jun-2022
        • (2022)An active one-shot learning approach to recognizing land usage from class-wise sparse satellite imagery in smart urban sensingKnowledge-Based Systems10.1016/j.knosys.2022.108997249(108997)Online publication date: Aug-2022

        View Options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Login options

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media