Abstract
In the event of a disaster, social media is often used to draw attention to affected areas and distressed people. The massive population and diversity in Indian languages warrant a novel real-time, big-data solution that can increase situational awareness, reduce special forces’ response time, and expedite decision-making. The proposed solution, VIKAS, streams text, images, videos, and audio from posts on microblogging platforms using keywords. It then uses the Google Translate and Transliteration APIs to handle multilingual and macaronic hybrid text, including Hinglish. An Apache Kafka event pipeline processes the sheer volume of posts asynchronously. Duplicate, uninformative, or bot-posted data (checked using the Botometer machine learning algorithm) is discarded. Scraped data is also verified through Google’s FactCheck Explorer API. Audio and video clips are processed leveraging speech-to-text methods. The solution incorporates a BERT pre-trained model and word embeddings for natural language processing tasks, including sentiment analysis and classification of textual data. Image classification and object identification are implemented using a ResNet deep learning model. This multimodal approach pinpoints locations using nearby landmarks, severity, and type of support needed, ranging from humanitarian aid and rescue relief to infrastructure damage. Easy-to-interpret visualizations on an accessible dashboard consolidate many details that can streamline resource distribution and personnel deployment. VIKAS was presented to the National Disaster Response Force at the national-level finals of the Smart India Hackathon 2022.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Said, N., et al.: Natural disasters detection in social media and satellite imagery: a survey. Multimed. Tools Appl. 78(22), 31267–31302 (2019). https://doi.org/10.1007/s11042-019-07942-1
Alam, F., Ofli, F., Imran, M.: Processing social media images by combining human and machine computing during crises. Int. J. Human-Comput. Interact. 34(4), 311–327 (2018). https://doi.org/10.1080/10447318.2018.1427831
Imran, M., Elbassuoni, S., Castillo, C., Diaz, F., Meier, P.: Practical extraction of disaster-relevant information from social media. In: WWW 2013 Companion – Proceedings of the 22nd International Conference on World Wide Web (2013). https://doi.org/10.1145/2487788.2488109
Rudra, K., Goyal, P., Ganguly, N., Imran, M., Mitra, P.: Summarising situational tweets in crisis scenarios: an extractive-abstractive approach. IEEE Trans. Comput. Soc. Syst. 6(5), 981–993 (2019). https://doi.org/10.1109/TCSS.2019.2937899
Imran, M., et al.: AIDR: Artificial intelligence for disaster response. In: Proceedings of the 23rd International Conference on World Wide Web (2014)
Sindhu, S., Nair, D.S., Maya, V.S., Thanseeha, M.T., Vishnu Hari, C.: Disaster management from social media using machine learning. In: 2019 9th International Conference on Advances in Computing and Communication (ICACC), pp. 246–252 (2019). https://doi.org/10.1109/ICACC48162.2019.8986198
Dinani, S.T., Caragea, D.: Disaster image classification using capsule networks. Int. Joint Conf. on Neural Netw. (IJCNN) 2021, 1–8 (2021). https://doi.org/10.1109/IJCNN52387.2021.9534448
Gautam, A.K., Misra, L., Kumar, A., Misra, K., Aggarwal, S., Shah, R.R.: Multimodal analysis of disaster tweets. In: 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), pp. 94–103 (2019). https://doi.org/10.1109/BigMM.2019.00-38
Roesslein, J.: Tweepy: Twitter for Python!. https://github.com/tweepy/tweepy (2020)
Hiraman, B.R., Chapte Viresh, M., Karve Abhijeet, C.: A study of apache kafka in big data stream processing. In: 2018 International Conference on Information, Communication, Engineering and Technology (ICICET), pp. 1–3 (2018). https://doi.org/10.1109/ICICET.2018.8533771
Sayyadiharikandeh, M., Varol, O., Yang, K.C., Flammini, A., Menczer, F.: Detection of novel social bots by ensembles of specialized classifiers. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 2725–2732, October 2020
Duarte, N., Llanso, E., Loup, A.C.: Mixed messages? the limits of automated social media content analysis. In: FAT 106 (2018)
Howard, A., Guo, Y.: Natural Language Processing with Disaster Tweets (2019)
Pennington, J., Socher, R., Manning, C.D.: GloVe: Global Vectors for Word Representation (2014)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Semeniuta, S., Severyn, A., Barth, E.: Recurrent Dropout without Memory Loss (2016)
Agarap, A.F.: Deep Learning using Rectified Linear Units (2018)
Han, J., Moraga, C.: The influence of the sigmoid function parameters on the speed of backpropagation learning. In: Mira, J., Sandoval, F. (eds.) IWANN 1995. LNCS, vol. 930, pp. 195–201. Springer, Heidelberg (1995). https://doi.org/10.1007/3-540-59497-3_175
Prechelt, L.: Early stopping - but when? In: Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the trade, pp. 55–69. Springer Berlin Heidelberg, Berlin, Heidelberg (1998). https://doi.org/10.1007/3-540-49430-8_3
Gholamy, A., Kreinovich, V., Kosheleva, O.: Why 70/30 or 80/20 relation between training and testing sets: a pedagogical explanation (2018)
Zahra, K., Imran, M., Ostermann, F.O.: Automatic identification of eyewitness messages on twitter during disasters. Inform. Process. Manag. 57(1), 102107 (2020). https://doi.org/10.1016/j.ipm.2019.102107
Evgeniou, T., Pontil, M.: Support vector machines: theory and applications. In: Paliouras, G., Karkaletsis, V., Spyropoulos, C.D. (eds.) ACAI 1999. LNCS (LNAI), vol. 2049, pp. 249–257. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44673-7_12
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018)
Zhang, A.: SpeechRecognition. https://github.com/Uberi/speech_recognition (2014)
Han, S.: Googletrans (2017)
Howard, J., et al.: “fastai” (2018)
Alam, F., Ofli, F., Imran, M.: CrisisMMD: multimodal twitter datasets from natural disasters. In: Proceedings of the 12th International AAAI Conference on Web and Social Media (ICWSM). Stanford, California, USA (2018)
Ofli, F., Alam, F., Imran, M.: Analysis of social media data using multimodal deep learning for disaster response. In: Proceedings of the 17th International Conference on Information Systems for Crisis Response and Management (ISCRAM). USA (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition (2015)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Prabhu, G.M., Gupta, T., Srujan, M.V., Soumya, A.R., Palorkar, A., Chowdhury, A. (2023). VIKAS: A Multimodal Framework to Aid in Effective Disaster Management. In: Prabhu, S., Pokhrel, S.R., Li, G. (eds) Applications and Techniques in Information Security . ATIS 2022. Communications in Computer and Information Science, vol 1804. Springer, Singapore. https://doi.org/10.1007/978-981-99-2264-2_22
Download citation
DOI: https://doi.org/10.1007/978-981-99-2264-2_22
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-2263-5
Online ISBN: 978-981-99-2264-2
eBook Packages: Computer ScienceComputer Science (R0)