Applying social media in emergency response: an attention-based bidirectional deep learning system for location reference recognition in disaster tweets

Abstract

Social media platforms like Twitter have been recognized as a reliable real-time information dissemination and collection medium, especially during disasters when traditional communication media fail. Information access improves situational awareness and is essential for successful disaster management. The response team primarily requires information about the people in danger and the need for and availability of resources, such as food, shelter, and medical supplies. These details can only be actionable with the location information. People’s tweets during disasters will be informal and not adhere to standard linguistic rules, causing traditional NLP methods to fail. This study focuses on location reference recognition, in which the system must identify any locations mentioned in tweets. Most existing solutions focus on rule-based systems and gazetteers, which depend on the completeness of the gazetteers and manually defined rules. Since people’s writing styles differ significantly, manually defining rules will be complex. This paper introduces a neural network architecture based on BiLSTM, CRF, and attention mechanisms. It exploits statistical linguistic properties also. Compared to state-of-the-art methods, the model demonstrated superior results in both in- and cross-domain scenarios on tweet datasets representing diverse disaster types from different regions and times. Empirical results demonstrate that supervised systems can replace gazetteer-based solutions. BiLSTM and CRF, in conjunction with attention mechanism, improve the sequential modelling in informal text. Our system excels in non-English tweets also. The observations have applications in location-based services like tracking news events, traffic management, and event localization.

Appendices

Appendix

A Preliminaries

The fundamental concepts encompassing the major components of the proposed system are explained in this section. Word embedding, BiLSTM, CRF and attention mechanisms are used in our system.

1.1 A.1 Text preprocessing

As the text data is unstructured in nature, it needs to be cleaned up to improve the reliability and adaptability of text processing systems. Users’ social media posts (tweets) are typically colloquial and contain a lot of irrelevant information. Preprocessing is used to mitigate these issues. The following are the common text preprocessing methods.

• Case conversion

• Stemming and lemmatization

• Removal of stopwords, URLs, punctuations, non-ASCII characters, extra spaces, etc.

• Expanding the informal abbreviations with the original words

Fig. 9

LSTM

1.2 A.2 Word embedding

Word embedding [42] is the most promising deep learning approach for word representation. The words are depicted as real-valued vectors. Semantically related words will have similar representations so that they will be placed closer in vector space.

1.2.1 A.2.1 RoBERTa model

Transformer-based pretrained language models like BERT (Bi-directional Encoder Representations from Transformers) [11] are breakthroughs in NLP. RoBERTa (Robustly Optimized BERT Pretraining Approach [44] is an improved variation of BERT. Compared to BERT, RoBERTa is pre-trained using larger batches and more data. RoBERTa outperformed BERT in [8, 22, 53, 67].

The proposed system applies RoBERTa-based word embedding. It generates contextual dynamic word vectors. Byte-pair encoding technique is used to process out-of-vocabulary words.

1.3 A.3 LSTM and BiLSTM

In sequential modelling, recurrent neural networks (RNNs) are demonstrated to be effective. However, RNNs suffers from long-term dependency issue [25]. Long short-term memory (LSTM) [24] is an improved variation of RNN. It solves problems of RNN in attaining long-term dependency in sequential modelling. It can retain information in the memory for a longer time. Both current input and previous output are processed to get the current output. Three gates, which are nonlinear - input gate, output gate, and forget gate - and a cell state are the major components of LSTM. This structure helps to regulate the flow of data from input to output.

Figure 9 depicts an LSTM architecture in detail.

The data progression through a single unit of LSTM is depicted in (19),(20), (21), (22), (23), (24) and (25), where,

$x_t$, $h_t$: input data and hidden state representation respectively at $t^{th}$ time step

$\hat{C_t}$: candidate cell state at $t^{th}$ time step

$C_t$: final cell state at $t^{th}$ time step

$F_t$, $I_t$ and $Out_t$: the data representation at the forget gate, the input gate and the output gate, respectively at time t

$W_f$, $W_i$, $W_o$: the weight parameters of forget gate, input gate and output gate

$b_f$, $b_i$ and $b_o$: the bias parameters of forget gate, input gate and output gate

$\sigma$: sigmoid function

$*$, $+$: pointwise multiplication and addition

$+\hspace{-5.52501pt}+\hspace{3.40001pt}$: concatenation operator

By forgetting the unwanted details, the forget gate retains the necessary information from previous states, the input gate captures the required data from the current input, and the output gate produces the current output.

$$\begin{aligned} u_t&=h_{t-1} +\hspace{-5.52501pt}+\hspace{3.40001pt}x_t \end{aligned}$$

(19)

$$\begin{aligned} F_t&= \sigma (W_f \cdot u_t + b_f) \end{aligned}$$

(20)

$$\begin{aligned} I_t&= \sigma (W_i \cdot u_t + b_i) \end{aligned}$$

(21)

$$\begin{aligned} \hat{C_t}&= tanh(W_c \cdot u_t + b_c) \end{aligned}$$

(22)

$$\begin{aligned} C_t&= F_t *C_{t-1} + I_t *\hat{C_t} \end{aligned}$$

(23)

$$\begin{aligned} Out_t&= \sigma (W_o \cdot u_t + b_o) \end{aligned}$$

(24)

$$\begin{aligned} h_t&= Out_t *tanh(C_t) \end{aligned}$$

(25)

Fig. 10

BiLSTM

Fig. 11

CRF

An LSTM processes the input only in one direction. However, a word’s semantics are influenced by both its left and right contexts. BiLSTM is a neural network which processes the input text in both directions. As a result, a feature vector with more contextual data is obtained. The BiLSTM architectural diagram is displayed in Fig. 10. It is composed of two LSTMs, one for each direction of data processing (left or right).

1.4 A.4 Conditional random field

We can consider location reference recognition as a sequence labelling task. Word embeddings are derived with a general natural language objective. As a result, it can be used for a wide range of tasks. Since these embeddings are not learned for a specific downstream task, there are chances of missing the most useful information for that task. Tasks like sentiment analysis, user review analysis, and fake news detection require the global semantics of a sentence, but sequence labelling tasks like POS tagging, and NER need neighboring word information.

In a sequence labelling task to determine the tag of a word, the most informative data are the tags of the very nearest words. Also, it needs a well-formed word embedding of the current word that embeds the general contextual information.

Conditional random field (CRF) was introduced by Lafferty (2001) [37]. It attained state-of-the-art performance for various sentence-level sequence labelling paradigms, including NER, POS tagging and chunking [47]. It is a discriminative model best suited for prediction problems where the state of the neighbours or contextual information influences the present prediction. Specifically, it performs sequence prediction by taking context into account. It models the inter-dependencies among neighboring word-level tags.

Figure 11 shows a symbolic representation of CRF with sequential tag interdependence.

RoBERTa word-embedding algorithm generates dynamic contextual embedding of natural language words. Hence, combining RoBERTa word embedding with CRF can generate a word-level representation giving more importance to neighboring words. We can utilize information from neighboring words to predict the current tag. It provides our system with a greater modelling capacity.

1.5 A.5 Attention mechanism

The concept underlying attention mechanism is the cognitive capacity to focus on certain parts of textual or visual data. This concept was originally applied to computer vision [39] by Larochelle and Hinton (2010). Later, Bahdanau et al. (2014) [6] implemented neural machine translation employing the attention technique. Afterwards, it was effectively used in various NLP systems, such as language modelling, information extraction, text classification, etc. [12, 69].

The attention mechanism enables the model to ignore irrelevant data and concentrate on the crucial parts. It prioritizes different input elements according to their importance. Different attention variants [6, 10, 63] have been implemented so far. The general idea in all the models is shown in Fig. 12.

Fig. 12

Attention Model

$x_1,x_2,...,x_t$ indicates the textual input elements. A compatibility score is computed between $x_i$s and a query vector q, for example, by calculating the dot product. The score shows the association between the input element and the query vector. This score is converted to a probability distribution, denoted as attention weight. The final context vector is generated by prioritizing the input elements based on their attention weight.

Table 11 Results of In-Domain Classification on flood dataset curated by Olimat et al. (Prec.: Precision, Rec.: Recall(, F1:F1 Score). Best values are highlighted in bold

Table 12 Results of In-Domain Classification on Hurricane Sandy, Christchurch Earthquake, Milan Blackout, and Turkey Earthquake (Prec.: Precision, Rec.: Recall, F1:F1 Score). Best values are highlighted in bold

B Disaster datasets

Details of 17 publicly available disaster-specific datasets are given here. These datasets, which describe various catastrophic occurrences, cover a wide geographic range (the Us, UK, India, New Zealand, Kuwait, etc.) in different languages (English, Turkish, and Arabic).

• The dataset collected and annotated by Al-Olimat et al. (2017) [2] consists of tweets posted by people during the Chennai Flood 2015, the Louisiana Flood 2016, and the Houston Flood 2016. It has a total of 4500 tweets. The Chennai, Louisiana and the Houston flood datasets have 2226, 1396 and 1701 location names, respectively.

• The dataset collected and annotated by Middleton et al. (2018) [50] consists of tweets posted during Hurricane Sandy 2012, the Christchurch Earthquake 2012, Milan Blackout 2013 and Turkey Earthquake 2013. These datasets [2, 50] were annotated in the BILOU scheme by Suwaileh et al. (2022) [60]. “B”, “I”, and “L” indicate the beginning token, the inside token and the last token of a multiple-token location name. “U” indicates a single-token location name, and “O” indicates tokens other than location names. The datasets are annotated for a higher level of granularity, such as street names, bridges, cities, etc. Several location references have multiple tokens - for example, the tweet " Adyar River flowing at first floor height of the nearby houses. Stay safe" has two tokens to refer to one location, i.e., Adyar and river.

• GeoCorpora [65]: It is a collection of tweets about various disasters. The authors gathered posts from Twitter that referenced natural disasters, infectious diseases, and riots by using keywords like "earthquake", "flu", "protest", "ebola", and others. The majority of the tweets are of events that happened in the United States (42%) and the United Kingdom(12%). Both fine-grained (street, river, mountain, etc.) and coarse-grained (country, state, country abbreviations, etc.) location references are given. GeoCorpora uses information from GeoNames to provide a comprehensive location description. GeoName identifier, corresponding toponym, country code, longitude and latitude are also given for each location reference token in the tweet text. This makes GeoCorpora suitable for geocoding models. Additionally, tweet metadata such as tweet_id, creation time, user location, etc., are given.

• Fine-Grained LOCation Tweet Corpus (FGLOCTweets [15]): It is a set of disaster-specific English tweets collected by keywords such as ’earthquake’, ’fire’, etc. The tweets correspond to events that occurred in different locations across the world. It references geopolitical entities, natural landforms, POIs and traffic ways. Several fine-grained locations and complex location expressions (’1km SW of Lake Henshaw’, ’25min out of Melbourne’) are included in this dataset. Tweets are annotated using the BMESO scheme (Beginning-Medium-End-Single-Outside). ’S’ is for single token location reference, and ’O’ is for tokens other than location references.

• Harvey 2017 [66]: It is a set of 1000 tweets collected during Hurricane Harvey 2017. It is annotated with a BIO (Beginning, Inside and Outside) scheme.

• IDRISI-RE [61]: This publicly available location mention recognition dataset includes tweets in both English and Arabic. The datasets are annotated in the BILOU scheme. The tweets span 41 disaster events of various types (flood, earthquake, bombing, covid19, etc.) in 23 countries across different continents. We used the gold version (manually annotated) in our experiments. Three Arabic and four English sub-datasets were used to test the proposed system: Hurricane Dorian 2019 (English), Ecuador Earthquake 2016 (English), Midwestern US Floods 2019 (English), Canada Wild Fires 2016 (English), Beirut Explosion 2020 (Arabic), Cairo bombing 2019 (Arabic) and Kuwait Flood 2018 (Arabic). All datasets include location references that are fine- and coarse-grained and have multiple components.

Table 13 Results of In-Domain Classification on Hurricane Harvey, GeoCorpora and FGLOCTweets (Prec.: Precision, Rec.: Recall, F1:F1 Score). Best values are highlighted in bold

Table 14 Results of In-Domain Classification on IDRISI-RE Dataset (P: Precision, R: Recall, F1:F1 Score). Best values are highlighted in bold

C Detailed performance values of location recognition systems on various datasets

The model numbering used in the result tables (Tables 11, 12, 13 and 14) is as follows: M1-SpaCyNER, M2-Stanford Stanza, M3-DBPedia SpotLight, M4-Geoparsepy, M5-Lample et al., M6-Gupta and Nishu, M7-Kumar and Singh, M8-LUKE, M9-Neuro TPR, M10-LORE, M11-nLORE, M12-GazPNE2, M13 - GPT System and M14-Proposed System. Macro averaged results are provided.