Entity-Aware Social Media Reading Comprehension

Abstract

Social media reading comprehension (SMRC) aims to answer specific questions conditioned on short social media messages, such as tweets. Sophisticated neural networks and pretrained language models have been successfully leveraged in SMRC, accompanying with a series of deliberately-designed data cleaning strategies. However, the existing SMRC techniques still suffer from unawareness of various entity mentions, i.e., the successive tokens (words, sub-words or characters) that fully or briefly describe named entities, such as abbreviated person names. This unavoidably brings negative effects into question answering towards the questions of “who”, “where”, “which organization”, etc. To address the issue, we propose to enhance the capacity of a SMRC model in recognizing entity mentions and, more importantly, construct an entity-aware encoder to incorporate latent information of entities into the understanding of questions and tweets. In order to obtain a self-contained entity-aware encoder, we build a two-channel encoder-shareable neural network for multitask learning. The encoder is driven to produce distributed representations that not only facilitate decoding of entity mentions but prediction of answers. In our experiments, we employ 12-layer transformer encoders for multi-task learning. Experiments on the benchmark dataset TweetQA show that our method achieves significant improvements. It is also proven that our method outperforms the state-of-the-art model NUT-RC, yielding improvements of 2.5% BLEU-1, 3% Meteor and 2.2% Rouge-L, respectively.