End-to-End Entity Linking Combined with Bert-based Siamese and Interaction Network

The Entity Linking (EL) task is actually more complex than it seems. Not only are most of the given texts short and unlabeled with candidate mention information, but the entities to be predicted generally do not appear in the training set of the pre-trained model. This paper introduces an end-to-end Entity Linking model combined with Bert-based Siamese and Interaction Network, which can complete the EL task quickly and accurately given only short textual information without pre-specified mention boundaries at the time of input. The model is divided into two stages to achieve end-to-end entity linking. The first part is a pretrained BERT encoder-based Siamese network (BSN) architecture. BSN includes a text encoder and an entity encoder. These encoders are used to obtain the word embedding of the input text and the entity description token of the knowledge base respectively. We can also calculate the probability that a span can be used as a mention. In the candidate generation process, we first take the average value of the token word vector in a mention as the representation vector of the mention. We further obtain the similarity between the mention and the entity in the knowledge base and the span corresponding to an entity probability. During training, the loss of mention detection and candidate generation is counted as the loss of the first part. In the second part, we add an interaction network (BIN) based on pretrained BERT encoder (BIN). Concatenate the mention context and candidate entities, obtain the representation vector of the mention context-candidate entity through the encoder. Then add a linear layer to obtain the score and reorder the original candidate entities to achieve final disambiguation. Besides, a corresponding solution is proposed for the Speed-Accuracy Tradeoff problem. In experiments, the model achieves near-SOTA performance on the Zero-shot EL dataset and the WikilinksNED dataset. And it is shown that the stage of Mention Detection (MD) and Candidate Entity Generation (CEG) are crucial in the end-to-end entity linking process, and the two tasks can benefit from each other from joint training.