A simple but effective span-level tagging method for discontinuous named entity recognition

Abstract

Discontinuous named entity recognition (NER) is a more challenging task compared to continuous NER. It aims to extract discontinuous entities composed of multiple no-adjacent spans, which requires representing and combining all the spans of each discontinuous entity. However, discontinuous NER may suffer from decoding ambiguity due to the large space of span combinations and the lack of association information between spans. To address this problem, we propose a simple yet effective span-level tagging scheme for discontinuous NER. The tagging scheme defines simple span-level tags to represent and associate all the spans of each discontinuous entity simultaneously, effectively solving the decoding ambiguity problem. Moreover, the proposed model employs a co-predictor consisting of a span-level graph-based predictor and a position-aware biaffine predictor to predict span-level tags. The span-level graph-based predictor enhances span representations by employing graph convolutional network on span-level graphs to capture the dependence between spans of each discontinuous entity. The position-aware biaffine predictor incorporates relative position information into biaffine to enrich the structural information of span representations. To verify the effectiveness of our method, we conduct experiments on three benchmark datasets (i.e., CADEC, ShARe 13 and ShARe 14). The results show our method significantly outperforms previous state-of-the-art methods.

Appendix A

1.1 Analysis of time complexity

We calculate the time complexity of our method. Some variables for calculating the time complexity are defined as follows: l: the maximum text length; m: the number of entities; k: the number of non-O spans (see Line 1 in Algorithm 1); $d_w$: word embedding size; $l_{b}$: the number of BERT’s layers; $d_{l}$: hidden layer dimension of LSTM; $d_h$: the size of the final word representation; $l_{g}$: the number of GCN’s layers; $d_{s}$: the size of the span representation; e: the number of edges in the span-level graph; c: the number of the span-level tags; $d_z$: the size of RPE. Our method mainly includes three components: encoder, co-predictor and decoding process.

Encoder   The word embeddings for target words are generated by BERT, whose time complexity is $O(l_{b}l^2d_w + l_{b} ld_{w}^2)$. To obtain the final word representations, these word embeddings are further fed to Bi-LSTM, whose time complexity is $O(ld_{l}^2+ld_{w}d_{l})$. Thus, the time complexity of the encoder is $O(l_{b} l^2 d_{w} + l_{b} l d_{w}^2 +l d_{l}^2 + ld_{w} d_{l})$.

Co-predictor   The tag scores of spans are predicted jointly by the span-level graph-based predictor and the position-aware biaffine predictor. The time complexity of the former is $O(l_{g} e d_{s} + l_{g} \frac{l(l+1)}{2} d_{s}^2 + d_{s} c)$ and that of the latter is $O(d_h^2 c + 2d_h c + (d_h c + c) d_z)$. Therefore, the time complexity of the co-predictor is $O(l_{g} e d_{s} + l_{g} \frac{l(l+1)}{2} d_{s}^2 + d_{s} c + d_h^2 c + 2d_h c + (d_h c+c) d_z)$.

Decoding Process   In the inference phase, all entities are decoded by the decoding algorithm, whose time complexity is $O(k^2 + mk + k)$.

Overall   Based on the above time complexity, the time complexity of our method in a single inference is represented as $O(l_{b} l^2 d_{w} + l_{b} l d_{w}^2 +l d_{l}^2 + ld_{w} d_{l} + l_{g} e d_{s} + l_{g} \frac{l(l+1)}{2} d_{s}^2 + d_{s} c + d_h^2 c + 2d_h c + (d_h c+c) d_z + k^2 + mk + k)$.