TEMPLATE: TempRel Classification Model Trained with Embedded Temporal Relation Knowledge

Abstract

In recent years, the mainstream Temporal Relation (TempRel) classification methods may not take advantage of the large amount of semantic information contained in golden TempRel labels, which is lost by the traditional discrete one-hot labels. To solve this problem, we propose a new approach that can make full use of golden TempRel label information and make the model perform better. Firstly we build a TempRel Classification (TC) model, which consists of a RoBERTa and a Classifier. Secondly, we establish fine-grained templates to automatically generate sentences to enrich golden TempRel label information and build an Enhanced Data-set. Thirdly we use the Enhanced Data-set to train the Knowledge Encoder, which has the same structure as the TC model, and get embedded knowledge. Finally, we get a TC model Trained with EMbedded temPoral reLATion knowldgE (TEMPLATE) using our designed Cosine balanced MSE loss function. Extensive experimental results show that our approach achieves new state-of-the-art results on TB-Dense and MATRES and outperforms the TC model trained with only traditional cross entropy loss function with up to 5.51%$F_1$ on TB-Dense and 2.02%$F_1$ on MATRES.