Abstract
In recent years, the mainstream Temporal Relation (TempRel) classification methods may not take advantage of the large amount of semantic information contained in golden TempRel labels, which is lost by the traditional discrete one-hot labels. To solve this problem, we propose a new approach that can make full use of golden TempRel label information and make the model perform better. Firstly we build a TempRel Classification (TC) model, which consists of a RoBERTa and a Classifier. Secondly, we establish fine-grained templates to automatically generate sentences to enrich golden TempRel label information and build an Enhanced Data-set. Thirdly we use the Enhanced Data-set to train the Knowledge Encoder, which has the same structure as the TC model, and get embedded knowledge. Finally, we get a TC model Trained with EMbedded temPoral reLATion knowldgE (TEMPLATE) using our designed Cosine balanced MSE loss function. Extensive experimental results show that our approach achieves new state-of-the-art results on TB-Dense and MATRES and outperforms the TC model trained with only traditional cross entropy loss function with up to 5.51%\(F_1\) on TB-Dense and 2.02%\(F_1\) on MATRES.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We consider EQUAL to be the same as SIMULTANEOUS.
References
Bethard, S., Savova, G., Palmer, M., Pustejovsky, J.: SemEval-2017 task 12: clinical tempeval. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 565â572 (2017)
Cassidy, T., McDowell, B., Chambers, N., Bethard, S.: An annotation framework for dense event ordering. In: ACL (Volume 2: Short Papers), pp. 501â506 (2014)
Cheng, F., Miyao, Y.: Classifying temporal relations by bidirectional LSTM over dependency paths. In: ACL (Volume 2: Short Papers), pp. 1â6 (2017)
Cohen, A.D., Rosenman, S., Goldberg, Y.: Relation classification as two-way span-prediction. arXiv preprint arXiv:2010.04829 (2020)
Goldfarb-Tarrant, S., Chakrabarty, T., Weischedel, R., Peng, N.: Content planning for neural story generation with aristotelian rescoring. In: EMNLP, pp. 4319â4338 (2020)
Han, R., Hsu, I.H., Yang, M., Galstyan, A., Weischedel, R., Peng, N.: Deep structured neural network for event temporal relation extraction. In: CoNLL, pp. 666â106 (2019)
Han, R., Ren, X., Peng, N.: ECONET: effective continual pretraining of language models for event temporal reasoning. In: EMNLP, pp. 5367â5380 (2021)
Jin, W., et al.: Forecastqa: a question answering challenge for event forecasting with temporal text data, pp. 4636â4650 (2021)
Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Ma, M.D., et al.: Eventplus: a temporal event understanding pipeline. arXiv preprint arXiv:2101.04922 (2021)
Ning, Q., Subramanian, S., Roth, D.: An improved neural baseline for temporal relation extraction. In: EMNLP-IJCNLP, pp. 6203â6209 (2019)
Ning, Q., Wu, H., Han, R., Peng, N., Gardner, M., Roth, D.: TORQUE: a reading comprehension dataset of temporal ordering questions. In: EMNLP, pp. 1158â1172 (2020)
Ning, Q., Wu, H., Roth, D.: A multi-axis annotation scheme for event temporal relations. In: ACL (Volume 1: Long Papers), pp. 1318â1328 (2018)
Tan, X., Pergola, G., He, Y.: Extracting event temporal relations via hyperbolic geometry. In: EMNLP, pp. 8065â8077 (2021)
Wang, H., Chen, M., Zhang, H., Roth, D.: Joint constrained learning for event-event relation extraction. In: EMNLP, pp. 696â706 (2020)
Wen, H., Ji, H.: Utilizing relative event time to enhance event-event temporal relation extraction. In: EMNLP, pp. 10431â10437 (2021)
Yin, W., Hay, J., Roth, D.: Benchmarking zero-shot text classification: datasets, evaluation and entailment approach. In: EMNLP-IJCNLP, pp. 3914â3923 (2019)
Zhang, S., Huang, L., Ning, Q.: Extracting temporal event relation with syntactic-guided temporal graph transformer. arXiv preprint arXiv:2104.09570 (2021)
Zhao, X., Lin, S.T., Durrett, G.: Effective distant supervision for temporal relation extraction. In: Proceedings of the Second Workshop on Domain Adaptation for NLP, pp. 195â203 (2021)
Zhou, Y., et al.: Clinical temporal relation extraction with probabilistic soft logic regularization and global inference. In: AAAI. vol. 35, pp. 14647â14655 (2021)
Acknowledgments
This work is supported by grant from the National Natural Science Foundation of China (No. 62076048), the Science and Technology Innovation Foundation of Dalian (2020JJ26GX035).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Âİ 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, T., Li, L. (2022). TEMPLATE: TempRel Classification Model Trained with Embedded Temporal Relation Knowledge. In: Lu, W., Huang, S., Hong, Y., Zhou, X. (eds) Natural Language Processing and Chinese Computing. NLPCC 2022. Lecture Notes in Computer Science(), vol 13551. Springer, Cham. https://doi.org/10.1007/978-3-031-17120-8_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-17120-8_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17119-2
Online ISBN: 978-3-031-17120-8
eBook Packages: Computer ScienceComputer Science (R0)