Abstract
Data augmentation is a simple but effective way to improve the effectiveness and the robustness of pre-trained models. However, they are difficult to adapt to token-level tasks such as named entity recognition (NER) because of the different semantic granularity and more fine-grained labels. Inspired by some mixup augmentations in computer vision, we proposed three sentence-level data augmentations including CMix, CombiMix, TextMosaic, and adapted them to the NER task. Through empirical experiments on three authoritative datasets (OntoNotes4, CoNLL-03, OntoNotes5), we found that these methods will improve the effectiveness of the models if controlling the number of augmented samples. Strikingly, the results show our approaches can greatly improve the robustness of the pre-trained model even over strong baselines and token-level data augmentations. We achieved state-of-the-art (SOTA) in the robustness evaluation of the CCIR CUP 2021. The code is available at https://github.com/jrmjrm01/SenDA4NER-NLPCC2022.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Li, J., Sun, A., Jianglei H., Li, C.: A survey on deep learning for named entity recognition. IEEE Trans. Knowl. Data Eng. 34(1), 50-70 (2020a)
Wang, Y., et al.: Application of pre-training models in named entity recognition. In: 2020 12th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), vol. 1. IEEE (2020)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL, pp. 4171–4186, Minneapolis, Minnesota (2019)
Wei, J, Zou, K. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv preprint arXiv:1901.11196 (2019)
Fritzler, A., Logacheva, V., Kretov, M.: Few-shot classification in named entity recognition task. In: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, pp. 993-1000 (2019)
Feng, S.Y., Gangal, V., Wei, J., et al.: A survey of data augmentation approaches for nlp. arXiv preprint arXiv:2105.03075 (2021)
Karimi, A., et al.: AEDA: an easier data augmentation technique for text classification. In: EMNLP (2021)
Yoon, S., Kim, G., Park, K.: SSMix: Saliency-Based Span Mixup for Text Classification. ArXiv, abs/2106.08062 (2021)
Sun, L., et al.: Mixup-transformer: dynamic data augmentation for NLP tasks. Coling (2020)
Lin, B., Yuchen, J., et al.: RockNER: A simple method to create adversarial examples for evaluating the robustness of named entity recognition models. In: EMNLP (2021)
Dai, X., Adel, H.: An analysis of simple data augmentation for named entity recognition. ArXiv abs/2010.11683 (2020)
Si, C., et al.: Better robustness by more coverage: adversarial and mixup data augmentation for robust finetuning. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1569-1576 (2021)
Yun, S., Han, D., Oh, S.J., et al.: Cutmix: Regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6023–6032 (2019)
Hendrycks, D., Mu, N., Cubuk, E.D., et al.: Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781 (2019)
Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)
Gui, T., et al.: TextFlint: unified multilingual robustness evaluation toolkit for natural language processing. ArXiv abs/2103.11441 (2021)
Weischedel, R., et al.: Ontonotes release 4.0. LDC2011T03, Philadelphia, Penn Linguist. Data Consortium (2011)
Pradhan, S.: Towards robust linguistic analysis using ontonotes. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, CoNLL 2013, Sofia, Bulgaria, 8–9 August 2013, pp. 143–152. ACL (2013)
Erik, F., Sang, T.K., De Meulder, F.: Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pp. 142–147 (2003)
Li, J., Sun, A., Han, J., et al.: A survey on deep learning for named entity recognition. IEEE Trans. Knowl. Data Eng. 34(1), 50–70 (2020)
Sun, C., Qiu, X., Xu, Y., Huang, X.: How to fine-tune bert for text classification? In: Sun, M., Huang, X., Ji, H., Liu, Z., Liu, Y. (eds.) CCL 2019. LNCS (LNAI), vol. 11856, pp. 194–206. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32381-3_16
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Longpre, S. et al.: How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers? Findings(2020)
Acknowledgement
This work was supported by the National Key Research and Development Project under Grant 2018YFB1404303, and 03 special project and 5G project of Jiangxi Province of China (Grant No.20203ABC03W08), and the National Natural Science Foundation of China under Grant 62061030 and 62106094, and the Natural Science Foundation of Zhejiang Province of China (Grant No.LQ20D010001). We would like to thank Xiangyu Shi for his contribution to the comparison experiment, and Professor Xipeng Qiu, Professor Xiangyang Xue, Professor Dongfeng Jia and Dr. Hang Yan for their guidance on this paper. Thanks to the reviewers for their hard work to help us improve the quality of this paper.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, R., Zhang, X., Jiang, J., Li, W., Wang, Y. (2022). How Effective and Robust is Sentence-Level Data Augmentation for Named Entity Recognition?. In: Lu, W., Huang, S., Hong, Y., Zhou, X. (eds) Natural Language Processing and Chinese Computing. NLPCC 2022. Lecture Notes in Computer Science(), vol 13551. Springer, Cham. https://doi.org/10.1007/978-3-031-17120-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-17120-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17119-2
Online ISBN: 978-3-031-17120-8
eBook Packages: Computer ScienceComputer Science (R0)
