Abstract
Information extraction (IE) for visually-rich documents (VRDs) has achieved SOTA performance recently thanks to the adaptation of Transformer-based language models, which shows the great potential of pre-training methods. In this paper, we present a new approach to improve the capability of language model pre-training on VRDs. Firstly, we introduce a new query-based IE model that employs span extraction instead of using the common sequence labeling approach. Secondly, to extend the span extraction formulation, we propose a new training task focusing on modelling the relationships among semantic entities within a document. This task enables target spans to be extracted recursively and can be used to pre-train the model or as an IE downstream task. Evaluation on three datasets of popular business documents (invoices, receipts) shows that our proposed method achieves significant improvements compared to existing models. The method also provides a mechanism for knowledge accumulation from multiple downstream IE tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Dang, T.A.N., Thanh, D.N.: End-to-end information extraction by character-level embedding and multi-stage attentional u-net. In: BMVC, p. 96 (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Hong, T., Kim, D., Ji, M., Hwang, W., Nam, D., Park, S.: BROS: a pre-trained language model for understanding texts in document (2021). https://openreview.net/forum?id=punMXQEsPr0
Hwang, W., Yim, J., Park, S., Yang, S., Seo, M.: Spatial dependency parsing for semi-structured document information extraction (2020)
Li, X., Sun, X., Meng, Y., Liang, J., Wu, F., Li, J.: Dice loss for data-imbalanced nlp tasks. arXiv preprint arXiv:1911.02855 (2019)
Liu, X., Gao, F., Zhang, Q., Zhao, H.: Graph convolution for multimodal information extraction from visually rich documents (2019). https://arxiv.org/abs/1903.11279
Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. ArXiv abs/1508.04025 (2015)
Park, S., et al.: Cord: a consolidated receipt dataset for post-OCR parsing. In: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver (2019)
Qian, Y., Santus, E., Jin, Z., Guo, J., Barzilay, R.: GraphIE: a graph-based framework for information extraction (2018). http://arxiv.org/abs/1810.13083
Tomanek, K., Hahn, U.: Reducing class imbalance during active learning for named entity annotation. In: Proceedings of the Fifth International Conference on Knowledge Capture. pp. 105–112. K-CAP ’09, Association for Computing Machinery, New York (2009). https://doi.org/10.1145/1597735.1597754
Vedova, L.D., Yang, H., Orchard, G.: An Invoice Reading System Using a Graph Convolutional Network, vol. 2, pp. 434–449 (2019). https://doi.org/10.1007/978-3-030-21074-8
Xu, Y., et al.: Layoutlmv2: multi-modal pre-training for visually-rich document understanding. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL) 2021, August 2021
Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., Zhou, M.: Layoutlm: pre-training of text and layout for document image understanding. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1192–1200 (2020)
Xu, Y., et al.: Layoutxlm: Multimodal pre-training for multilingual visually-rich document understanding (2021)
Yu, W., Lu, N., Qi, X., Gong, P., Xiao, R.: PICK: processing key information extraction from documents using improved graph learning-convolutional networks. In: 2020 25th International Conference on Pattern Recognition (ICPR) (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Nguyen, TA.D., Vu, H.M., Son, N.H., Nguyen, MT. (2021). A Span Extraction Approach for Information Extraction on Visually-Rich Documents. In: Barney Smith, E.H., Pal, U. (eds) Document Analysis and Recognition – ICDAR 2021 Workshops. ICDAR 2021. Lecture Notes in Computer Science(), vol 12917. Springer, Cham. https://doi.org/10.1007/978-3-030-86159-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-86159-9_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86158-2
Online ISBN: 978-3-030-86159-9
eBook Packages: Computer ScienceComputer Science (R0)
