ABSTRACT
Table-to-text generation is designed to generate descriptive natural language for structured tables that conforms to objective facts and follows the source data. The current challenge in this field is to capture the structural information of the table and improve the quality of the generated text. The existing sequence-to-sequence approach is to linearize the table, which leads to miss captured structure information and is not conducive to the model learning contextual semantics. In this paper, we introduce structural-aware self-attention, which focuses on table structure to capture cell relationships between the same row or column. In this way, the generated descriptive text can more accurately reflect the correlation between the cells in the table, discarding irrelevant information. In order to adapt the pre-trained language model to the table-to-text generation task, we introduce prefix-tuning. Traditional fine-tuning methods update all model parameters, which leads to increased training costs. In contrast, using prefix-tuning for a more lightweight approach can improve model performance considerably. Attaching continuous prompts to tokens helps the model better understand the structure and semantics of the input sequence. All of our models are extended based on T5 and have strong competitiveness in the ToTTo dataset and Hitab dataset compared with several classical baselines.
- Junwei Bao, Duyu Tang, Nan Duan, Zhao Yan, Yuanhua Lv, Ming Zhou, and Tiejun Zhao. 2018. Table-to-Text: Describing Table Region with Natural Language. arxiv:1805.11234 [cs.CL]Google Scholar
- Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. 2021. HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation. arXiv preprint arXiv:2108.06712 (2021).Google Scholar
- Bhuwan Dhingra, Manaal Faruqui, Ankur Parikh, Ming-Wei Chang, Dipanjan Das, and William Cohen. 2019. Handling Divergent Reference Texts when Evaluating Table-to-Text Generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 4884–4895. https://doi.org/10.18653/v1/P19-1483Google ScholarCross Ref
- Rémi Lebret, David Grangier, and Michael Auli. 2016. Neural Text Generation from Structured Data with Application to the Biography Domain. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, 1203–1213. https://doi.org/10.18653/v1/D16-1128Google ScholarCross Ref
- Xiang Lisa Li and Percy Liang. 2021. Prefix-Tuning: Optimizing Continuous Prompts for Generation. arxiv:2101.00190 [cs.CL]Google Scholar
- Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, and Zhifang Sui. 2017. Table-to-text Generation by Structure-aware Seq2seq Learning. arxiv:1711.09724 [cs.CL]Google Scholar
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, 311–318. https://doi.org/10.3115/1073083.1073135Google ScholarDigital Library
- Ankur P Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. ToTTo: A Controlled Table-To-Text Generation Dataset. In Proceedings of EMNLP.Google ScholarCross Ref
- Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. AdapterFusion: Non-Destructive Task Composition for Transfer Learning. arxiv:2005.00247 [cs.CL]Google Scholar
- Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. arxiv:1910.10683 [cs.LG]Google Scholar
- Sascha Rothe, Shashi Narayan, and Aliaksei Severyn. 2020. Leveraging Pre-trained Checkpoints for Sequence Generation Tasks. Transactions of the Association for Computational Linguistics 8 (2020), 264–280. https://doi.org/10.1162/tacl_a_00313Google ScholarCross Ref
- Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get To The Point: Summarization with Pointer-Generator Networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vancouver, Canada, 1073–1083. https://doi.org/10.18653/v1/P17-1099Google ScholarCross Ref
- Thibault Sellam, Dipanjan Das, and Ankur P. Parikh. 2020. BLEURT: Learning Robust Metrics for Text Generation. In Annual Meeting of the Association for Computational Linguistics.Google Scholar
- Zhihong Shao, Minlie Huang, Jiangtao Wen, Wenfei Xu, and Xiaoyan Zhu. 2019. Long and Diverse Text Generation with Planning-based Hierarchical Variational Model. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 3257–3268. https://doi.org/10.18653/v1/D19-1321Google ScholarCross Ref
- Yixuan Su, David Vandyke, Sihui Wang, Yimai Fang, and Nigel Collier. 2021. Plan-then-Generate: Controlled Data-to-Text Generation via Planning. In Findings of the Association for Computational Linguistics: EMNLP 2021. Association for Computational Linguistics, Punta Cana, Dominican Republic, 895–909. https://doi.org/10.18653/v1/2021.findings-emnlp.76Google ScholarCross Ref
- Fei Wang, Zhewei Xu, Pedro Szekely, and Muhao Chen. 2022. Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning. arxiv:2205.03972 [cs.CL]Google Scholar
- Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. Challenges in Data-to-Document Generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, 2253–2263. https://doi.org/10.18653/v1/D17-1239Google ScholarCross Ref
- Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, and Dragomir Radev. 2019. CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 1962–1979. https://doi.org/10.18653/v1/D19-1204Google ScholarCross Ref
- Jeffrey O Zhang, Alexander Sax, Amir Zamir, Leonidas Guibas, and Jitendra Malik. 2020. Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks. arxiv:1912.13503 [cs.LG]Google Scholar
- Mengjie Zhao, Tao Lin, Fei Mi, Martin Jaggi, and Hinrich Schütze. 2020. Masking as an Efficient Alternative to Finetuning for Pretrained Language Models. arxiv:2004.12406 [cs.CL]Google Scholar
Index Terms
- Structure-aware Table-to-Text Generation with Prefix-tuning
Recommendations
Table-to-text generation by structure-aware seq2seq learning
AAAI'18/IAAI'18/EAAI'18: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial IntelligenceTable-to-text generation aims to generate a description for a factual table which can be viewed as a set of field-value records. To encode both the content and the structure of a table, we propose a novel structure-aware seq2seq architecture which ...
Three-stage Logical Table-to-Text Generation based on Type Control
ACAI '22: Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial IntelligenceTable-to-Text generation is to express the information in the table in words. Considering the simplicity and logic of the statement, the task of logical Table-to-Text generation is derived. Logical Table-to-Text generation is to generate logically ...
TableSF: A Structural Bias Framework for Table-To-Text Generation
Artificial Neural Networks and Machine Learning – ICANN 2023AbstractTable-to-text generation is to generate a description from the tabular data. Existing methods typically encoded table content in a fixed order and relied heavily on the table row or column sequence. They generated error text descriptions when the ...
Comments