Abstract
When it comes to text-to-SQL tasks, the model needs to learn context-based representations of schema along with natural language utterances. We present a simple and effective method for text-to-SQL tasks, Column-Mask-Augmented Training (CMAT), to make up for the insufficiency of training data. To exploit the synthesized data, we propose the clause prediction (CP) object for multi-task learning, which forces the model to capture contextual features of the schema items. Besides, we add the fuzzy match and subword match to the schema linking strategy in RAT-SQL. As a result, our method significantly increases the recall and F1 value of schema linking and achieves a competitive result with RAT-SQL and GraPPa on Spider.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A schema item may have multiple classes. In this case, the target distribution follows the discrete uniform distribution.
References
Bogin, B., Berant, J., Gardner, M.: Representing schema structure with graph neural networks for text-to-SQL parsing. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4560–4565 (2019)
Cao, R., Zhu, S., Liu, C., Li, J., Yu, K.: Semantic parsing with dual learning. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 51–64 (2019)
Deng, X., Hassan, A., Meek, C., Polozov, O., Sun, H., Richardson, M.: Structure-grounded pretraining for text-to-SQL. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1337–1350 (2021)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Levenshtein, V.I., et al.: Binary codes capable of correcting deletions, insertions, and reversals. In: Soviet physics doklady, vol. 10, pp. 707–710. Soviet Union (1966)
Lin, Y., Yang, S., Stoyanov, V., Ji, H.: A multi-lingual multi-task architecture for low-resource sequence labeling. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 799–809 (2018)
Loshchilov, I., Hutter, F.: Fixing weight decay regularization in adam (2018)
Rubin, O., Berant, J.: SmBoP: semi-autoregressive bottom-up semantic parsing. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 311–324 (2021)
Shao, B., et al.: Weakly supervised multi-task learning for semantic parsing. In: IJCAI, pp. 3375–3381 (2019)
Shi, P., et al.: Learning contextual representations for semantic parsing with generation-augmented pre-training. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 13806–13814 (2021)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wang, B., Shin, R., Liu, X., Polozov, O., Richardson, M.: RAT-SQL: relation-aware schema encoding and linking for text-to-SQL parsers. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7567–7578 (2020)
Yu, T., et al.: GraPPa: grammar-augmented pre-training for table semantic parsing. arXiv preprint arXiv:2009.13845 (2020)
Yu, T., et al.: Spider: a large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3911–3921 (2018)
Zettlemoyer, L.S., Collins, M.: Learning to map sentences to logical form: structured classification with probabilistic categorial grammars. In: Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, pp. 658–666 (2005)
Acknowledgements
This research was supported by Peking University Education Big Data Project, grant number 2020ZDB04.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chang, C. (2021). CMAT: Column-Mask-Augmented Training for Text-to-SQL Parsers. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Communications in Computer and Information Science, vol 1517. Springer, Cham. https://doi.org/10.1007/978-3-030-92310-5_59
Download citation
DOI: https://doi.org/10.1007/978-3-030-92310-5_59
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92309-9
Online ISBN: 978-3-030-92310-5
eBook Packages: Computer ScienceComputer Science (R0)