Abstract
The language interaction between humans and robots is one of the critical issues in the field of home service robots. In particular, irrelevant information in feature vectors interferes with the extraction task during Chinese instruction parsing. Moreover, the relations between feature vectors of different time steps affect the accuracy of action sequence extraction. In this paper, overlapping action entities in Chinese instructions are labeled through span-based mode, and a Joint Extraction Model of Action Sequences with Partition Coding(JEAPC) is proposed for Chinese instructions. The JEAPC is divided into four modules: BERT, partition encoder, and two decoders. BERT is utilized to obtain the feature vector of each Chinese character in the instructions. The partition encoder is composed of an entity gate, and an action gate, in which the features in the vector are classified, and the irrelevant features are filtered through multi-dimensional vector operations. Furthermore, adversarial training is employed to improve the robustness of JEAPC. Extensive experiments are conducted on a self-built Chinese instructions dataset(FCI) and three entity and relation extraction datasets (CoNLL04, ADE, and SciERC). The experimental results show that the JEAPC can accurately generate action sequences from Chinese instructions and obtain optimal results compared to several competitive approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Martins, P.H., Custódio, L., Ventura, R.: A deep learning approach for understanding natural language commands for mobile service robots. arXiv preprint arXiv:1807.03053 (2018)
Ishikawa, S., Sugiura, K.: Target-dependent UNITER: a transformer-based multimodal language comprehension model for domestic service robots. IEEE Robot. Autom. Lett. 6(4), 8401–8408 (2021)
Chen, H., Tan, H., Kuntz, A., Bansal, M., Alterovitz, R.: Enabling robots to understand incomplete natural language instructions using commonsense reasoning. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, pp. 1963–1969. IEEE (2020)
Li, X., et al.: Entity-relation extraction as multi-turn question answering. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 1340–1350. Association for Computational Linguistics (2019)
Eberts, M., Ulges, A.: Span-based joint entity and relation extraction with transformer pre-training. In: Proceedings of the 24th European Conference on Artificial Intelligence (ECAI) - Including 10th Conference on Prestigious Applications of Artificial Intelligence (PAIS), vol. 325, pp. 2006–2013. Santiago de Compostela, Spain (2020)
Zhao, S., Cai, Z., Chen, H., Wang, Y., Liu, F., Liu, A.: Adversarial training based lattice LSTM for Chinese clinical named entity recognition. J. Biomed. Inform. 99, 103290 (2019)
Misra, D.K., Sung, J., Lee, K., Saxena, A.: Tell me dave: context-sensitive grounding of natural language to manipulation instructions. Int. J. Robot. Res. 35(1–3), 281–300 (2016)
Zhang, S., Jiang, J., He, Z., Zhao, X., Fang, J.: A novel slot-gated model combined with a key verb context feature for task request understanding by service robots. IEEE Access 7, 105937–105947 (2019)
Mensio, M., Bastianelli, E., Tiddi, I., Rizzo, G.: Mitigating bias in deep nets with knowledge bases: the case of natural language understanding for robots. In: Proceedings of the AAAI 2020 Spring Symposium on Combining Machine Learning and Knowledge Engineering in Practice (AAAI-MAKE), Palo Alto, CA, USA, vol. 2600, p. 20 (2020)
Sharma, S., Gupta, J., Tuli, S., Paul, R.: Goalnet: inferring conjunctive goal predicates from human plan demonstrations for robot instruction following. arXiv preprint arXiv:2205.07081 (2022)
Zhao, S., Hu, M., Cai, Z., Chen, H., Liu, F.: Dynamic modeling cross-and self-lattice attention network for Chinese NER. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 16, pp. 14515–14523. AAAI Press (2021)
Wei, Z., Su, J., Wang, Y., Tian, Y., Chang, Y.: A novel cascade binary tagging framework for relational triple extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1476–1488. Association for Computational Linguistics, Online (2020)
Wang, Y., Yu, B., Zhang, Y., Liu, T., Zhu, H., Sun, L.: Tplinker: single-stage joint extraction of entities and relations through token pair linking. In: Proceedings of the 28th International Conference on Computational Linguistics (COLING), Barcelona, Spain (Online), pp. 1572–1582. International Committee on Computational Linguistics (2020)
Yan, Z., Zhang, C., Fu, J., Zhang, Q., Wei, Z.: A partition filter network for joint entity and relation extraction. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online and Punta Cana, Dominican Republic, pp. 185–197. Association for Computational Linguistics (2021)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), Volume 1 (Long and Short Papers), Minneapolis, MN, USA, pp. 4171–4186. Association for Computational Linguistics (2019)
Joshi, M., Chen, D., Liu, Y., Weld, D.S., Zettlemoyer, L., Levy, O.: Spanbert: improving pre-training by representing and predicting spans. Trans. Assoc. Comput. Linguist. 8, 64–77 (2020)
Li, Y., Wang, C., Lin, Y., Lin, Y., Chang, L.: Span-based relational graph transformer network for aspect-opinion pair extraction. Knowl. Inf. Syst. 64(5), 1305–1322 (2022)
Li, Z., Song, M., Zhu, Y., Zhang, L.: Chinese nested named entity recognition based on boundary prompt. In: Yuan, L., Yang, S., Li, R., Kanoulas, E., Zhao, X. (eds.) WISA 2023. LNCS, vol. 14094, pp. 331–343. Springer, Singapore (2023). https://doi.org/10.1007/978-981-99-6222-8_28
Nong, W., Zhang, T., Yang, S., Hu, N., He, X.: HfGCN: hierarchical fused GCN for joint entity and relation extraction. In: 2021 IEEE International Conference on Big Knowledge (ICBK), Auckland, New Zealand, pp. 307–314. IEEE (2021)
Tran, T., Kavuluru, R.: Neural metric learning for fast end-to-end relation extraction. arXiv preprint arXiv:1905.07458 (2019)
Luan, Y., Wadden, D., He, L., Shah, A., Ostendorf, M., Hajishirzi, H.: A general framework for information extraction using dynamic span graphs. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 3036–3046. Association for Computational Linguistics (2019)
Miyato, T., Dai, A.M., Goodfellow, I.J.: Adversarial training methods for semi-supervised text classification. In: 5th International Conference on Learning Representations (ICLR), Toulon, France. OpenReview.net (2017)
Roth, D., Yih, W.: A linear programming formulation for global inference in natural language tasks. In: Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004, Boston, Massachusetts, USA, pp. 1–8. Association for Computational Linguistics (2004)
Gurulingappa, H., Rajput, A.M., Roberts, A., Fluck, J., Hofmann-Apitius, M., Toldo, L.: Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. J. Biomed. Inform. 45(5), 885–892 (2012)
Luan, Y., He, L., Ostendorf, M., Hajishirzi, H.: Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), Brussels, Belgium, pp. 3219–3232. Association for Computational Linguistics (2018)
Acknowledgement
This work was supported in part by the Natural Science Foundation of Xinjiang Uygur Autonomous Region, China Grant No. 2022D01A59, National Natural Science Foundation of China under Grand No. U20A20167, Key Research Foundation of Integration of Industry and Education and the Development of New Business Studies Research Center, Innovation Capability Improvement Plan Project of Hebei Province under Grand No. 22567637H, Hebei Province Central Leading Local Science and Technology Development Project, 246Z1817G.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, B., Wang, H., Li, X., Zhao, F. (2024). JEAPC: A Joint Extraction Model of Action Sequence from Chinese Instructions for Home Service Robot. In: Jin, C., Yang, S., Shang, X., Wang, H., Zhang, Y. (eds) Web Information Systems and Applications. WISA 2024. Lecture Notes in Computer Science, vol 14883. Springer, Singapore. https://doi.org/10.1007/978-981-97-7707-5_44
Download citation
DOI: https://doi.org/10.1007/978-981-97-7707-5_44
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-7706-8
Online ISBN: 978-981-97-7707-5
eBook Packages: Computer ScienceComputer Science (R0)