research-article

Transformer-based Question Text Generation in the Learning System

Authors:

Jun LiAuthors Info & Claims

ICIAI '22: Proceedings of the 2022 6th International Conference on Innovation in Artificial Intelligence

Pages 50 - 56

https://doi.org/10.1145/3529466.3529484

Published: 04 June 2022 Publication History

Abstract

Question text generation from the triple in knowledge graph exists some challenges in learning system. One is the generated question text is difficult to be understood; the other is it considers few contexts. Therefore, this paper focuses on question text generation. Based on the traditional Bi-LSTM+Attention network model, we import Transformer model into question generation to get the simple question with some triples. In addition, this paper proposes a method to get the diverse expressions of questions (a variety of expressions of a question), that is, to take advantage of the semantic similarity algorithm based on Bi-LSTM with the help of a question database constructed in advance. Finally, a corresponding comparison experiment is designed, and the experimental results demonstrated that the accuracy of question generation experiment based on the Transformer model is 8.36% higher than the traditional Bi-LSTM + Attention network model.

References

[1]

The Refined Learning Process Modeling and the Analysis of the Learning Effect Based on PLS-SEM[D], Wuhan University of Technology, 2018

[2]

Ni J, Young T, V Pandelea, Recent Advances in Deep Learning-based Dialogue Systems[J]. 2021.

[3]

Yin W, Yu M, Xiang B. Simple Question Answering by Attentive Convolutional Neural Network[C]. International Conference on Computational Linguistics (COLING), Barcelona, Spain, 2016, 1746-1756.

[4]

Hao Y C, Zhang Y Z, Liu K, He S Z. An End-to-End Model for Question Answering over Knowledge Base with Cross Attention Combining Global Knowledge[C]. Association for Computational Linguistics (ACL), New York, USA, 2017, 221-231.

[5]

Z Xie, Z Zeng, G Zhou. Knowledge Base Question Answering Based on Deep Learning Models[J]. Natural Language Understanding and Intelligent Applications (NLPCC), Beijing, China, 2016, 300-311.

[6]

DieterFensel, Umutcanimek, KevinAngele, Introduction: What Is a Knowledge Graph?[M]. 2020.

[7]

Tang J, Yang Y, Carton S, Context-aware Natural Language Generation with Recurrent Neural Networks[J]. 2016.

[8]

I Sutskever, O Vinyals, QV Le. Sequence to sequence learning with neural networks[J]. Advances in neural information, 2014, 1-9.

[9]

Kyunghyun Cho, Dzmitry Bahdanau. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation[C]. Association for Computational Linguistics(ACL), New York, USA, 2014, 67-88.

[10]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit. Attention Is All Your Need[J]. Computation and Language, 2017, 15:5.

[11]

Z Yang, D Yang, C Dyer, X He, A Smola. Hierarchical attention networks for document classification[C]. Association for Computational Linguistics (ACL), Berlin, Germany, 2016, 300-312.

[12]

Santos C N D, Zadrozny B . Learning Character-level Representations for Part-of-Speech Tagging[C]. International Conference on Machine Learning (ICML). JMLR.org, 2014.

[13]

Jacob, Devlin and Ming-Wei, Chang and Kenton, Lee and Kristina, Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of NAACL-HLT 2019.

[14]

Wu C S, Hoi S, Socher R, TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue[J]. 2020.

[15]

Zhengya Z, Jin W, Hongmei C, An Approach for Spatial Index of Text Information Based on Cosine Similarity[J]. Computer Science, 2005, 32(9):160-163.

[16]

Kishore Papineni, Salim Roukos, Todd Ward. Bleu: A Method for Automatic Evaluation of Machine Translation[C]. Association for Computational Linguistics(ACL), New York, USA,2002, 311-318.

[17]

Vasile Rus, Mihai C Lintean. A Comparison of Greedy and Optimal Assessment of Natural Language Student Input Using Word-to-Word Similarity Metrics[C]. North American Chapter of the ACL(NAACL), New York, USA, 2012, 157-162.

Cited By

Liu FCao QHuang XLiu H(2025)Sentimentally enhanced conversation recommender systemComplex & Intelligent Systems10.1007/s40747-024-01766-911:2Online publication date: 8-Jan-2025
https://doi.org/10.1007/s40747-024-01766-9
Li BYang PSun YHu ZYi M(2024)Advances and challenges in artificial intelligence text generation人工智能文本生成的进展与挑战Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.230041025:1(64-83)Online publication date: 8-Feb-2024
https://doi.org/10.1631/FITEE.2300410
Liu HCao QHuang XLiu FZhang CAn J(2024)Multi-source information contrastive learning collaborative augmented conversational recommender systemsComplex & Intelligent Systems10.1007/s40747-024-01442-y10:4(5529-5543)Online publication date: 11-May-2024
https://doi.org/10.1007/s40747-024-01442-y
Show More Cited By

Recommendations

Multi-level wavelet network based on CNN-Transformer hybrid attention for single image deraining
Abstract
Removing rain streaks from rainy images can improve the accuracy of computer vision applications such as object detection. In order to make full use of the frequency domain analysis characteristics of wavelet and combine the advantages of ...
A deep learning methods for intrusion detection systems based machine learning in MANET
SCA '19: Proceedings of the 4th International Conference on Smart City Applications

Deep learning is a subset of machine learning where algorithms are created and function similar to those in machine learning, but there are numerous layers of these algorithms each providing a different interpretation to the data it feeds on. Mobile Ad-...
DATransformer: Deep Attention Transformer
ICIIT '24: Proceedings of the 2024 9th International Conference on Intelligent Information Technology

Large-scale Transformer-based models have garnered significant attention in recent years. Due to their large number of parameters, these models demand substantial computing resources and time. Therefore, it is imperative to enhance their efficiency. To ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICIAI '22: Proceedings of the 2022 6th International Conference on Innovation in Artificial Intelligence

March 2022

240 pages

ISBN:9781450395502

DOI:10.1145/3529466

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 June 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICIAI 2022

ICIAI 2022: 2022 the 6th International Conference on Innovation in Artificial Intelligence

March 4 - 6, 2022

Guangzhou, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
92
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu FCao QHuang XLiu H(2025)Sentimentally enhanced conversation recommender systemComplex & Intelligent Systems10.1007/s40747-024-01766-911:2Online publication date: 8-Jan-2025
https://doi.org/10.1007/s40747-024-01766-9
Li BYang PSun YHu ZYi M(2024)Advances and challenges in artificial intelligence text generation人工智能文本生成的进展与挑战Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.230041025:1(64-83)Online publication date: 8-Feb-2024
https://doi.org/10.1631/FITEE.2300410
Liu HCao QHuang XLiu FZhang CAn J(2024)Multi-source information contrastive learning collaborative augmented conversational recommender systemsComplex & Intelligent Systems10.1007/s40747-024-01442-y10:4(5529-5543)Online publication date: 11-May-2024
https://doi.org/10.1007/s40747-024-01442-y
Berger JKoß JStamatakis MHoppe AEwerth RWartena C(2024)Question Generation Capabilities of “Small" Large Language ModelsNatural Language Processing and Information Systems10.1007/978-3-031-70242-6_18(183-194)Online publication date: 20-Sep-2024
https://doi.org/10.1007/978-3-031-70242-6_18
Zhang CHuang XAn J(2023)MACRExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.118981213:PBOnline publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1016/j.eswa.2022.118981

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten