skip to main content
10.1145/3568562.3568583acmotherconferencesArticle/Chapter ViewAbstractPublication PagessoictConference Proceedingsconference-collections
research-article

Method for Evaluating Quality of Automatically Generated Product Descriptions

Published: 01 December 2022 Publication History

Abstract

In recent years, deep learning accuracy has improved. Widely diverse content has been generated automatically. Along with studies of automatic content generation, difficulties associated with evaluation methods for the generated content have become important. The evaluation methods differ among the generated content types. As described herein, we propose evaluation methods aimed at motivating readers to purchase products through automatically generated product descriptions. Specifically, after extracting and modifying the 17 evaluation items from related research, we categorized evaluation items according to three axes signifying grammar, content, and attributes. Furthermore, to demonstrate the benefits of the proposed axes, we conduct user experiments of two types using the results obtained from deep learning of three types. Experiment 1 targets subjects who have not decided what to buy. Experiment 2 targets subjects with an image of a product that they want to buy. As results, we identified the benefits of the proposed method in both cases.

References

[1]
Masayuki Asahara. 2018. NWJC2Vec: Word embedding dataset from ‘NINJAL Web Japanese Corpus’. Terminology: International Journal of Theoretical and Applied Issues in Specialized Communication 24, 2 (Feb. 2018), 7–25.
[2]
Elizabeth Clark, Asli Celikyilmaz, and Noah A Smith. 2019. Sentence mover’s similarity: Automatic evaluation for multi-sentence texts. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2748–2760.
[3]
Siyu Duan, Wei Li, Jing Cai, Yancheng He, and Yunfang Wu. 2021. Query-Variant Advertisement Text Generation with Association Knowledge. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 412–421.
[4]
Jonas Freiknecht and Wolfgang Effelsberg. 2020. Procedural Generation of Interactive Stories Using Language Models. In International Conference on the Foundations of Digital Games (Bugibba, Malta) (FDG ’20). Association for Computing Machinery, New York, NY, USA, Article 97, 8 pages. https://doi.org/10.1145/3402942.3409599
[5]
Kenji Fukumoto, Rinji Suzuki, Hiroyuki Terada, Masafumi Bato, and Akiyo Nadamoto. 2021. Comparison of Deep Learning Models for Automatic Generation of Product Description on E-commerce site. In Proceedings of the 23rd International Conference on Information Integration and Web Intelligence (iiWAS2021). 227–235.
[6]
Ben Goodrich, Vinay Rao, Peter J. Liu, and Mohammad Saleh. 2019. Assessing The Factual Accuracy of Generated Text. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (Anchorage, AK, USA). Association for Computing Machinery, New York, NY, USA, 166–175. https://doi.org/10.1145/3292500.3330955
[7]
Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations.
[8]
Junyi Li, Wayne Xin Zhao, Zhicheng Wei, Nicholas Jing Yuan, and Ji-Rong Wen. 2021. Knowledge-Based Review Generation by Coherence Enhanced Text Planning. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada) (SIGIR ’21). Association for Computing Machinery, New York, NY, USA, 183–192. https://doi.org/10.1145/3404835.3462865
[9]
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74–81.
[10]
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311–318.
[11]
Yu-Ping Ruan, Zhen-Hua Ling, and Xiaodan Zhu. 2020. Condition-Transforming Variational Autoencoder for Generating Diverse Short Text Conversations. 19, 6, Article 79 (oct 2020), 13 pages. https://doi.org/10.1145/3402884
[12]
Yu Sawai, Peinan Zhang, and Akifumi Yoshimoto. 2020(in Japanese). A Proposal for Evaluation Metrics and Assistant Tools for Automatically Generated Textual Ads. In The proceedings of the 34th Annual Conference of the Japanese Society for Artificial Intelligence. 4pages.
[13]
Fei Sun, Peng Jiang, Hanxiao Sun, Changhua Pei, Wenwu Ou, and Xiaobo Wang. 2018. Multi-source pointer network for product title summarization. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 7–16.
[14]
Chris van der Lee, Albert Gatt, Emiel van Miltenburg, Sander Wubben, and Emiel Krahmer. 2019. Best practices for the human evaluation of automatically generated text. In Proceedings of the 12th International Conference on Natural Language Generation. Association for Computational Linguistics, Tokyo, Japan, 355–368. https://doi.org/10.18653/v1/W19-8643
[15]
Yongzhen Wang, Jian Wang, Heng Huang, Hongsong Li, and Xiaozhong Liu. 2020. Evolutionary product description generation: A dynamic fine-tuning approach leveraging user click behavior. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 119–128.
[16]
Feifei Xu, Xinpeng Wang, Yunpu Ma, Volker Tresp, Yuyi Wang, Shanlin Zhou, and Haizhou Du. 2020. Controllable Multi-Character Psychology-Oriented Story Generation. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (Virtual Event, Ireland) (CIKM ’20). Association for Computing Machinery, 1675–1684. https://doi.org/10.1145/3340531.3411937
[17]
Jingjing Xu, Xuancheng Ren, Junyang Lin, and Xu Sun. 2018. Diversity-promoting GAN: A cross-entropy based generative adversarial network for diversified text generation. In Proceedings of the 2018 conference on empirical methods in natural language processing. 3940–3949.
[18]
Ruqing Zhang, Jiafeng Guo, Yixing Fan, Yanyan Lan, and Xueqi Cheng. 2020. Query Understanding via Intent Description Generation. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 1823–1832. https://doi.org/10.1145/3340531.3411999
[19]
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675(2019).
[20]
Tao Zhang, Jin Zhang, Chengfu Huo, and Weijun Ren. 2019. Automatic generation of pattern-controlled product description in e-commerce. In The World Wide Web Conference. 2355–2365.
[21]
Zhiling Zhang and Kenny Zhu. 2021. Diverse and Specific Clarification Question Generation with Keywords. In Proceedings of the Web Conference 2021 (Ljubljana, Slovenia) (WWW ’21). Association for Computing Machinery, New York, NY, USA, 3501–3511. https://doi.org/10.1145/3442381.3449876
[22]
Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Texygen: A benchmarking platform for text generation models. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 1097–1100.

Index Terms

  1. Method for Evaluating Quality of Automatically Generated Product Descriptions

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    SoICT '22: Proceedings of the 11th International Symposium on Information and Communication Technology
    December 2022
    474 pages
    ISBN:9781450397254
    DOI:10.1145/3568562
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 December 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Content generation
    2. Deep Learning
    3. Experiment method
    4. Product description

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    SoICT 2022

    Acceptance Rates

    Overall Acceptance Rate 147 of 318 submissions, 46%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 88
      Total Downloads
    • Downloads (Last 12 months)49
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 23 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media