Skip to main content
SpringerLink
Log in
Book cover

International Conference on Web Information Systems Engineering

WISE 2021: Web Information Systems Engineering – WISE 2021 pp 162–176Cite as

Multi-Task Learning with Personalized Transformer for Review Recommendation

  • Conference paper
  • First Online:

  • 1208 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13081))

Abstract

Drastic increase in item/product review volume has caused serious information overload. Traditionally, product reviews are exhibited in chronological or popularity order without personalization. The review recommendation model provides users with attractive reviews and efficient consumption experience, allowing users to grasp the characteristics of items in seconds. However, the sparsity of interactions between users and reviews appears to be a major challenge. And the multi-relationship context, especially potential semantic feature in reviews, is not fully exploited. To address these problems, Multi-Task Learning model incorporating Personalized Transformer (MTL-PT) is proposed to provide users with an interesting review list. It contains three tasks: the main task models user’s preference to reviews with the proposed poly aggregator, incorporating the user-item-aware semantic feature. Two auxiliary tasks model the quality of reviews, and user-item interactions, respectively. These tasks collaboratively learn the multi-relationship among user, item, and review. The shared latent features of user/item link the three tasks together. Especially, the personalized semantic features of reviews are also fused into the tasks with the proposed personalized transformer. Two new real-world datasets for personalized review recommendation are collected and constructed. Extensive experiments are conducted on them. Compared with the state-of-the-arts, the results validate the effectiveness of our model for review recommendation.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Since the inputs of personalized transformers in the user-review task and the item-review task are similar, here we only take the personalized transformer input in the user-review task as an example.

  2. 2.

    https://www.reddit.com/ and https://www.taptap.com/.

  3. 3.

    If user replies a review, we assume user is interested in the review, otherwise not.

  4. 4.

    The datasets and code would be public later.

  5. 5.

    huggingface.co/bert-base-uncased and huggingface.co/bert-base-chinese.

  6. 6.

    Since other non-personalized review recommendation models [2, 6] learn to reproduce the ranking of scores-based recommendation, and SR is the optimal situation of them, we didn’t select other non-personalized models as baseline any more.

References

  1. Agarwal, D., Chen, B., Pang, B.: Personalized recommendation of user comments via factor models. In: EMNLP, pp. 571–582 (2011)

    Google Scholar 

  2. Chen, C., Yang, Y., Zhou, J., Li, X., Bao, F.S.: Cross-domain review helpfulness prediction based on convolutional neural networks with auxiliary domain discriminators. In: NAACL, pp. 602–607 (2018)

    Google Scholar 

  3. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL, pp. 4171–4186 (2019)

    Google Scholar 

  4. Diaz, G.O., Ng, V.: Modeling and prediction of online product review helpfulness: a survey. In: ACL, pp. 698–708 (2018)

    Google Scholar 

  5. Du, J., Zheng, L., He, J., Rong, J., Wang, H., Zhang, Y.: An interactive network for end-to-end review helpfulness modeling. Data Sci. Eng. 5(3), 261–279 (2020)

    Article  Google Scholar 

  6. Fan, M., Feng, C., Guo, L., Sun, M., Li, P.: Product-aware helpfulness prediction of online reviews. In: WWW, pp. 2715–2721 (2019)

    Google Scholar 

  7. He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.: Neural collaborative filtering. In: WWW, pp. 173–182 (2017)

    Google Scholar 

  8. Huang, C., Jiang, W., Wu, J., Wang, G.: Personalized review recommendation based on users’ aspect sentiment. TOIT 20(4), 1–26 (2020)

    Article  Google Scholar 

  9. Humeau, S., Shuster, K., Lachaux, M., Weston, J.: Poly-encoders: architectures and pre-training strategies for fast and accurate multi-sentence scoring. In: ICLR (2020)

    Google Scholar 

  10. Lu, Y., Tsaparas, P., Ntoulas, A., Polanyi, L.: Exploiting social context for review quality prediction. In: WWW, pp. 691–700 (2010)

    Google Scholar 

  11. Ma, J., Wen, J., Zhong, M., Chen, W., Li, X.: MMM: multi-source multi-net micro-video recommendation with clustered hidden item representation learning. Data Sci. Eng. 4(3), 240–253 (2019)

    Article  Google Scholar 

  12. Martin, L., Pu, P.: Prediction of helpful reviews using emotions extraction. In: AAAI, pp. 1551–1557 (2014)

    Google Scholar 

  13. Moghaddam, S., Jamali, M., Ester, M.: Review recommendation: personalized prediction of the quality of online reviews. In: CIKM, pp. 2249–2252 (2011)

    Google Scholar 

  14. Moghaddam, S., Jamali, M., Ester, M.: ETF: extended tensor factorization model for personalizing prediction of review helpfulness. In: WSDM, pp. 163–172 (2012)

    Google Scholar 

  15. Momeni, E., Cardie, C., Diakopoulos, N.: A survey on assessment and ranking methodologies for user-generated content on the web. ACM Comput. Surv. 48(3), 41:1–41:49 (2016)

    Google Scholar 

  16. Paul, D., Sarkar, S., Chelliah, M., Kalyan, C., Nadkarni, P.P.S.: Recommendation of high quality representative reviews in e-commerce. In: RecSys, pp. 311–315 (2017)

    Google Scholar 

  17. Prado, T.R.P., Moro, M.M.: Review recommendation for points of interest’s owners. In: HT, pp. 295–304 (2017)

    Google Scholar 

  18. Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. CoRR arXiv:1205.2618 (2012)

  19. Suresh, V., Roohi, S., Eirinaki, M., Varlamis, I.: Using social data for personalizing review rankings. In: RecSys (2014)

    Google Scholar 

  20. Tang, J., Gao, H., Hu, X., Liu, H.: Context-aware review helpfulness rating prediction. In: RecSys, pp. 1–8 (2013)

    Google Scholar 

  21. Tang, J., Gao, H., Liu, H.: mTrust: discerning multi-faceted trust in a connected world. In: WSDM, pp. 93–102 (2012)

    Google Scholar 

  22. Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)

    Google Scholar 

  23. Wang, C., Chen, G., Wei, Q.: A temporal consistency method for online review ranking. Knowl. Based Syst. 143, 259–270 (2018)

    Article  Google Scholar 

  24. Wang, S., Li, C., Zhao, K., Chen, H.: Context-aware recommendations with random partition factorization machines. Data Sci. Eng. 2(2), 125–135 (2017)

    Article  Google Scholar 

  25. Wu, C., Wu, F., An, M., Huang, J., Huang, Y., Xie, X.: Neural news recommendation with attentive multi-view learning. In: IJCAI, pp. 3863–3869 (2019)

    Google Scholar 

  26. Wu, C., Wu, F., An, M., Huang, J., Huang, Y., Xie, X.: NPA: neural news recommendation with personalized attention. In: SIGKDD, pp. 2576–2584 (2019)

    Google Scholar 

  27. Xiong, W., Litman, D.J.: Empirical analysis of exploiting review helpfulness for extractive summarization of online reviews. In: COLING, pp. 1985–1995 (2014)

    Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (61902439, U19112031), Guangdong Basic and Applied Basic Research Foundation (2021A1515011902, 2019A1515011159), National Science Foundation for Post-Doctoral Scientists of China under Grant (2019M663237), Macao Young Scholars Program (UMMTP2020-MYSP-016), the Major Science and Technology Research Programs of Zhongshan City (2019B2006, 2019A40027, 2021A1003).

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Sun Yat-sen University, Guangzhou, China

    Haiming Wang, Wei Liu & Jian Yin

  2. Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, China

    Haiming Wang, Wei Liu & Jian Yin

  3. SKL of Internet of Things for Smart City, University of Macau, Macau, China

    Wei Liu

Authors
  1. Haiming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Liu .

Editor information

Editors and Affiliations

  1. School of Computer Science and Engineering, The University of New South Wales, Sydney, NSW, Australia

    Wenjie Zhang

  2. Peking University, Beijing, China

    Lei Zou

  3. Zayed University, Dubai, United Arab Emirates

    Zakaria Maamar

  4. Swinburne University of Technology, Melbourne, VIC, Australia

    Lu Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, H., Liu, W., Yin, J. (2021). Multi-Task Learning with Personalized Transformer for Review Recommendation. In: Zhang, W., Zou, L., Maamar, Z., Chen, L. (eds) Web Information Systems Engineering – WISE 2021. WISE 2021. Lecture Notes in Computer Science(), vol 13081. Springer, Cham. https://doi.org/10.1007/978-3-030-91560-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-91560-5_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-91559-9

  • Online ISBN: 978-3-030-91560-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation