Abstract
Embedded representations that express the user’s personality are essential for personalizing the output of machine learning. However, annotating training data to learn the embedding is difficult because one cannot directly observe a person’s internal personality. This paper proposes a method for learning user embedding representations from social networking service data to make language models behave with personality. The method focuses on text posted by social networking service users and obtains the user’s embedded representation by learning a model that predicts and generates sentences before and after the text input to the social networking service. Evaluation experiments showed that the proposed method could learn embedded expressions that reflected the user’s attributes, such as location or personality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Christian, H., Suhartono, D., Chowanda, A., Zamli, K.Z.: Text based personality prediction from multiple social media data sources using pre-trained language model and model averaging. J. Big Data 8(1), 1–20 (2021). https://doi.org/10.1186/s40537-021-00459-1
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies NAACL-HLT, pp. 4171–4186. Association for Computational Linguistics (2019)
Ding, T., Bickel, W.K., Pan, S.: Multi-view unsupervised user feature embedding for social media-based substance use prediction. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017), pp. 2275–2284 (Sep 2017)
Huang, X., Paul, M.J., Dernoncourt, F., Burke, R., Dredze, M.: User factor adaptation for user embedding via multitask learning. In: Proceedings of the Second Workshop on Domain Adaptation for NLP, pp. 172–182 (Apr 2021)
Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: Proceedings of the 31st International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 32, pp. 1188–1196 (2014)
Li, L., Zhang, Y., Chen, L.: Personalized transformer for explainable recommendation. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL 2021), pp. 4947–4957 (2021)
Miller, M.E., Spatz, E.: A unified view of a human digital twin. Hum.-Intell. Syst. Integration 4, 23–33 (2022)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), p. 807–814 (2010)
Pal, A., Eksombatchai, C., Zhou, Y., Zhao, B., Rosenberg, C., Leskovec, J.: Pinnersage: Multi-modal user embedding framework for recommendations at pinterest. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD 2020), pp. 2311–2320 (2020)
Pan, S., Ding, T.: Social media-based user embedding: A literature review. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI 2019)
Plant, R., Gkatzia, D., Giuffrida, V.: CAPE: Context-aware private embeddings for private language learning. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing(EMNLP 2021), pp. 7970–7978 (Nov 2021)
Polignano, M., Musto, C., de Gemmis, M., Lops, P., Semeraro, G.: Together is better: Hybrid recommendations combining graph embeddings and contextualized word representations. In: Fifteenth ACM Conference on Recommender Systems (RecSys 2021)
Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020)
Rothmann, S., Coetzer, E.: The big five personality dimensions and job performance. South African J. Indust. Psychol. 29, 68–74 (10 2003)
Shengli, W.: Is human digital twin possible? Comput. Methods Programs Biomed. Update 1, 100014 (2021)
Song, Y., Lee, C.J.: Learning user embeddings from emails. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: vol. 2, Short Papers (EACL 2017), pp. 733–738, Valencia, Spain (Apr 2017)
Tagliabue, J., Yu, B., Bianchi, F.: The embeddings that came in from the cold: Improving vectors for new and rare products with content-based inference. In: Fourteenth ACM Conference on Recommender Systems (RecSys 2020), pp. 577–578 (2020)
Uban, A.S., Chulvi, B., Rosso, P.: Understanding patterns of anorexia manifestations in social media data with deep learning. In: Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Improving Access, pp. 224–236 (Jun 2021)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Vu, H., Abdurahman, S., Bhatia, S., Ungar, L.: Predicting responses to psychological questionnaires from participants’ social media posts and question text embeddings. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1512–1524 (2020)
Wang, T., Brovman, Y.M., Madhvanath, S.: Personalized embedding-based e-commerce recommendations at ebay. arXiv. vol. cs.IR (2021)
Welch, C., Gu, C., Kummerfeld, J., Perez-Rosas, V., Mihalcea, R.: Leveraging similar users for personalized language modeling with limited data. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL2022), pp. 1742–1752 (2022)
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., Weston, J.: Personalizing dialogue agents: I have a dog, do you have pets too? In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), pp. 2204–2213 (2018)
Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: Bertscore: Evaluating text generation with BERT. In: Proceedings of the 8th International Conference on Learning Representations (ICLR 2020) (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Otsuka, A., Hama, K., Nomoto, N., Ishii, R., Fukayama, A., Nakamura, T. (2023). Learning User Embeddings with Generating Context of Posted Social Network Service Texts. In: Coman, A., Vasilache, S. (eds) Social Computing and Social Media. HCII 2023. Lecture Notes in Computer Science, vol 14025. Springer, Cham. https://doi.org/10.1007/978-3-031-35915-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-35915-6_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35914-9
Online ISBN: 978-3-031-35915-6
eBook Packages: Computer ScienceComputer Science (R0)