Skip to main content
Log in

Improving sparsity and new user problems in collaborative filtering by clustering the personality factors

  • Published:
Electronic Commerce Research Aims and scope Submit manuscript

Abstract

In collaborative filtering recommender systems, items recommended to an active user are selected based on the interests of users similar to him/her. Collaborative filtering systems suffer from the ‘sparsity’ and ‘new user’ problems. The former refers to the insufficiency of data about users’ preferences and the latter addresses the lack of enough information about the new-coming user. Clustering users is an effective way to improve the performance of collaborative filtering systems in facing the aforementioned problems. In previous studies, users were clustered based on characteristics such as ratings given by them as well as their age, gender, occupation, and geographical location. On the other hand, studies show that there is a significant relationship between users’ personality traits and their interests. To alleviate the sparsity and new user problems, this paper presents a new collaborative filtering system in which users are clustered based on their ‘personality traits’. In the proposed method, the personality of each user is described according to the big-5 personality model and users with similar personality are placed in the same cluster using K-means algorithm. The unknown ratings of the sparse user-item matrix are then estimated based on the clustered users, and recommendations are found for a new user according to a user-based approach which relays on the interests of the users with similar personality to him/her. In addition, for an existing user in the system, recommendations are offered in an item-based approach in which the similarity of items is estimated based on the ratings of users similar to him/her in personality. The proposed method is compared to some former collaborative filtering systems. The results demonstrate that in facing the data sparsity and new user problems, this method reduces the mean absolute error and improves the precision of the recommendations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Elahi, M., Braunhofer, M., Ricci, F., & Tkalcic, M. (2013). Personality-based active learning for collaborative filtering recommender systems. In AI* IA 2013: Advances in Artificial Intelligence, pp. 360–371, Springer.

  2. Burke, R. (2007). Hybrid web recommender systems. In P. Brusilovsky, A. Kobsa, & W. Nejdl (Eds.), The Adaptive web, Volume 4321 of the series Lecture Notes in Computer Science (pp. 377–408). Springer, Berlin/Heidelberg.

  3. Bilge, A., & Polat, H. (2013). A scalable privacy-preserving recommendation scheme via bisecting k-means clustering. Information Processing and Management, 49, 912–927.

    Article  Google Scholar 

  4. Park, D. H., Kim, H. K., Choi, I. Y., & Kim, J. K. (2012). A literature review and classification of recommender systems research. Expert Systems with Applications, 39, 10059–10072.

    Article  Google Scholar 

  5. Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734–749.

    Article  Google Scholar 

  6. Ricci, F., Rokach, L., & Shapira, B. (2015). Recommender systems: Introduction and challenges. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender systems handbook (2nd ed., Vol. 54, pp. 1–34). Boston, MA: Springer US.

    Chapter  Google Scholar 

  7. Elkahky, A., Song, Y., & He, X. (2015). A multi-view deep learning approach for cross domain user modeling in recommendation systems. In International World Wide Web Conference Committee (IW3C2), Florence, Italy, May 18–22.

  8. den Oord, A. V., Dieleman, S., & Schrauwen, B. (2013). Deep content based music recommendation. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, (Eds.), Advances in Neural Information Processing Systems 26, pp. 2643–2651.

  9. Wang, H., Wang, N., & Yeung, D. (2015). Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Australia, August 10–13, pp. 1235–1244.

  10. Wang, X., & Wang, Y. (2014) Improving content-based and hybrid music recommendation using deep Llearning. In Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, Florida, USA, November 3–7, pp. 627–636.

  11. Cheng, H., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., et al. (2016). Wide & deep learning for recommender systems, In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, Boston, MA, USA, September 15, pp. 7–10.

  12. Bobadilla, J., Ortega, F., Hernando, A., & Gutiérrez, A. (2013). Recommender systems survey. Knowledge-Based Systems, 46, 109–132.

    Article  Google Scholar 

  13. Cacheda, F., Carneiro, V., Fernández, D., & Formoso, V. (2011). Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems. ACM Transactions on the Web (TWEB), 5, 2.

    Google Scholar 

  14. Dang, T. T., Duong, T. H., & Nguyen, H. S. (2014). A hybrid framework for enhancing correlation to solve cold-start problem in recommender systems. In Seventh IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), pp. 1–5.

  15. Hu, R. (2010). Design and user issues in personality-based recommender systems. In Proceedings of the Fourth ACM Conference on Recommender Systems, Como, Italy, pp. 357–360.

  16. Tsai, C. F., & Hung, C. (2012). Cluster ensembles in collaborative filtering recommendation. Applied Soft Computing, 12, 1417–1425.

    Article  Google Scholar 

  17. Tkalcic, M., & Chen, L. (2015). Personality and recommender systems. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender systems handbook (2nd ed., Vol. 54, pp. 715–739). Boston, MA: Springer.

    Chapter  Google Scholar 

  18. Xie, F., Xu, M., & Chen, Z. (2012). RBRA: A simple and efficient rating-based recommender algorithm to cope with sparsity in recommender systems. In 26th International Conference on Advanced Information Networking and Applications Workshops (WAINA), pp. 306–311.

  19. Ancy, S., & Unnikrishnan, V. (2015). A study on clustering and collaborative filtering. In International Conference on Computation of Power, Energy Information and Commuincation (ICCPEIC), Chennai, Tamilnadu, India, April 22–23, pp. 0039–0042.

  20. Feist, G. J. (2013). The scientific personality. In G. J. Feist & M. Gorman (Eds.), Handbook of the psychology of science (pp. 95–122). New York: Springer.

    Google Scholar 

  21. Amichai-Hamburger, Y., & Vinitzky, G. (2010). Social network use and personality. Computers in Human Behavior, 26(6), 1289–1295.

    Article  Google Scholar 

  22. Kalish, Y., & Robins, G. (2006). Psychological predispositions and network structure: The relationship between individual predispositions, structural holes and network closure. Social Networks, 28(1), 56–84.

    Article  Google Scholar 

  23. Qiu, L., Lin, H., Ramsay, J., & Yang, F. (2012). You are what you tweet: Personality expression and perception on twitter. Journal of Research in Personality, 46(6), 710–718.

    Article  Google Scholar 

  24. Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences of the United States of America, 110(15), 5802–5805.

    Article  Google Scholar 

  25. Skowron, M., Ferwerda, B., Tkalčič, M., & Schedl, M. (2016). Fusing social media cues: Personality prediction from Twitter and Instagram. In WWW’16 Companion, Proceedings of the 25th International Conference Companion on World Wide Web, Montreal, Quebec, Canada, April 11–15, pp. 107–108.

  26. Golbeck, J., Robles, C., & Turner, K. (2011). Predicting personality with social media. In Proceedings of the International Conference on Human Factors in Computing Systems, CHI 2011, Extended Abstracts Volume, Vancouver, BC, Canada, May 7–12, pp. 253–262.

  27. Olguın, D. O., Gloor, P. A., & Pentland, A. S. (2009).Capturing individual and group behavior with wearable sensors. In Proceedings of the 2009 AAAI Spring Symposium on Human Behavior Modeling, Stanford, CA.

  28. Roberts, S. G., Wilson, R., Fedurek, P., & Dunbar, R. (2008). Individual differences and personal social network size and structure. Personality and Individual Differences, 44(4), 954–964.

    Article  Google Scholar 

  29. Golkar, I., & Kaedi, M. (2016). Developing a model for estimating the extraversion degree of social network members using the information extracted from the graph structure. Journal of Modeling in Engineering, 13(43), 91–106.

    Google Scholar 

  30. Golkar Amnieh, I., & Kaedi, M. (2015). Using estimated personality of social network members for finding influential nodes in viral marketing. Cybernetics and Systems, 46(5), 355–378.

    Article  Google Scholar 

  31. Golbeck, J., Robles, C., Edmondson, M., & Turner, K. (2011). Predicting personality from Twitter. In 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing, Boston, MA, USA, October 9–11, pp. 149–156.

  32. Quercia, D., Lambiotte, R., Stillwell, D., Kosinski, M., & Crowcroft J. (2012). The personality of popular Facebook users. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, Seattle, WA, USA, February 11–15, pp. 955–964.

  33. Quercia, D., Kosinski, M., Stillwell, D., & Crowcroft, J. (2011). Our twitter profiles, our selves: Predicting personality with twitter. In IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing, Boston, MA, USA, October 9–11, pp. 180–185.

  34. Farnadi, G., Sitaraman, G., Sushmita, S., Celli, F., Kosinski, M., Stillwell, D., et al. (2016). Computational personality recognition in social media. User Modeling and User-Adapted Interaction (Special Issue on Personality in Personalized Systems), 26(2), 109–114.

    Article  Google Scholar 

  35. McCrae, R. R., & John, O. P. (1992). An introduction to the five-factor model and its applications. Journal of Personality, 60, 175–215.

    Article  Google Scholar 

  36. Oren, T., & Ghasem-Aghaee, N. (2003). Personality representation processable in fuzzy logic for human behavior simulation. In Proceedings of the 2003 Summer Computer Simulation Conference, Montreal, PQ, Canada, July 20–24, pp. 11–18.

  37. Costa, P. T., & MacCrae, R. R. (1992). Revised NEO personality inventory (NEO PI-R) and NEO five-factor inventory (NEO FFI): Professional manual. Odessa: Psychological Assessment Resources.

    Google Scholar 

  38. Rosellini, A. J., & Brown, T. A. (2011). The NEO Five-factor inventory: Latent structure and relationships with dimensions of anxiety and depressive disorders in a large clinical sample. Assessment, 18(1), 27–38.

    Article  Google Scholar 

  39. Sajadinejad, M. S., Molavi, H., Asgari, K., Kalantari, M., & Adibi, P. (2012). Personality dimensions and type D personality in female patients with ulcerative colitis. Journal of Research in Medical Sciences, 17(10), 898–904.

    Google Scholar 

  40. Xue, G. R., Lin, C., Yang, Q., Xi, W., Zeng, H.-J., Yu, Y., et al. (2005). Scalable collaborative filtering using cluster-based smoothing. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Salvador, Brazil, August 15–19, pp. 114–121.

  41. Wang, P. (2012). A personalized collaborative recommendation approach based on clustering of customers. Physics Procedia, 24, 812–816.

    Article  Google Scholar 

  42. Ullah, F., Sarwar, G., Lee, S. C., Park, Y. K., Moon, K. D., & Kim, J. T. (2012). Hybrid recommender system with temporal information. In International Conference on Information Networking (ICOIN), Bali, Indonesia, February 1–3, pp. 421–425.

  43. Li, X. (2011). Collaborative filtering recommendation algorithm based on cluster. In International Conference on Computer Science and Network Technology (ICCSNT), December 24–26, pp. 2682–2685.

  44. Altingovde, I. S., Subakan, O. N., & Ulusoy, O. (2013). Cluster searching strategies for collaborative recommendation systems. Information Processing and Management, 49, 688–697.

    Article  Google Scholar 

  45. Yanxiang, L., Deke, G., Fei, C., & Honghui, C. (2013). User-based clustering with top-N recommendation on cold-start problem. In Third International Conference on Intelligent System Design and Engineering Applications (ISDEA), Hong Kong, China, January 16–18, pp. 1585–1589.

  46. Dakhel G. M., & Mahdavi, M. (2011). A new collaborative filtering algorithm using K-means clustering and neighbors’ voting. In 11th International Conference on Hybrid Intelligent Systems (HIS), Malacca, Malaysia, December 5–8, pp. 179-184.

  47. Tintarev, N., Dennis, M., & Mastho, J. (2013). User modeling, adaptation, and personalization. In Lecture Notes in Computer Science, 7899, pp. 190–202.

  48. Cantador, I., Fernández-tobías, I., & Bellogín, A. (2013). Relating personality types with user preferences in multiple entertainment domains. In EMPIRE 1st Workshop on Emotions and Personality in Personalized Services, Rome, June 10.

  49. Roshchina, A., Cardiff, J., & Rosso, P. (2011). A comparative evaluation of personality estimation algorithms for the twin recommender system. In Proceedings of the 3rd International Workshop on Search and Mining User-Generated Contents, Glasgow, United Kingdom, October 24–28, pp. 11–18.

  50. Hu R., & Pu, P. (2011). Enhancing collaborative filtering systems with personality information. In Proceedings of the Fifth ACM Conference on Recommender Systems, Chicago, IL, USA, October 23–27, pp. 197–204.

  51. Hu R., & Pu, P. (2009). Acceptance issues of personality-based recommender systems. In Proceedings of the Third ACM Conference on Recommender Systems, pp. 221–224.

  52. Hu, R., & Pu, P. (2010). Using personality information in collaborative filtering for new users. In Proceedings of the 2nd ACM RecSys’10 Workshop on Recommender Systems and the Social Web, New York, October 22–25, pp. 17–24.

  53. Tkalcic, M., Kunaver, M., Tasic, J., & Kosir, A. (2009). Personality based user similarity measure for a collaborative recommender system. In Proceedings of the 5th Workshop on Emotion in Human-Computer Interaction-Real World Challenges, pp. 30–37.

  54. Tkalčič, M., Kunaver, M., Košir, A., & Tasič, J. (2011). Addressing the new user problem with a personality based user similarity measure. In F. Ricci, G. Semeraro, M. de Gemmis, P. Lops, J. Masthoff, F. Grasso, & J. Ham, (Eds.), Joint Proceedings of the Workshop on Decision Making and Recommendation Acceptance Issues in Recommender Systems (DEMRA 2011) and the 2nd Workshop on User Models for Motivational Systems: The Affective and the Rational Routes to Persuasion (UMMS 2011), CEUR Workshop Proceedings, 740, pp. 19–26.

  55. Jeong, W. H., Kim, S. J., Park, D. S., & Kwak, J. (2013). Performance improvement of a movie recommendation system based on personal propensity and secure collaborative filtering. Journal of Information Processing Systems, 9, 157–172.

    Article  Google Scholar 

  56. Wu, W., Chen, L., & He, L. (2013). Using personality to adjust diversity in recommender systems. In Proceedings of the 24th ACM Conference on Hypertext and Social Media, Paris, France, May 1–3, pp. 225–229.

  57. Hu, R., & Pu, P. (2010). A study on user perception of personality-based recommender systems. In User Modeling, Adaptation, and Personalization, Volume 6075 of the series Lecture Notes in Computer Science, pp. 291–302, Springer.

  58. Lampropoulos, A. S., Lampropoulou, P. S., & Tsihrintzis, G. A. (2012). A cascade-hybrid music recommender system for mobile services based on musical genre classification and personality diagnosis. Multimedia Tools and Applications, 59, 241–258.

    Article  Google Scholar 

  59. Braunhofer, M., Elahi, M., & Ricci, F. (2015). User personality and the new user problem in a context-aware point of interest recommender system. In I. Tussyadiah, & A. Inversini (Eds.), Information and Communication Technologies in Tourism, Proceedings of the International Conference in Lugano, Switzerland, February 3–6, pp. 537–549.

  60. Wang, J. (2015). A collaborative filtering systems based on personality information. In 2015 International Industrial Informatics and Computer Engineering Conference, Shaanxi, China, January pp. 10–11.

  61. Braunhofer, M., Codina, V., & Ricci, F. (2014). Switching hybrid for cold-starting context-aware recommender systems. In RecSys ‘14, Proceedings of the 8th ACM Conference on Recommender Systems, Silicon Valley, USA, October 6–10, pp. 349–352.

  62. Braunhofer, M., Elahi, M., Ge, M., & Ricci, F. (2014). Context dependent preference acquisition with personality-based active learning in mobile recommender systems. In P. Zaphiris, & A. Ioannou (Eds.), Learning and Collaboration Technologies. Technology-Rich Environments for Learning and Collaboration. Springer. Volume 8524 of the series Lecture Notes in Computer Science, Heraklion, Crete, Greece, June 22–27, pp. 105–116.

    Google Scholar 

  63. Fernández-Tobías, I., Braunhofer, M., Elahi, M., Ricci, F., & Cantador, I. (2016). Alleviating the new user problem in collaborative filtering by exploiting personality information. User Modeling and User-Adapted Interaction (Special Issue on Personality in Personalized Systems), 26(2), 221–255.

    Article  Google Scholar 

  64. Sadr, M. M. (2016). The role of personality traits predicting emotion regulation strategies. International Academic Journal of Humanities, 3(4), 13–24.

    Google Scholar 

  65. Revelle, W., Wilt, J., & Rosenthal, A. (2010). Individual differences in cognition: New methods for examining the personality-cognition link. In A. Gruszka, G. Matthews, & B. Szymura (Eds.), Handbook of individual differences in cognition (pp. 27–49). New York: Springer.

    Chapter  Google Scholar 

  66. Steel, P., Schmidt, J., & Shultz, J. (2008). Refining the relationship between personality and subjective well-being. Psychological Bulletin, 134(1), 138–161.

    Article  Google Scholar 

  67. Meriac, J. P., Hoffman, B. J., Woehr, D. J., & Fleisher, M. S. (2008). Further evidence for the validity of assessment center dimensions: A meta-analysis of the incremental criterion-related validity of dimension ratings. Journal of Applied Psychology, 93(5), 1042–1052.

    Article  Google Scholar 

  68. Hogan, J., Barrett, P., & Hogan, R. (2007). Personality measurement, faking, and employment selection. Journal of Applied Psychology, 92(5), 1270–1285.

    Article  Google Scholar 

  69. Su, X., & Khoshgoftaar, T. M. (2009). A survey of collaborative filtering techniques. Advances in Artificial Intelligence, 2009, 4–23.

    Article  Google Scholar 

  70. Wang, Q., Cao, W., & Liu, Y. (2014). A Novel Clustering Based Collaborative Filtering Recommendation System Algorithm. In Advanced Technologies, Embedded and Multimedia for Human-centric Computing, Volume 260 of the series Lecture Notes in Electrical Engineering, pp. 673–680, Springer.

  71. Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001). Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web, Hong Kong, China, May 1–5, pp. 285–295.

  72. Amatriain, X., & Pujol, J. M. (2015). Data mining methods for recommender systems. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender systems handbook (2nd ed., Vol. 54, pp. 227–262). Boston, MA: Springer US.

    Chapter  Google Scholar 

  73. Ghazarian, S., & Nematbakhsh, M. A. (2015). Enhancing memory-based collaborative filtering for group recommender systems. Expert Systems with Applications, 42, 3801–3812.

    Article  Google Scholar 

  74. Rubens, N., Elahi, M., Sugiyama, M., & Kaplan, D. (2015). Active Learning in Recommender Systems. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender systems handbook (2nd ed., pp. 809–846). Boston, MA: Springer US.

    Chapter  Google Scholar 

  75. Scandell, D. J., & Wlazelek, B. G. (1996). Self-presentation strategies on the Neo-five factor inventory: Implications for detecting faking. Psychological Reports, Part 2, 79(3), 1115–1121.

    Article  Google Scholar 

  76. Fronczyk, K. (2014). The identification of random or careless responding in questionnaires: The example of the NEO-FFI. Annals of Psychology, 2, 457–473.

    Google Scholar 

  77. Cattell, R. B., Eber, H. W., & Tatsuoka, M. M. (1970). Handbook for the sixteen personality factor questionnaire (16PF). Champaign, IL: IPAT.

    Google Scholar 

  78. Haertzen, C. A., & Ross, F. E. (1978). Using four chance profiles to estimate carelessness. Psychological Reports, 41, 1079–1087.

    Article  Google Scholar 

  79. Marjanovic, Z., Holden, R., Struthers, W., Cribbie, R., & Greenglass, E. (2015). The inter-item standard deviation (ISD): An index that discriminates between conscientious and random responders. Personality and Individual Differences, 84, 79–83.

    Article  Google Scholar 

  80. PourMohammadBagher, L., Kaedi, M., Ghasem-Aghaee, N., & Ören, T. I. (2007). Anger evaluation for fuzzy agents with dynamic personality. Mathematical and Computer Modelling of Dynamical Systems, 15(6), 535–553.

    Article  Google Scholar 

  81. Ghasem-Aghaee, N., PoorMohamadBagher, L., Kaedi, M., & Ören, T. I. (2007). Anger filter in agent simulation of human behavior, In Proceedings of the 18th IASTED International Conference: Modelling and Simulation, Montreal, QC, Canada, May 30–June 1, pp. 44–46.

Download references

Acknowledgements

The authors would like to thank Dr. Mohammad Ali Nematbakhsh for his support and assistance to improve the manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Marjan Kaedi.

Appendix

Appendix

The NEO-FFI questionnaire comprises 60 items, 12 items to assess each of the five personality factors (six items are positively worded and six items are negatively worded for each personality factor). For each item, the respondent should describe how accurately that item describes him/her. The respondent selects the response based on the five-point Likert scale (ranging from strongly disagree to strongly agree). Then, the following scores are assigned to the answers of positively worded items:

Strongly disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), Strongly agree (5) and the following scores are used for the negatively worded items:

Strongly disagree (5), Disagree (4), Neither agree nor disagree (3), Agree (2), Strongly agree (1).

Finally, the personality factors are assessed by summing up the scores. The NEO-FFI is a copyrighted questionnaire. Because of copyright compliance policy, only few sample statements are listed below:

Personality factor

Sample statement

Positive/negative

Neuroticism

I worry about things

Positive

Extraversion

I think a lot before I speak or act

Negative

Conscientiousness

I get chores done right away

Positive

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yusefi Hafshejani, Z., Kaedi, M. & Fatemi, A. Improving sparsity and new user problems in collaborative filtering by clustering the personality factors. Electron Commer Res 18, 813–836 (2018). https://doi.org/10.1007/s10660-018-9287-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10660-018-9287-x

Keywords