Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Human Behavior Understanding

HBU 2016: Human Behavior Understanding pp 148–158Cite as

First Impressions - Predicting User Personality from Twitter Profile Images

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9997))

Abstract

This paper proposes a computer vision based pipeline for inferring the perceived personality of users from their Twitter profile images. We humans make impressions on a daily basis during communication. The perception of personality of a person gives information about the person’s behaviour and is an important attribute in developing rapport. The personality assessment in this paper is referred to as first impressions, which is similar to how humans create a mental image of another person by just looking at their profile pictures. In the proposed automated pipeline, hand crafted (engineered) and learnt feature descriptors are computed on user profile images. The effect of image background is assessed on the perception of the personality from a profile picture. A multivariate regression approach is used to predict the big five personality traits - agreeableness, conscientiousness, extraversion, openness and neuroticism. We study the correlation between the big five personality traits generated from Tweet analysis with the proposed profile image based framework. The experiments show high correlation for scene based first impressions perception. It is interesting to note that the results generated by analysing a profile image uploaded by a user in a particular point in time are in sync with the first impression traits generated by investigating Tweets posted over a longer duration of time.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://www.cs.cmu.edu/~deva/papers/face/index.html.

  2. 2.

    http://www.robots.ox.ac.uk/~vgg/research/caltech/phog.html.

  3. 3.

    http://www.cse.oulu.fi/CMV/Downloads/LPQMatlab.

  4. 4.

    https://github.com/sometimesfood/spact-matlab.

  5. 5.

    http://www.vlfeat.org/matconvnet.

References

  1. Ferwerda, B., Schedl, M., Tkalcic, M.: Personality & emotional states: understanding users music listening needs. In: UMAP 2015 Extended Proceedings (2015)

    Google Scholar 

  2. Lepri, B., Staiano, J., Rigato, G., Kalimeri, K., Finnerty, A., Pianesi, F., Sebe, N., Pentland, A.: The sociometric badges corpus: a multilevel behavioral dataset for social behavior in complex organizations. In: 2012 International Conference on Privacy, Security, Risk and Trust (PASSAT), pp. 623–628. IEEE (2012)

    Google Scholar 

  3. Mairesse, F., Walker, M.: Automatic recognition of personality in conversation. In: Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pp. 85–88. Association for Computational Linguistics (2006)

    Google Scholar 

  4. Mohammadi, G., Vinciarelli, A.: Automatic personality perception: prediction of trait attribution based on prosodic features. IEEE Trans. Affect. Comput. 3, 273–284 (2012)

    Article  Google Scholar 

  5. Pianesi, F., Mana, N., Cappelletti, A., Lepri, B., Zancanaro, M.: Multimodal recognition of personality traits in social interactions. In: Proceedings of the 10th International Conference on Multimodal Interfaces, pp. 53–60. ACM (2008)

    Google Scholar 

  6. Camastra, F., Vinciarelli, A.: Automatic personality perception. In: Camastra, F., Vinciarelli, A. (eds.) Machine Learning for Audio, Image and Video Analysis. Advanced Information and Knowledge Processing, pp. 485–498. Springer, London (2015)

    Chapter  Google Scholar 

  7. John, O.P., Srivastava, S.: The Big Five trait taxonomy: History, measurement, and theoretical perspectives. Handb. Pers. Theory Res. 2(1999), 102–138 (1999)

    Google Scholar 

  8. Joshi, J., Gunes, H., Goecke, R.: Automatic prediction of perceived traits using visual cues under varied situational context. In: ICPR, pp. 2855–2860 (2014)

    Google Scholar 

  9. Rojas, M., Masip, D., Todorov, A., Vitria, J.: Automatic prediction of facial trait judgments: appearance vs. structural models. PLoS ONE 6(8), e23323 (2011)

    Article  Google Scholar 

  10. Celiktutan, O., Gunes, H.: Automatic prediction of impressions in time and across varying context: personality, attractiveness and likeability. IEEE Trans. Affect. Comput. (2016)

    Google Scholar 

  11. Celiktutan, O., Eyben, F., Sariyanidi, E., Gunes, H., Schuller, B.: MAPTRAITS 2014: the first audio/visual mapping personality traits challenge. In: Proceedings of the 2014 Workshop on Mapping Personality Traits Challenge and Workshop, pp. 3–9. ACM (2014)

    Google Scholar 

  12. Kaya, H., Salah, A.A.: Continuous mapping of personality traits: a novel challenge and failure conditions. In: Proceedings of the 2014 Workshop on Mapping Personality Traits Challenge and Workshop, pp. 17–24. ACM (2014)

    Google Scholar 

  13. Lin, L., Czarnuch, S., Malhotra, A., Yu, L., Schröder, T., Hoey, J.: Affectively aligned cognitive assistance using Bayesian affect control theory. In: Pecchia, L., Chen, L.L., Nugent, C., Bravo, J. (eds.) IWAAL 2014. LNCS, vol. 8868, pp. 279–287. Springer, Heidelberg (2014). doi:10.1007/978-3-319-13105-4_41

    Google Scholar 

  14. Todorov, A., Porter, J.M.: Misleading first impressions different for different facial images of the same person. Psychol. Sci. 25(7), 1404–1417 (2014)

    Article  Google Scholar 

  15. McKeown, G., Valstar, M., Cowie, R., Pantic, M., Schroder, M.: The SEMAINE database: annotated multimodal records of emotionally colored conversations between a person and a limited agent. IEEE Trans. Affect. Comput. 3(1), 5–17 (2012)

    Article  Google Scholar 

  16. Biel, J.I., Gatica-Perez, D.: The YouTube lens: crowdsourced personality impressions and audiovisual analysis of vlogs. IEEE Trans. Multimedia 15(1), 41–55 (2013)

    Article  Google Scholar 

  17. Ferwerda, B., Schedl, M., Tkalcic, M.: Using Instagram picture features to predict users’ personality. In: Tian, Q., Sebe, N., Qi, G.-J., Huet, B., Hong, R., Liu, X. (eds.) MMM 2016. LNCS, vol. 9516, pp. 850–861. Springer, Heidelberg (2016). doi:10.1007/978-3-319-27671-7_71

    Chapter  Google Scholar 

  18. Celli, F., Bruni, E., Lepri, B.: Automatic personality and interaction style recognition from Facebook profile pictures. In: Proceedings of the 22nd ACM international conference on Multimedia, pp. 1101–1104. ACM (2014)

    Google Scholar 

  19. Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2879–2886 (2012)

    Google Scholar 

  20. Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: Proceedings of the ACM international conference on Image and video retrieval (CIVR), pp. 401–408 (2007)

    Google Scholar 

  21. Dhall, A., Asthana, A., Goecke, R., Gedeon, T.: Emotion recognition using PHOG and LPQ features. In: Proceedings of the IEEE Conference Automatic Faces and Gesture Recognition workshop FERA, pp. 878–883 (2011)

    Google Scholar 

  22. Ojansivu, V., Heikkilä, J.: Blur insensitive texture classification using local phase quantization. In: Elmoataz, A., Lezoray, O., Nouboud, F., Mammass, D. (eds.) ICISP 2008. LNCS, vol. 5099, pp. 236–243. Springer, Heidelberg (2008). doi:10.1007/978-3-540-69905-7_27

    Chapter  Google Scholar 

  23. Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: British Machine Vision Conference, vol. 1, p. 6 (2015)

    Google Scholar 

  24. Wu, J., Rehg, J.M.: CENTRIST: a visual descriptor for scene categorization. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1489–1501 (2011)

    Article  Google Scholar 

  25. Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. arXiv preprint arXiv:1405.3531 (2014)

  26. Rosipal, R.: Nonlinear partial least squares: an overview. In: Chemoinformatics and Advanced Machine Learning Perspectives: Complex Computational Methods and Collaborative Techniques. ACCM, IGI Global (2011)

    Google Scholar 

  27. Guo, G., Mu, G.: Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 657–664 (2011)

    Google Scholar 

  28. Schwartz, W.R., Kembhavi, A., Harwood, D., Davis, L.S.: Human detection using partial least squares analysis. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 24–31 (2009)

    Google Scholar 

  29. Schwartz, W.R., Guo, H., Davis, L.S.: A robust and scalable approach to face identification. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 476–489. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15567-3_35

    Chapter  Google Scholar 

  30. Dhall, A., Joshi, J., Radwan, I., Goecke, R.: Finding happiest moments in a social context. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part II. LNCS, vol. 7725, pp. 613–626. Springer, Heidelberg (2013). doi:10.1007/978-3-642-37444-9_48

    Chapter  Google Scholar 

  31. Liu, L., Preotiuc-Pietro, D., Samani, Z.R., Moghaddam, M.E., Ungar, L.: Analyzing personality through social media profile picture choice. In: Tenth International AAAI Conference on Web and Social Media (2016)

    Google Scholar 

  32. Park, G., Schwartz, H.A., Eichstaedt, J.C., Kern, M.L., Kosinski, M., Stillwell, D.J., Ungar, L.H., Seligman, M.E.: Automatic personality assessment through social media language. J. Pers. Soc. Psychol. 108(6), 934 (2015)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by AGE-WELL NCE Inc., a member of the Networks of Centres of Excellence program and Alzheimer’s Association grant ETAC-14-321494.

Author information

Authors and Affiliations

  1. David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, Canada

    Abhinav Dhall & Jesse Hoey

Authors
  1. Abhinav Dhall
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jesse Hoey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abhinav Dhall .

Editor information

Editors and Affiliations

  1. Université Pierre et Marie Curie , Paris, France

    Mohamed Chetouani

  2. University of Pittsburgh , Pittsburgh, Pennsylvania, USA

    Jeffrey Cohn

  3. Bogazici University , Bebek, Istanbul, Turkey

    Albert Ali Salah

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Dhall, A., Hoey, J. (2016). First Impressions - Predicting User Personality from Twitter Profile Images. In: Chetouani, M., Cohn, J., Salah, A. (eds) Human Behavior Understanding. HBU 2016. Lecture Notes in Computer Science(), vol 9997. Springer, Cham. https://doi.org/10.1007/978-3-319-46843-3_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46843-3_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46842-6

  • Online ISBN: 978-3-319-46843-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation