Abstract
Knowing people’s personalities is useful in various real-world applications, such as personnel selection. Traditionally, we have to rely on qualitative methodologies, e.g. surveys or psychology tests to determine a person’s traits. However, recent advances in machine learning have it possible to automate this process by inferring personalities from textual data. Despite of its success, text-based method ignores the facial expression and the way people speak, which can also carry important information about human characteristics. In this work, a personality mining framework is proposed to exploit all the information from videos, including visual, auditory, and textual perspectives. Using a state-of-art cascade network built on advanced gradient boosting algorithms, the result produced by our proposed methodology can achieve lower the prediction errors than most current machine learning algorithms. Our multimodal mixture density boosting network especially perform well with small sample size datasets, which is useful for learning problems in psychology fields where big data is often not available.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Alam, F., Riccardi, G.: Predicting personality traits using multimodal information. In: Proceedings of the 2014 ACM Multi Media on Workshop on Computational Personality Recognition, pp. 15–18. ACM (2014)
Biel, J.I., Gatica-Perez, D.: The youtube lens: crowdsourced personality impressions and audiovisual analysis of vlogs. IEEE Trans. Multimed. 15(1), 41–55 (2013)
Bishop, C.M.: Mixture density networks (1994)
Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools Prof. Programmer 25(11), 120–123 (2000)
Buettner, R.: Innovative personality-based digital services. In: PACIS, p. 278 (2016)
Chen, L., Feng, G., Leong, C.W., Lehman, B., Martin-Raugh, M., Kell, H., Lee, C.M., Yoon, S.Y.: Automated scoring of interview videos using Doc2Vec multimodal feature extraction paradigm. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction, pp. 161–168. ACM (2016)
Costa, P.T., McCrae, R.R.: Four ways five factors are basic. Personality Individ. Differ. 13(6), 653–665 (1992)
Farnadi, G., Sitaraman, G., Sushmita, S., Celli, F., Kosinski, M., Stillwell, D., Davalos, S., Moens, M.F., De Cock, M.: Computational personality recognition in social media. User Model. User-Adapted Interact. 26(2–3), 109–142 (2016)
Farnadi, G., Sushmita, S., Sitaraman, G., Ton, N., De Cock, M., Davalos, S.: A multivariate regression approach to personality impression recognition of vloggers. In: Proceedings of the 2014 ACM Multi Media on Workshop on Computational Personality Recognition, pp. 1–6. ACM (2014)
Farnadi, G., Zoghbi, S., Moens, M.F., De Cock, M.: Recognising personality traits using facebook status updates. In: Proceedings of the Workshop on Computational Personality Recognition (WCPR 2013) at the 7th International AAAI Conference on Weblogs and Social Media (ICWSM 2013). AAAI (2013)
Golbeck, J., Robles, C., Turner, K.: Predicting personality with social media. In: CHI 2011 Extended Abstracts on Human Factors in Computing Systems, pp. 253–262. ACM (2011)
Goldberg, L.R.: An alternative description of personality: the big-five factor structure. J. Pers. Soc. Psychol. 59(6), 1216 (1990)
Goldberg, L.R., Johnson, J.A., Eber, H.W., Hogan, R., Ashton, M.C., Cloninger, C.R., Gough, H.G.: The international personality item pool and the future of public-domain personality measures. J. Res. Pers. 40(1), 84–96 (2006)
Haghighat, M., Abdel-Mottaleb, M., Alhalabi, W.: Discriminant correlation analysis: real-time feature level fusion for multimodal biometric recognition. IEEE Trans. Inf. Forensics Secur. 11(9), 1984–1996 (2016)
High, R.: The Era of Cognitive Systems: An Inside Look at IBM Watson and How It Works. IBM Corporation, Redbooks, New York (2012)
Kosinski, M., Matz, S.C., Gosling, S.D., Popov, V., Stillwell, D.: Facebook as a research tool for the social sciences: opportunities, challenges, ethical considerations, and practical guidelines. Am. Psychol. 70(6), 543 (2015)
Mairesse, F., Walker, M.: Personage: personality generation for dialogue. In: Annual Meeting-Association For Computational Linguistics, vol. 45, p. 496 (2007)
Pennebaker, J.W., Francis, M.E., Booth, R.J.: Linguistic Inquiry and Word Count: LIWC 2001, vol. 71. Lawrence Erlbaum Associates, Mahway (2001)
Pennebaker, J.W., King, L.A.: Linguistic styles: language use as an individual difference. J. Pers. Soc. Psychol. 77(6), 1296 (1999)
Pentland, A.: Social dynamics: signals and behavior. In: International Conference on Developmental Learning, vol. 5 (2004)
Ponce-López, V., et al.: ChaLearn LAP 2016: first round challenge on first impressions - dataset and results. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9915, pp. 400–418. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49409-8_32
Sarkar, C., Bhatia, S., Agarwal, A., Li, J.: Feature analysis for computational personality recognition using youtube personality data set. In: Proceedings of the 2014 ACM Multi Media on Workshop on Computational Personality Recognition, pp. 11–14. ACM (2014)
Spearman, C.: The proof and measurement of association between two things. Am. J. Psychol. 100(3/4), 441–471 (1987)
Verhoeven, B., Daelemans, W., et al.: Evaluating content-independent features for personality recognition. In: Proceedings of the 2014 ACM Multi Media on Workshop on Computational Personality Recognition, pp. 7–10. ACM (2014)
Vinciarelli, A., Mohammadi, G.: A survey of personality computing. IEEE Trans. Affect. Comput. 5(3), 273–291 (2014)
Zhou, Z.H., Feng, J.: Deep forest: towards an alternative to deep neural networks. arXiv preprint arXiv:1702.08835 (2017)
Acknowledgment
This work was financially supported by our industry partner at UTS and was technically supported by our faculty with the infrastructures and computing power for empirical works. We would like to send our appreciation to fellow researchers who provided the datasets for this research. We would also like to thanks our colleagues, family and friends who have been supportive during the experimenting and paper writing process.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Vo, N.N.Y., Liu, S., He, X., Xu, G. (2018). Multimodal Mixture Density Boosting Network for Personality Mining. In: Phung, D., Tseng, V., Webb, G., Ho, B., Ganji, M., Rashidi, L. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2018. Lecture Notes in Computer Science(), vol 10937. Springer, Cham. https://doi.org/10.1007/978-3-319-93034-3_51
Download citation
DOI: https://doi.org/10.1007/978-3-319-93034-3_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93033-6
Online ISBN: 978-3-319-93034-3
eBook Packages: Computer ScienceComputer Science (R0)