Skip to main content
SpringerLink
Log in

Investigating the Effect of Linguistic Features on Personality and Job Performance Predictions

  • Conference paper
  • First Online:
Social Computing and Social Media (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14025))

Included in the following conference series:

  • 738 Accesses

Abstract

Personality traits are known to have a high correlation with job performance. On the other hand, there is a strong relationship between language and personality. In this paper, we presented a neural network model for inferring personality and hirability. Our model was trained only from linguistic features but achieved good results by incorporating transfer learning and multi-task learning techniques. The model improved the F1 score 5.6% point on the Hiring Recommendation label compared to previous work. The effect of different Automatic Speech Recognition systems on the performance of the models was also shown and discussed. Lastly, our analysis suggested that the model makes better judgments about hirability scores when the personality traits information is not absent.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.ibm.com/cloud/watson-text-to-speech.

  2. 2.

    https://huggingface.co/roberta-base.

References

  1. Antol, S., et al.: Vqa: Visual question answering. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2425–2433 (2015)

    Google Scholar 

  2. Baevski, A., Zhou, H., Mohamed, A., Auli, M.: wav2vec 2.0: A framework for self-supervised learning of speech representations (2020). https://doi.org/10.48550/ARXIV.2006.11477

  3. Baltrusaitis, T., Ahuja, C., Morency, L.P.: Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41, 423–443 (2019)

    Article  Google Scholar 

  4. Barrick, M.R., Mount, M.K.: The big five personality dimensions and job performance: A meta-analysis. Pers. Psychol. 44(1), 1–26 (1991). https://doi.org/10.1111/j.1744-6570.1991.tb00688.x

    Article  Google Scholar 

  5. Boyd, R.L., Pennebaker, J.W.: Language-based personality: a new approach to personality in a digital world. Current Opinion in Behavioral Sciences 18, 63–68 (2017). https://doi.org/10.1016/j.cobeha.2017.07.017, big data in the behavioural sciences

  6. Brown, T.B., et al.: Language models are few-shot learners (2020). https://doi.org/10.48550/ARXIV.2005.14165

  7. Chen, L., Zhao, R., Leong, C.W., Lehman, B., Feng, G., Hoque, M.E.: Automated video interview judgment on a large-sized corpus collected online. In: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 504–509 (2017). https://doi.org/10.1109/ACII.2017.8273646

  8. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (sep 1995). https://doi.org/10.1023/A:1022627411411

  9. Dai, Y., Jayaratne, M., Jayatilleke, B.: Explainable personality prediction using answers to open-ended interview questions. Front. Psychol. 13 (2022). https://doi.org/10.3389/fpsyg.2022.865841

  10. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding (2018). https://doi.org/10.48550/ARXIV.1810.04805

  11. Finnerty, A.N., Muralidhar, S., Nguyen, L.S., Pianesi, F., Gatica-Perez, D.: Stressful first impressions in job interviews. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction. pp. 325–332. ICMI ’16, Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2993148.2993198

  12. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Teh, Y.W., Titterington, M. (eds.) Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, vol. 9, pp. 249–256. PMLR, Chia Laguna Resort, Sardinia, Italy (13–15 May 2010)

    Google Scholar 

  13. Goodstein, L.D., Lanyon, R.I.: Applications of personality assessment to the workplace: a review. J. Bus. Psychol. 13, 291–322 (1999). https://doi.org/10.1023/A:1022941331649

    Article  Google Scholar 

  14. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1026–1034 (2015). https://doi.org/10.1109/ICCV.2015.123

  15. Hemamou, L., Felhi, G., Vandenbussche, V., Martin, J.C., Clavel, C.: Hirenet: A hierarchical attention model for the automatic analysis of asynchronous video job interviews. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 573–581 (07 2019). https://doi.org/10.1609/aaai.v33i01.3301573

  16. Hsu, C.W., Chang, C.C., Lin, C.J.: A practical guide to support vector classification. Tech. rep., Department of Computer Science, National Taiwan University (2003). http://www.csie.ntu.edu.tw/ cjlin/papers.html

    Google Scholar 

  17. John, O.P., Srivastava, S.: The big five trait taxonomy: History, measurement, and theoretical perspectives (1999)

    Google Scholar 

  18. Junior, J.C.S.J., Lapedriza, A., Palmero, C., Baró, X., Escalera, S.: Person perception biases exposed: Revisiting the first impressions dataset. In: 2021 IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW), pp. 13–21 (2021). https://doi.org/10.1109/WACVW52041.2021.00006

  19. Katada, S., Okada, S.: Biosignal-based user-independent recognition of emotion and personality with importance weighting. Multimedia Tools Appl. 81(21), 30219–30241 (sep 2022). https://doi.org/10.1007/s11042-022-12711-8

  20. Kendall, A., Gal, Y., Cipolla, R.: Multi-task learning using uncertainty to weigh losses for scene geometry and semantics (2017). https://doi.org/10.48550/ARXIV.1705.07115

  21. Kwon, S., Choeh, J.Y., Lee, J.W.: User-personality classification based on the non-verbal cues from spoken conversations. Int. J. Comput. Intell. Syst. 6, 739–749 (05 2013). https://doi.org/10.1080/18756891.2013.804143

  22. Liao, Y., Kodagoda, S., Wang, Y., Shi, L., Liu, Y.: Understand scene categories by objects: A semantic regularized scene classifier using convolutional neural networks. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 2318–2325. IEEE Press (2016). https://doi.org/10.1109/ICRA.2016.7487381

  23. Liu, Y., et al.: Roberta: A robustly optimized bert pretraining approach (2019). https://doi.org/10.48550/ARXIV.1907.11692

  24. Mawalim, C.O., Okada, S., Nakano, Y.I., Unoki, M.: Multimodal bigfive personality trait analysis using communication skill indices and multiple discussion types dataset. In: Meiselwitz, G. (ed.) Social Computing and Social Media. Design, Human Behavior and Analytics - 11th International Conference, SCSM 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, IN: Proceedings, Part I. Lecture Notes in Computer Science, vol. 11578, pp. 370–383. Springer (2019). https://doi.org/10.1007/978-3-030-21902-4_27

  25. Mujtaba, D.F., Mahapatra, N.R.: Multi-task deep neural networks for multimodal personality trait prediction. In: 2021 International Conference on Computational Science and Computational Intelligence (CSCI), pp. 85–91 (2021). https://doi.org/10.1109/CSCI54926.2021.00089

  26. Muralidhar, S., Nguyen, L.S., Frauendorfer, D., Odobez, J.M., Schmid Mast, M., Gatica-Perez, D.: Training on the job: Behavioral analysis of job interviews in hospitality. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction, pp. 84–91. ICMI ’16, Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2993148.2993191

  27. Naim, I., Tanveer, M., Gildea, D., Hoque, E.: Automated prediction and analysis of job interview performance: The role of what you say and how you say it (05 2015). https://doi.org/10.1109/FG.2015.7163127

  28. Nakano, Y.I., Hirose, E., Sakato, T., Okada, S., Martin, J.C.: Detecting change talk in motivational interviewing using verbal and facial information. In: Proceedings of the 2022 International Conference on Multimodal Interaction, pp. 5–14. ICMI ’22, Association for Computing Machinery, New York, NY, USA (2022). https://doi.org/10.1145/3536221.3556607

  29. Nguyen, L., Gatica-Perez, D.: I would hire you in a minute: Thin slices of nonverbal behavior in job interviews, pp. 51–58 (11 2015). https://doi.org/10.1145/2818346.2820760

  30. Nguyen, L., Gatica-Perez, D.: Hirability in the wild: analysis of online conversational video resumes. IEEE Trans. Multimed. 18, 1422–1437 (07 2016). https://doi.org/10.1109/TMM.2016.2557058

  31. Nguyen, L.S., Frauendorfer, D., Mast, M.S., Gatica-Perez, D.: Hire me: computational inference of hirability in employment interviews based on nonverbal behavior. IEEE Trans. Multimedia 16(4), 1018–1031 (2014). https://doi.org/10.1109/TMM.2014.2307169

    Article  Google Scholar 

  32. Okada, S., Komatani, K.: Investigating effectiveness of linguistic features based on speech recognition for storytelling skill assessment. In: Mouhoub, M., Sadaoui, S., Ait Mohamed, O., Ali, M. (eds.) Recent Trends and Future Technology in Applied Intelligence, pp. 148–157. Springer International Publishing, Cham (2018)

    Chapter  Google Scholar 

  33. Okada, S., Nguyen, L., Aran, O., Gatica-Perez, D.: Modeling dyadic and group impressions with intermodal and interperson features. ACM Trans. Multimed. Comput., Commun. Appl. 15, 1–30 (01 2019). https://doi.org/10.1145/3265754

  34. Pennebaker, J., Boyd, R., Jordan, K., Blackburn, K.: The development and psychometric properties of LIWC2015. University of Texas at Austin (2015). https://doi.org/10.15781/T29G6Z

  35. Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I.: Robust speech recognition via large-scale weak supervision (2022). https://doi.org/10.48550/ARXIV.2212.04356

  36. Ramesh, A., et al.: Zero-shot text-to-image generation (2021) arxiv:2102.12092

  37. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models (2021)

    Google Scholar 

  38. Uhrig, J., Cordts, M., Franke, U., Brox, T.: Pixel-level encoding and depth layering for instance-level semantic labeling. In: German Conference on Pattern Recognition (2016)

    Google Scholar 

  39. Vaswani, A., et al: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 6000–6010. NIPS’17, Curran Associates Inc., Red Hook, NY, USA (2017)

    Google Scholar 

  40. Wu, Q., Teney, D., Wang, P., Shen, C., Dick, A., Van Den Hengel, A.: Visual question answering: a survey of methods and datasets. Comput. Vis. Image Underst. 163, 21–40 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

This work was also partially supported by the Japan Society for the Promotion of Science (JSPS) KAKENHI (No. 22K21304, No. 22H04860 and 22H00536), JST AIP Trilateral AI Research, Japan (No. JPMJCR20G6) and JST Moonshot R &D program (JPMJMS2237-3). Hung is supported by the Japanese Government (MEXT) Scholarship.

Author information

Authors and Affiliations

  1. Japan Advanced Institute of Science and Technology (JAIST), Nomi, Japan

    Hung Le, Sixia Li, Candy Olivia Mawalim & Shogo Okada

  2. The University of Fukuchiyama, Fukuchiyama, Japan

    Hung-Hsuan Huang

  3. Educational Testing Service, Princeton, USA

    Chee Wee Leong

Authors
  1. Hung Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sixia Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Candy Olivia Mawalim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hung-Hsuan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chee Wee Leong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shogo Okada
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hung Le .

Editor information

Editors and Affiliations

  1. University of Bucharest, Bucharest, Romania

    Adela Coman

  2. University of Tsukuba, Tsukuba, Japan

    Simona Vasilache

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Le, H., Li, S., Mawalim, C.O., Huang, HH., Leong, C.W., Okada, S. (2023). Investigating the Effect of Linguistic Features on Personality and Job Performance Predictions. In: Coman, A., Vasilache, S. (eds) Social Computing and Social Media. HCII 2023. Lecture Notes in Computer Science, vol 14025. Springer, Cham. https://doi.org/10.1007/978-3-031-35915-6_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35915-6_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35914-9

  • Online ISBN: 978-3-031-35915-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation