Abstract
Estimating the degree of multiple personality traits in a single image is challenging due to the presence of multiple people, occlusion, poor quality etc. Unlike existing methods which focus on the classification of a single personality using images, this work focuses on estimating different personality traits using a single image. We believe that when the image contains multiple persons and modalities, one can expect multiple emotions and expressions. This work separates given input images into different faces of people, recognized text, meta-text and background information using face segmentation, text recognition and scene detection techniques. Contrastive learning is explored to extract features from each segmented region based on clustering. The proposed work fuses textual and visual features extracted from the image for estimating the degree of multiple personality traits. Experimental results on our benchmark datasets show that the proposed model is effective and outperforms the existing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ventura, C., Masip, D., Lapedriza, A.: Interpreting CNN models for apparent personality trait regression. In: Proceedings CVPRW, pp. 55–63 (2017)
Sun, X., Huang, J., Zheng, S., Rao, X., Wang, M.: Personality assessment based on multimodal attention network learning with category-based mean square error. IEEE Trans. Image Process. 31, 2162–2174 (2022)
Alamsyah, D., Widhiarsho, W., Hasan, S., et al.: Handwriting analysis for personality trait features identification using CNN. In: Proceedings ICoDSA, pp. 232–238 (2022)
Biswas, K., Shivakumara, P., Pal, U., Chakraborti, T., Lu, T., Ayub, M.N.B.: Fuzzy and genetic algorithm-based approach for classification of personality traits oriented social media images. Knowl.-Based Syst. 241, 108024 (2022)
Wu, H., et al.: CVT: introducing convolutions to vision transformers. In: Proceedings ICCV, pp. 22–31 (2021)
Google Cloud Vision AI. https://cloud.google.com/vision. Accessed 25 June 2021
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: Proceedings ICML, pp. 1597–1607 (2020)
Kumar, P.K.N., Gavriova, M.L.: Latent personality traits assessment from social network activity using contextual language embedding. IEEE Trans. Comput. Soc. Syst. 9(2), 38–649 (2021)
Anglekar, S., Chaudhari, U., Chitanvis, A., Shankarmani, R.: A deep learning based self-assessment tool for personality traits and interview preparations. In: Proceedings ICCICT (2021)
Dickmond, L., Hameed, V.A., Rana, M.E.: A study of machine learning based approaches to extract personality information from curriculum vitae. In: Proceedings DeSE (2021)
Kulsoom, S., Latif, S., Saba, T., Latif, R.: Students’ personality assessment using deep learning from university admission statement of purpose. In: Proceedings CDMA (2022)
Gahmousse, A., Gattal, A., Djeddi, C., Siddiqi, I.: Handwriting based personality identification using textural features. In: Proceedings ICDABI (2020)
Biswas, K., Shivakumara, P., Pal, U., Lu, T., Blumenstein, M., Lladós, J.: Classification of aesthetic natural scene images using statistical and semantic features. Multimedia Tools Appl., 1–26 (2022)
Biswas, K., Shivakumara, P., Pal, U., Lu, T.: A new ontology-based multimodal classification system for social media images of personality traits. SIViP 17(2), 543–551 (2023)
Beyan, C., Zunino, A., Shahid, M., Murino, V.: Personality traits classification using deep visual activity-based nonverbal features of key-dynamic images. IEEE Trans. Affect. Comput. 12(4), 1084–1099 (2019)
Xu, J., Tian, W., Lv, G., Liu, S., Fan, Y.: Prediction of the big five personality traits using static facial images of college students with different academic backgrounds. IEEE Access 9, 76822–76832 (2021)
Yu, J., Kai, C., Rui, X.: Hierarchical interactive multimodal transformer for aspect-based multimodal sentiment analysis. IEEE Trans. Affect. Comput. (2022). https://doi.org/10.1109/TAFFC.2022.3171091
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings CVPR (2021)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings CVPR, pp. 770–778 (2016)
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2017)
Radford, A., et al.: Learning transferable visual models from natural language supervision. In: Proceedings ICML, pp. 8748–8763 (2021)
Agarap, A.F.: Deep learning using rectified linear units (ReLU) (2019). arXiv preprint arXiv:1803.08375
Sammut, C., Webb, G.I.: Mean squared error. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning, p. 653. Springer, Boston (2011). https://doi.org/10.1007/978-0-387-30164-8_528
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014). arXiv preprint arXiv:1412.6980
Guntuku, S.C., Lin, W., Carpenter, J., Ng, W.K., Ungar, L.H., Preoţiuc-Pietro, D.: Studying personality through the content of posted and liked images on twitter. In: Proceedings ACM on Web Science Conference, pp. 223–227 (2017)
Costa, P.T. Jr.: Revised NEO personality inventory and neo five-factor inventory. Prof. Manual (1992)
Zhu, H., Li, L., Zhao, S., Jiang, H.: Evaluating attributed personality traits from scene perception probability. Pattern Recogn. Lett. 116, 121–126 (2018)
Acknowledgement
The work was supported by Ministry of Higher Education Malaysia via Fundamental Research Grant Scheme with Grant no: FRGS/1/2020/ICT02/UM/02/4. And also, this work was partially supported by Technology Innovation Hub (TIH), Indian Statistical Institute, Kolkata, India.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Biswas, K., Shivakumara, P., Pal, U., Sarkar, R. (2023). A New Contrastive Learning Based Model for Estimating Degree of Multiple Personality Traits Using Social Media Posts. In: Lu, H., Blumenstein, M., Cho, SB., Liu, CL., Yagi, Y., Kamiya, T. (eds) Pattern Recognition. ACPR 2023. Lecture Notes in Computer Science, vol 14407. Springer, Cham. https://doi.org/10.1007/978-3-031-47637-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-47637-2_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47636-5
Online ISBN: 978-3-031-47637-2
eBook Packages: Computer ScienceComputer Science (R0)