Skip to main content
SpringerLink
Log in

SMPC: boosting social media popularity prediction with caption

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Social media popularity prediction refers to using multi-modal content to predict the popularity of a post offered by an internet user. It is an effective way to explore advanced forecasting trends and make more popularity-sensitive strategic decisions for the future. Existing methods attempt to explore various multi-model features to solve this task, which only focus on local information, lacking global understanding for the post’s content. In this paper, we propose social media popularity prediction with caption (SMPC), a novel architecture that integrates the caption as the global representation into the existing multi-model-feature-based popularity prediction method. To make good use of the generated captions, we process them in word-level, sentence-level and length-level ways, obtaining three kinds of caption features. To incorporate caption features, we exploit seven variants of the architecture by concatenating features in all the possible manners, for the feature fusion and training different combinations for the CatBoost regression. Extensive experiments are conducted on Social Media Prediction Dataset (SMPD) and show that the proposed approaches can achieve competing results against state-of-the-art models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Szabo, G., Huberman, B.A.: Predicting the popularity of online content. Commun. ACM 53(8), 80–88 (2010)

    Article  Google Scholar 

  2. Kang, P., Lin, Z., Teng, S., Zhang, G., Guo, L., Zhang, W.: Catboost-based framework with additional user information for social media popularity prediction. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2677–2681 (2019)

  3. He, Z., He, Z., Wu, J., Yang, Z.: Feature construction for posts and users combined with lightgbm for social media popularity prediction. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2672–2676 (2019)

  4. Chen, J., Liang, D., Zhu, Z., Zhou, X., Ye, Z., Mo, X.: Social media popularity prediction based on visual-textual features with xgboost. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2692–2696 (2019)

  5. Wu, B., Mei, T., Cheng, W.-H., Zhang, Y.: Unfolding temporal dynamics: predicting social media popularity using multi-scale temporal decomposition. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)

  6. Li, L., Situ, R., Gao, J., Yang, Z., Liu, W.: A hybrid model combining convolutional neural network with xgboost for predicting social media popularity. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1912–1917 (2017)

  7. Bielski, A., Trzcinski, T.: Pay attention to virality: understanding popularity of social media videos with the attention mechanism. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2335–2337 (2018)

  8. Jing, P., Su, Y., Nie, L., Bai, X., Liu, J., Wang, M.: Low-rank multi-view embedding learning for micro-video popularity prediction. IEEE Trans. Knowl. Data Eng. 30(8), 1519–1532 (2017)

    Article  Google Scholar 

  9. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  10. Yang, J., Leskovec, J.: Patterns of temporal variation in online media. In: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, pp. 177–186 (2011)

  11. Matsubara, Y., Sakurai, Y., Prakash, B.A., Li, L., Faloutsos, C.: Rise and fall patterns of information diffusion: model and implications. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 6–14 (2012)

  12. Figueiredo, F., Almeida, J.M., Gonçalves, M.A., Benevenuto, F.: Trendlearner: early prediction of popularity trends of user generated content. Inf. Sci. 349, 172–187 (2016)

    Article  Google Scholar 

  13. Hu, Y., Hu, C., Fu, S., Shi, P., Ning, B.: Predicting the popularity of viral topics based on time series forecasting. Neurocomputing 210, 55–65 (2016)

    Article  Google Scholar 

  14. Shen, H., Wang, D., Song, C., Barabási, A.-L.: Modeling and predicting popularity dynamics via reinforced Poisson processes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28 (2014)

  15. Gao, S., Ma, J., Chen, Z.: Modeling and predicting retweeting dynamics on microblogging platforms. In: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, pp. 107–116 (2015)

  16. Bao, P., Shen, H.-W., Jin, X., Cheng, X.-Q.: Modeling and predicting popularity dynamics of microblogs using self-excited Hawkes processes. In: Proceedings of the 24th International Conference on World Wide Web, pp. 9–10 (2015)

  17. Daley, D.J., Kendall, D.G.: Stochastic rumours. IMA J. Appl. Math. 1(1), 42–55 (1965)

    Article  MathSciNet  Google Scholar 

  18. Maki, D.P., Maki, D.P., Mali, D., Thompson, M., Thompson, M.: Mathematical Models and Applications: with Emphasis on the Social, Life, and Management Sciences. Prentice Hall (1973)

  19. Xiong, F., Liu, Y., Zhang, Z.-J., Zhu, J., Zhang, Y.: An information diffusion model based on retweeting mechanism for online social media. Phys. Lett. A 376(30–31), 2103–2108 (2012)

    Article  Google Scholar 

  20. Wang, H., Li, Y., Feng, Z., Feng, L.: Retweeting analysis and prediction in microblogs: an epidemic inspired approach. China Commun. 10(3), 13–24 (2013)

    Article  Google Scholar 

  21. Yang, J., Leskovec, J.: Modeling information diffusion in implicit networks. In: 2010 IEEE International Conference on Data Mining, pp. 599–608. IEEE (2010)

  22. Vilares, D., Alonso, M.A., Gómez-Rodríguez, C.: On the usefulness of lexical and syntactic processing in polarity classification of twitter messages. J. Am. Soc. Inf. Sci. 66(9), 1799–1816 (2015)

    Google Scholar 

  23. Wu, B., Cheng, W.-H., Zhang, Y., Mei, T.: Time matters: Multi-scale temporalization of social media popularity. In: Proceedings of the 24th ACM International Conference on Multimedia, pp. 1336–1344 (2016)

  24. Wu, B., Cheng, W.-H., Zhang, Y., Huang, Q., Li, J., Mei, T.: Sequential prediction of social media popularity with deep temporal context networks. arXiv preprint arXiv:1712.04443 (2017)

  25. Zhang, W., Wang, W., Wang, J., Zha, H.: User-guided hierarchical attention network for multi-modal social image popularity prediction. In: Proceedings of the 2018 World Wide Web Conference, pp. 1277–1286 (2018)

  26. Zhang, D., Yao, L., Chen, K., Wang, S., Chang, X., Liu, Y.: Making sense of spatio-temporal preserving representations for EEG-based human intention recognition. IEEE Trans. Cybern. 50(7), 3033–3044 (2019)

    Article  Google Scholar 

  27. Luo, M., Chang, X., Nie, L., Yang, Y., Hauptmann, A.G., Zheng, Q.: An adaptive semisupervised feature analysis for video semantic recognition. IEEE Trans. Cybern. 48(2), 648–660 (2017)

    Article  Google Scholar 

  28. Chen, K., Yao, L., Zhang, D., Wang, X., Chang, X., Nie, F.: A semisupervised recurrent convolutional attention model for human activity recognition. IEEE Trans. Neural Netw. Learn. Syst. 31(5), 1747–1756 (2019)

    Article  Google Scholar 

  29. Liu, Z., Wu, S., Jin, S., Liu, Q., Ji, S., Lu, S., Cheng, L.: Investigating pose representations and motion contexts modeling for 3D motion prediction. IEEE Trans. Pattern Anal. Mach. Intell. 45(1), 681–97 (2022)

    Article  Google Scholar 

  30. Liu, A.-A., Zhou, H., Nie, W., Liu, Z., Liu, W., Xie, H., Mao, Z., Li, X., Song, D.: Hierarchical multi-view context modelling for 3D object classification and retrieval. Inf. Sci. 547, 984–995 (2021)

    Article  Google Scholar 

  31. Rennie, S.J., Marcheret, E., Mroueh, Y., Ross, J., Goel, V.: Self-critical sequence training for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7008–7024 (2017)

  32. Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156–3164 (2015)

  33. Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3128–3137 (2015)

  34. Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. Adv. Neural Inf. Process. Syst. 31 (2018)

  35. Ferov, M., Modrỳ, M.: Enhancing lambdamart using oblivious trees. arXiv preprint arXiv:1609.05610 (2016)

  36. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst. 26 (2013)

  37. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)

  38. Wu, B., Cheng, W.-H., Liu, P., Liu, B., Zeng, Z., Luo, J.: Smp challenge: an overview of social media prediction challenge 2019. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2667–2671 (2019)

  39. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer (2014)

  40. Paddlepaddle: An easy-to-use, easy-to-learn deep learning platform. http://www.paddlepaddle.org/

  41. Ding, K., Wang, R., Wang, S.: Social media popularity prediction: a multiple feature fusion approach with deep neural networks. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2682–2686 (2019)

  42. Xu, K., Lin, Z., Zhao, J., Shi, P., Deng, W., Wang, H.: Multimodal deep learning for social media popularity prediction with attention mechanism. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4580–4584 (2020)

  43. Wang, K., Wang, P., Chen, X., Huang, Q., Mao, Z., Zhang, Y.: A feature generalization framework for social media popularity prediction. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4570–4574 (2020)

Download references

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China (2021YFF0901600), the National Natural Science Foundation of China (U21B2024, 62002257), the China Postdoctoral Science Foundation (2021M692395) and the Baidu Program. Besides, we sincerely thank to the Baidu Program for the Paddlepaddle platform.

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, China

    An-An Liu, Xiaowen Wang, Ning Xu, Jing Liu & Yuting Su

  2. Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, 230088, China

    An-An Liu

  3. Peking University, Beijing, 100871, China

    Quan Zhang

  4. New Media Center, People’s Daily, Beijing, 100000, China

    Shenyuan Zhang

  5. Kuaishou Technology, Beijing, 100000, China

    Yejun Tang

  6. State Key Laboratory of Communication Content Cognition, People’s Daily Online, Beijing, 100733, China

    Junbo Guo & Guoqing Jin

  7. Baidu Inc., Beijing, 100085, China

    Xuanya Li

Authors
  1. An-An Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaowen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ning Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuting Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Quan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shenyuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yejun Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Junbo Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Guoqing Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Xuanya Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ning Xu or Xuanya Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, AA., Wang, X., Xu, N. et al. SMPC: boosting social media popularity prediction with caption. Multimedia Systems 29, 577–586 (2023). https://doi.org/10.1007/s00530-022-01030-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-01030-5

Keywords

Search

Navigation