Skip to main content
SpringerLink
Log in

An ensemble of deep learning algorithms for popularity prediction of flickr images

  • 1167: Data Science on Multimedia Data: Challenges and Applications
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Content popularity prediction is a technique that helps service providers to apply user behavior analysis to provide better services or products. Popularity prediction on social networks is mainly performed using different contents, such as texts, images, voices, or videos. Recent studies on popularity prediction on images show significant results and provide several effective factors for popular content. These studies mainly use feature-based, time-series-based, or deep-learning based approaches to handle popularity prediction tasks. However, in complex data containing diverse subjects, popularity prediction is a challenging task. The first challenge in this task is to use images effectively along with their textual contents, like user data, text content, and time data. Since, the nature of data on social networks originally is noisy, this further increases the complexity of data and degrades the performance of the prediction model. To address these difficulties, we focus on image content along with textual features to design a hybrid deep-based model. Furthermore, to deal with the noise, several data adaptation and normalization approaches are proposed to generate the proper format of data as input of the proposed hybrid model. The proposed hybrid model consists of three sub-models to extract features simultaneously from image content and the normalized user-time data. In the final step, an ensemble of hybrid networks is proposed based on the constructed component classifiers. For popularity prediction, data from Flickr are employed in these experiments. The experimental results of this study on the Flickr dataset show that the proposed method achieves 87.55% classification accuracy and results in 9.04% improvement. Furthermore, the proposed hybrid model significantly outperforms the baseline methods in this study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://github.com/maxpumperla/hyperas

References

  1. Adele Lu Jia (2018) Predicting the implicit and the explicit video popularity in a User Generated Content site with enhanced social features. Comput Netw 140:112–125

    Article  Google Scholar 

  2. Aditya Khosla, Atish Das Sarma, Raffay Hamid (2014) What Makes an Image Popular?," in International World Wide Web Conference Committee, Seoul, Korea

  3. Aiello LM et al (2013) “Sensing trending topics in Twitter,” in IEEE Trans. Multimedia 15:1268–1282

    Google Scholar 

  4. Albelwi S, Mahmood A (2017) A framework for designing the architectures of deep convolutional neural networks. Entropy 19(6):242

    Article  Google Scholar 

  5. Aloufi S, Zhu S, El Saddik A (2017) On the prediction of flickr image popularity by analyzing heterogeneous social sensory data. MDPI, sensors 17:3–631

    Article  Google Scholar 

  6. R. Bandari, S. Asur, B.A. Huberman (2012) The pulse of news in social media: fore- casting popularity in Proceedings of the Sixth International AAAI Conference on Weblogs and Social Media

  7. P. Bao, H.W. Shen, X. Jin, X.Q. Cheng (2015) Modeling and predicting popularity dynamics of microblogs using self-excited Hawkes processes," in Proceedings of International Conference on World Wide Web,

  8. Sebastian Bruch Xuanhui Wang Mike Bendersky Marc Najork (2019) An Analysis of the Soft-max Cross Entropy Loss for Learning-to-Rank with Binary Relevance," in Proceedings of the 2019 ACM SIGIR International Conference on the Theory of Information Retrieval (ICTIR 2019)

  9. Q. Cao, H. Shen, K. Cen, W. Ouyang, X. Cheng, "DeepHawkes: bridging the gap between prediction and understanding of information cascades," in in: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, ACM, 2017.

  10. Carta S, Podda AS, Recupero DR, Saia R, Usai G (2020) Popularity Prediction of Instagram Posts. Special Issue: Emerging Trends and Challenges in Supervised Learning Tasks 11(9):453

    Google Scholar 

  11. Jia-Ren Chang, Yong-Sheng Chen (2015) Batch-normalized maxout network in network," in arXiv preprint

  12. Chen G, Kong Q, Nan Xu, Mao W (2019) NPP: A neural popularity prediction model for social media content. Neurocomputing Elsevier 333:221–230

    Article  Google Scholar 

  13. Chittaragi NB, Prakash A, Koolagudi SG (2018) Dialect identification using spectral and prosodic features on single and ensemble classifiers. Arabian Journal for Science and Engineering- Special Issue: Computer Engineering and Computer Science 43:4289–4302

    Article  Google Scholar 

  14. Chollet F, "https://github.com/fchollet/keras," 2015.

  15. Shaunak De, Abhishek Maity, Vritti Goel, Sanjay Shitole, Avik Bhattacharya, "Predicting the popularity of Instagram posts for a lifestyle magazine using deep learning," in International Conference on Circuits, Systems, Communication and Information Technology Applications (CSCITA), 2017.

  16. Deng S, Huang L, Guandong Xu, Xindong Wu, Zhaohui Wu (2016) On deep learning for trust-aware recommendations in social networks. IEEE Transactions on Neural Networks and Learning Systems 28(5):1164–1177

    Article  Google Scholar 

  17. J. Donahue, et al. (2015) Long-term recurrent convolutional networks for visual recognition and description. IEEE Conference on Computer Vision and Pattern Recognition

  18. G. Gürsun, M. Crovella, I. Matta, "Describing and forecasting video access patterns," in Proceedings of IEEE INFOCOM, 2011.

  19. Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun (2016) Deep Residual Learning for Image Recognition. CVPR

  20. He KM, Gkioxari G, Dollár P et al, (2017) "Mask R-CNN," in Proceedings of IEEE International Conference on Computer Vision

  21. Hoo-Chang S, Roth HR, GaoM, (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298

    Article  Google Scholar 

  22. Hu Y, Hu C, Fu S, Shi P, Ning B (2016) Predicting the popularity of viral topics based on time series forecasting. Neurocomputing 210:55–65

    Article  Google Scholar 

  23. Huan E-Y, Wen G-H (2020) Transfer learning with deep convolutional neural network for constitution classification with face image. Multimedia Tools and Applications 79:11905–11919

    Article  Google Scholar 

  24. In https://github.com/social-media-prediction/TPIC2017, 2017.

  25. Kong S, Ye F, Feng L (2014) Predicting future retweet counts in a microblog. J Comput Inf Syst 10(4):1393–1404

    Google Scholar 

  26. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Advances in neural information processing systems, 2012.

  27. C. Li, J. Ma, X. Guo, Q. Mei, "DeepCas: an end-to-end predictor of information cascades," in Proceedings of the 26th International Conference on World Wide Web, International World Wide Web Conferences Steering Committee, 2017.

  28. M. Lucic, K. Kurach, M. Michalski, S. Gelly, and, "Are GANs Created Equal?" in ArXiv e-prints, 2017.

  29. Ma JJ, Wen GH, Hu Yet al. (2018) Tongue image constitution recognition based on complexity perception," in arXiv preprint

  30. Xiaoxin Mao, Xi Zhao, Jun Lin, Enrique Herrera-Viedma, "Utilizing multi-source data in popularity prediction for shop-type," Elsevier: Knowledge-Based Systems, vol. 165, pp. 253–267, 2019.

  31. Giovanni Mariani, Florian Scheidegger, Roxana Istrate, Costas Bekas, Cristiano Malossi, "BAGAN: Data Augmentation with Balancing GAN," Computer Vision and Pattern Recognition, 2018.

  32. S. Mishra, M.-A. Rizoiu, L. Xie, "Feature driven and point process approaches for popularity prediction," in Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, ACM, 2016.

  33. H. Pinto, J.M. Almeida, M.A. Gonçalves, "Using early view patterns to predict the popularity of youtube videos," in Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, ACM, 2013.

  34. Siddharth Pal, Yuxiao Dong, Bishal Thapa, Nitesh V. Chawla, Ananthram Swami, Ram Ramanathan, "Deep Learning for Network Analysis: Problems, Approaches and Challenges," in MILCOM 2016 - 2016 IEEE Military Communications Conference, Baltimore, MD, USA, 2016.

  35. R. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng, (2007) Self-taught learning: transfer learning from unlabeled data, Proceedings of the 24th international conference on Machine learning,

  36. M.A. Rizoiu , L. Xie , S. Sanner , M. Cebrian , H. Yu , P.V. Hentenryck, (2017) Expecting to be HIP: hawkes intensity processes for social media popularity," in Proceedings of International Conference on World Wide Web

  37. Rousidis D, Koukaras P, Tjortjis C (2020) Social media prediction: a literature review. Springer, Multimedia Tools and Applications

    Google Scholar 

  38. Sen PC, Hajra M, Ghosh M (2020) Supervised Classification Algorithms in Machine Learning: A Survey and Review. Emerging Technology in Modelling and Graphics Springer Singapore Singapore, In book, pp 99–111

    Google Scholar 

  39. Husanbir Singh Pannu, Sahil Ahuja, Nitin Dang, Sahil Soni, Avleen Kaur Malhi (2020) Deep learning based image classification for intestinal hemorrhage. Multimedia Tools and Applications

  40. B. Samanta , A. De , A. Chakraborty , N. Ganguly, (2017) A large margin point process combining reinforcement and competition for modeling hashtag popularity. Proceedings of the Twenty Sixth International Joint Conference on Artificial Intelligence

  41. ShakirulIslam M, AhmedFoysal F, NafisNeehal, EnamulKarim, AkhterHossain S  (2019) InceptB: A CNN based classification approach for recognizing traditional bengali games. Elsevier 143:595–602

  42. H. Shen, D. Wang, C. Song, "Modeling and predicting popularity dynamics via reinforced Poisson processes," in Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, 2014.

  43. Smith LN (2018) A Disciplined Approach to Neural Network Hyper-Parameters: Part 1 – Learning Rate. Batch Size, Momentum, and Weight Decay 8:1803

    Google Scholar 

  44. I. Sutskever, O. Vinyals, Q.V. Le, (2014) Sequence to sequence learning with neural networks," in Proceedings of Advances in Neural Information Processing Systems

  45. Szabo G, Huberman BA (2010) Predicting the popularity of online content. Commun ACM 53:80–88

    Article  Google Scholar 

  46. Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2016) Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. Computer Vision and Pattern Recognitionar/Xiv 2:1602

    Google Scholar 

  47. TensorFlow , "https://www.tensorflow.org," 2018.

  48. Tomasz Trzci´nski, Przemysław Rokita, (2017) Predicting popularity of online videos using Support Vector Regression. IEEE Trans Multimedia 19(11):2561–2570

    Article  Google Scholar 

  49. M. Tsagkias, W. Weerkamp, M. De Rijke, "Predicting the volume of comments on online news stories," in Proceedings of the 18th ACM Conference on Information and Knowledge Management, ACM, 2009.

  50. T. Trzcinski, P. l. Andruszkiewicz, T. Bochenski, and P. law Rokita, "Recurrent Neural Networks for Online Video Popularity Prediction," in arXiv: 1707.06807, 2017.

  51. O. Tsur, A. Rappoport, "What’s in a hashtag? Content based prediction of the spread of ideas in microblogging communities," in Proceedings of the Fifth ACM International Conference on Web Search and Data Mining, ACM, pp. 643–652, 2012.

  52. Wang Y, Li H, Jia P, Zhang G, Wang T, Hao X (2019) Multi-scale densenets-based aircraft detection from remote sensing images. Sensors (Basel) 19(23):5270

    Article  Google Scholar 

  53. David Warde-Farley, Ian J. Goodfellow, Aaron Courville, Yoshua Bengio (2013) An empirical analysis of dropout in piecewise linear networks, in arXiv preprint .

  54. L. Weng, F. Menczer, Y.-Y. Ahn (2014) Predicting successful memes using network and community structure," in Proceedings of International Conference on Web and Social Media

  55. Bo Wu, Wen-Huang Cheng, Yongdong Zhang, Tao Mei, "Time Matters: Multi-scale Temporalization of Social Media Popularity," in Proceedings of the 24th ACM international conference on Multimedia, 2016.

  56. Bo Wu, Tao Mei, Wen-Huang Cheng, Yongdong Zhang, "Unfolding Temporal Dynamics: Predicting Social Media Popularity Using Multi-Scale Temporal Decomposition," in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16), 2016.

  57. Bo Wu, Wen-Huang Cheng, Yongdong Zhang, Qiushi Huang, Jintao Li, Tao Mei (2017) Sequential prediction of social media popularity with deep temporal context networks," in Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17)

  58. Bo Wu, Wen-Huang Cheng, Yongdong Zhang, Qiushi Huang, Jintao Li, Tao Me, "Sequential Prediction of Social Media Popularity with Deep Temporal Context Networks," in Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17), 2017.

  59. Jaewon Yang, Jure Leskovec (2011) Patterns of temporal variation in online. In Proc. of WSDM

  60. F. Yu, Q. Liu, S. Wu, L. Wang, T. Tan, "A dynamic recurrent model for next basket recommendation," In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, ACM, 2016.

  61. Q. Zhao, M.A. Erdogdu, H.Y. He, A. Rajaraman, J. Leskovec (2015) "a self-ex- citing point process model for predicting tweet popularity," in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM

  62. Alireza Zohourian, Hedieh Sajedi, Arefeh Yavary, "Popularity Prediction of Images and Videos on Instagram," in 4th International Conference on Web Research (ICWR), 2018.

Download references

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering Department, University of Tabriz, 29 Bahman St, Tabriz, Iran

    Shadi Alijani, Jafar Tanha &  Leyli Mohammadkhanli

Authors
  1. Shadi Alijani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jafar Tanha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leyli Mohammadkhanli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jafar Tanha.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alijani, S., Tanha, J. & Mohammadkhanli, . An ensemble of deep learning algorithms for popularity prediction of flickr images. Multimed Tools Appl 81, 3253–3274 (2022). https://doi.org/10.1007/s11042-021-11517-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11517-4

Keywords

Search

Navigation