Skip to main content
SpringerLink
Log in

SmartEyes: Social Multimedia Analysis Platform for Open Data Providers

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

It is essential to collect and analyze information from various data sources to develop smart cities. A huge amount of social media data are generated daily from various social activities in different formats, such as text, images, audio, or video clips. Thus, we aim to develop SmartEyes, a flexible platform for collecting, integrating, and analyzing social multimedia for open data providers for smart cities and environments. Our platform SmartEyes consists of useful components, which can be easily extended integrated to quickly develop different social media analysis services to listen and analyze data from different social media sources with the diversification of data types. We emphasize the ability to flexibly integrate artificial intelligence applications into the system to analyze social events effectively and serve smart cities in creating open data providers. We also introduce four case study applications based on our platform, including a face recognition system for celebrity recognition in news videos, an object detection system for brand logo recognition, a video highlighting system for summarizing football matches, and a text analysis system serving for keyword occurrences and emotional text analysis for admissions of universities. In these applications, we have collected and analyzed more than 5000 videos from various Youtube channels, and thousands of posts from admission pages of universities on Facebook. Each application demonstrates a unique meaning to each specific situation for open data providers in smart cities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Adarsh P, Rathi P, Kumar M. Yolo v3-tiny: object detection and recognition using one stage improved model. In: 2020 6th international conference on advanced computing and communication systems (ICACCS). 2020. p. 687–94.

  2. Ahmed S, Mahmood Q. An authentication based scheme for applications using JSON web token. In: 2019 22nd international multitopic conference (INMIC). 2019. p. 1–6.

  3. Bisong E. Google colaboratory. Berkeley: Apress; 2019. p. 59–64. https://doi.org/10.1007/978-1-4842-4470-8_7.

    Book  Google Scholar 

  4. Bogdan Batrinca PCT. Social media analytics: a survey of techniques, tools and platforms. AI Soc. 2015;323(6088):89–116.

    Article  Google Scholar 

  5. Boyko N, Basystiuk O, Shakhovska N. Performance evaluation and comparison of software for face recognition, based on dlib and opencv library. In: 2018 IEEE second international conference on data stream mining processing (DSMP). 2018. p. 478–82.

  6. Chen X, Zhang Q, Han J, Han X, Liu Y, Fang Y. Object detection of optical remote sensing image based on improved faster RCNN. In: 2019 IEEE 5th international conference on computer and communications (ICCC). 2019. p. 1787–91.

  7. Chi X, Liu B, Niu Q, Wu Q. Web load balance and cache optimization design based nginx under high-concurrency environment. In: 2012 third international conference on digital manufacturing automation. 2012. p. 1029–32.

  8. Clayton H, Gilbert E. Vader: a parsimonious rule-based model for sentiment analysis of social media text. Behav Res. 2014;323(6088):1–8.

    Google Scholar 

  9. Dong Yu LD. Automatic speech recognition. In: Automatic speech recognition. A deep learning approach. New York: Springer; 2015. p. 1–7.

  10. Elizabeth MM, Chareen S, Elison-Bowers P. Image and video disclosure of substance use on social media websites. Comput Hum Behav. 2010;26(6088):1405–11.

    Google Scholar 

  11. free A. open resource with Machine Learning papers, c., evaluation tables: Paper With Code. 2020. https://paperswithcode.com/.

  12. Geitgey A. Modern face recognition with deep learning. 2018. https://github.com/ageitgey/face_recognition.

  13. Hearst MA. Support vector machines. IEEE Intell Syst. 1998;13(4):18–28. https://doi.org/10.1109/5254.708428.

    Article  Google Scholar 

  14. Kim HM, Sabri S, Kentc A. Smart cities for technological and social innovation case studies, current trends, and future steps. Amsterdam: Elsevier; 2021.

    Google Scholar 

  15. Le TC, Nguyen QV, Tran MT. Flexible platform for integration, collection, and analysis of social media for open data providers in smart cities. In: Dang TK, Küng J, Takizawa M, Chung TM, editors. Future data and security engineering. Cham: Springer International Publishing; 2020. p. 304–24.

    Chapter  Google Scholar 

  16. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL. Microsoft coco: common objects in context. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T, editors. Computer Vision—ECCV 2014. Cham: Springer International Publishing; 2014. p. 740–55.

    Chapter  Google Scholar 

  17. Liu H. Highlight extraction in soccer videos by using multimodal analysis. In: 2017 13th international conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD). 2017. p. 2169–73.

  18. Min C, Shiwen M, Yunhao L. Big data: a survey. Mobile Netw Appl. 2014;323(6088):171–209.

    Google Scholar 

  19. Nguyen DQ, Nguyen AT. Phobert: pre-trained language models for Vietnamese. 2020.

  20. Ren H, Li Z. Object detection using edge histogram of oriented gradient. In: 2014 IEEE international conference on image processing (ICIP). 2014. p. 4057–61.

  21. Scott AC, Kristopher K, Danielle SM. Sentiment analysis and social cognition engine (seance): an automatic tool for sentiment, social cognition, and social-order analysis. Behav Res. 2016;323(6088):804–6.

    Google Scholar 

  22. Sharma S, Shanmugasundaram K, Ramasamy SK. Farec–cnn based efficient face recognition technique using dlib. In: 2016 international conference on advanced communication control and computing technologies (ICACCCT). 2016. p. 192–95.

  23. Shunji M, Nishida H, Yamada H. Optical character recognition. New York: Wiley; 1999. p. 1–7.

    Google Scholar 

  24. Vaughn B. Virtualized list. 2020. https://bvaughn.github.io/react-virtualized.

  25. Wild Home LF. A public benchmark for face verification. 2020. http://vis-www.cs.umass.edu/lfw.

  26. William I, Ignatius Moses Setiadi DR, Rachmawanto EH, Santoso HA, Sari CA. Face recognition using facenet (survey, performance test, and comparison). In: 2019 fourth international conference on informatics and computing (ICIC). 2019. p. 1–6.

  27. Womg A, Shafiee MJ, Li F, Chwyl B. Tiny ssd: a tiny single-shot detection deep convolutional neural network for real-time embedded object detection. In: 2018 15th conference on computer and robot vision (CRV). 2018. p. 95–101.

  28. Zeng H, Zhang Z, Shi L. Research and implementation of video codec based on ffmpeg. In: 2016 international conference on network and information systems for computers (ICNISC). 2016. p. 184–88.

  29. Zhou X, Wang D, Krähenbühl P. Objects as points. 2019. arXiv:1904.07850 [CoRR abs].

  30. Zulko. Automatic soccer highlights. 2014. http://zulko.github.io/blog/2014/07/04/automatic-soccer-highlights-compilations-with-python.

Download references

Funding

Thanh-Cong Le was funded by Vingroup Joint Stock Company and supported by the Domestic Master/PhD Scholarship Programme of Vingroup Innovation Foundation (VINIF), Vingroup Big Data Institute (VINBIGDATA), code VINIF.2020.ThS.JVN.05. This work was also funded by Gia Lam Urban Development and Investment Company Limited, Vingroup, and partially supported by Vingroup Innovation Foundation (VINIF) under project code VINIF.2019.DA19.

Author information

Authors and Affiliations

  1. University of Science, VNU-HCM, Ho Chi Minh City, Vietnam

    Thanh-Cong Le, Quoc-Vuong Nguyen & Minh-Triet Tran

  2. John von Neumann Institute, VNU-HCM, Ho Chi Minh City, Vietnam

    Thanh-Cong Le & Minh-Triet Tran

  3. Vietnam National University, Ho Chi Minh City, Vietnam

    Thanh-Cong Le, Quoc-Vuong Nguyen & Minh-Triet Tran

Authors
  1. Thanh-Cong Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Quoc-Vuong Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minh-Triet Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Minh-Triet Tran.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Future Data and Security Engineering 2020” guest edited by Tran Khanh Dang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Le, TC., Nguyen, QV. & Tran, MT. SmartEyes: Social Multimedia Analysis Platform for Open Data Providers. SN COMPUT. SCI. 3, 33 (2022). https://doi.org/10.1007/s42979-021-00843-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-021-00843-x

Keywords

Search

Navigation