Skip to main content
SpringerLink
Log in

Towards robust and reliable multimedia analysis through semantic integration of services

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Thanks to ubiquitous Web connectivity and portable multimedia devices, it has never been so easy to produce and distribute new multimedia resources such as videos, photos, and audio. This ever-increasing production leads to an information overload for consumers, which calls for efficient multimedia retrieval techniques. Multimedia resources can be efficiently retrieved using their metadata, but the multimedia analysis methods that can automatically generate this metadata are currently not reliable enough for highly diverse multimedia content. A reliable and automatic method for analyzing general multimedia content is needed. We introduce a domain-agnostic framework that annotates multimedia resources using currently available multimedia analysis methods. By using a three-step reasoning cycle, this framework can assess and improve the quality of multimedia analysis results, by consecutively (1) combining analysis results effectively, (2) predicting which results might need improvement, and (3) invoking compatible analysis methods to retrieve new results. By using semantic descriptions for the Web services that wrap the multimedia analysis methods, compatible services can be automatically selected. By using additional semantic reasoning on these semantic descriptions, the different services can be repurposed across different use cases. We evaluated this problem-agnostic framework in the context of video face detection, and showed that it is capable of providing the best analysis results regardless of the input video. The proposed methodology can serve as a basis to build a generic multimedia annotation platform, which returns reliable results for diverse multimedia analysis problems. This allows for better metadata generation, and improves the efficient retrieval of multimedia resources.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. For the sake of simplicity, false positives are not included in the example of Fig. 2, however, they are considered in the final platform (Section 4.2).

  2. http://opencv.org/

  3. Results: http://users.ugent.be/~bjdmeest/data_SemIntegrationFramework.zip

  4. http://www.ces.clemson.edu/~stb/research/headtracker/seq/

References

  1. (2014). World Telecommunication/ICT Indicators database 2014, 18 edn. International Telecommunication Union (ITU)

  2. Atrey PK, Hossain MA, El Saddik A, Kankanhalli MS (2010) Multimodal fusion for multimedia analysis: a survey. Multimedia systems 16(6):345–379

    Article  Google Scholar 

  3. Beckett D (2004) RDF/XML syntax specification (revised). http://www.w3.org/TR/REC-rdf-syntax/. Accessed on April 15th, 2013

  4. Berners-Lee T, Hendler J, Lassila O, et al. (2001) The semantic web. Sci. Am. 284(5):28–37

    Article  Google Scholar 

  5. Chamin Morikawa CM, Kiyoharu Aizawa KA (2012) Iconic visual queries for face image retrieval. Journal of Convergence 3(3):39–46

    Google Scholar 

  6. De Roo J (2013) Euler Yet another proof Engine. http://eulersharp.sourceforge.net/. Accessed on April 7th

  7. Drummond N, Shearer R (2006) The open world assumption. Presentation at eSI Workshop: The Closed World of Databases meets the Open World of the Semantic Web

  8. Erdmann M, Maedche A, Schnurr HP, Staab S (2000) From manual to semi-automatic semantic annotation: About ontology-based text annotation tools. In: Proceedings of the COLING-2000 Workshop on Semantic Annotation and Intelligent Content, pp. 79–85. Association for Computational Linguistics

  9. Fensel D, Bussler C (2002) Semantic web enabled web services. In: Jarke M, Lakemeyer G, Koehler J (eds) Proceedings of the 25th Annual German Conference on AI: KI 2002: Advances in Artificial Intelligence, vol 25, pp 319–319. Springer, Aachen

    Google Scholar 

  10. Hanani U, Shapira B, Shoval P (2001) Information filtering: Overview of issues, research and systems. User Model User-Adap Inter 11(3):203–259. doi:10.1023/A:1011196000674

    Article  MATH  Google Scholar 

  11. Hauptmann AG (2005) Lessons for the future from a decade of informedia video analysis research. In: Leow WK, Lew M, Chua TS, Ma WY, Chaisorn L, Bakker E (eds) Image and Video Retrieval, Lecture Notes in Computer Science, vol 3568, pp 1–10. Springer, Berlin

    Google Scholar 

  12. Hjelmås E, Low BK (2001) Face detection: A survey. Comp Vision Image Underst 83(3):236–274. doi:10.1006/cviu.2001.0921

    Article  MATH  Google Scholar 

  13. Huang YP, Lai SL (2012) Novel query-by-humming/singing method with fuzzy inference system. Journal of Convergence 3(4):1–8

    Google Scholar 

  14. Jaeger MC, Rojec-Goldmann G, Muhl G (2004) QoS aggregation for web service composition using workflow patterns. In: Proceedings of the Eighth IEEE International Conference on Enterprise Distributed Object Computing (EDOC), vol 8, pp 149–159. IEEE, Monterey

    Book  Google Scholar 

  15. Lanthaler M, Gütl C (2013) Hydra: A Vocabulary for Hypermedia-Driven Web APIs. In: Bizer C, Heath T, Berners-Lee T, Hausenblas M, Auer S (eds) Proceedings of the WWW2013 Workshop on Linked Data on the Web (LDOW), vol 6. Rio de Janeiro, Brazil

    Google Scholar 

  16. Ma M, Park DW, Kim SK, An S (2012) Online recognition of handwritten korean and english characters. Journal of Information Processing Systems 8(4):653–669. doi:10.3745/JIPS.2012.8.4.653

    Article  Google Scholar 

  17. Menasce DA (2004) Composing web wervices: A QoS view. IEEE Internet Computing 8(6):88–90. doi:10.1109/MIC.2004.57

    Article  Google Scholar 

  18. Ohkawara T, Aikebaier A, Enokido T, Takizawa M (2012) Quorums-based replication of multimedia objects in distributed systems. Human-centric Computing and Information Sciences 2(1):11. doi:10.1186/2192-1962-2-11

    Article  Google Scholar 

  19. Pauwels P, Bod R (2013) Including the power of interpretation through a simulation of Peirce’s process of inquiry. Literary and Linguistic Computing (LLC) 28(3):452–460

    Article  Google Scholar 

  20. Sarkar K, Nasipuri M, Ghose S (2012) Machine learning based keyphrase extraction: Comparing decision trees, naïve bayes, and artificial neural networks. Journal of Information Processing Systems 8(4):693–712. doi:10.3745/JIPS.2012.8.4.693

    Article  Google Scholar 

  21. Satone M, Kharate GK (2012) Face recognition based on pca on wavelet subband of average-half-face. Journal of Information Processing Systems 8(3):483–494. doi:10.3745/JIPS.2012.8.3.483

    Article  Google Scholar 

  22. Schapire RE (2003) The boosting approach to machine learning: An overview. Nonlinear Estimation and Classification. Lecture Notes in Statistics 171(7):149–172

    Article  MathSciNet  MATH  Google Scholar 

  23. Silas S, Ezra K, Blessing Rajsingh E (2012) A novel fault tolerant service selection framework for pervasive computing. Human-centric Computing and Information Sciences 2(1):5. doi:10.1186/2192-1962-2-5

    Article  Google Scholar 

  24. Smith A (2013) Smartphone ownership – 2013 update. Pew Research Center: Washington DC:12

  25. Smith DR (1985) The design of divide and conquer algorithms. Sci Comput Program 5(0):37–58. doi:10.1016/0167-6423(85)90003-6

    Article  MathSciNet  MATH  Google Scholar 

  26. Smith JR, Schirling P (2006) Metadata standards roundup. MultiMedia IEEE 13(2):84–88

    Article  Google Scholar 

  27. Verborgh R, Steiner T, Van Deursen D, De Roo J, Van de Walle R, Gabarró Vallés J (2013) Capturing the functionality of Web services with functional descriptions. Multimedia Tools and Applications 64(2):365–387

    Article  Google Scholar 

  28. Verborgh R, Van Deursen D, Mannens E, Poppe C, Van de Walle R (2012) Enabling context-aware multimedia annotation by a novel generic semantic problem-solving platform. Multimedia Tools and Applications 61(1):105–129. doi:10.1007/s11042-010-0709-6

    Article  Google Scholar 

  29. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 511–518. Kauai, HI, USA

  30. Wolfgang P (1994) Design patterns for object-oriented software development, 1 edn. Addison-Wesley (C)

  31. Yen NY, Kuo SYF (2012) An intergrated approach for internet resources mining and searching. Journal of Convergence 3(2):37–44

    Google Scholar 

  32. Zadeh LA (1988) Fuzzy logic. Computer 21(4):83–93. doi:10.1109/2.53

    Article  Google Scholar 

  33. Zeng L, Benatallah B, Ngu AH, Dumas M, Kalagnanam J, Chang H (2004) QoS-aware middleware for web services composition. IEEE Trans Softw Eng 30(5):311–327

    Article  Google Scholar 

  34. Zhu Y, Jin Q (2012) An adaptively emerging mechanism for context-aware service selections regulated by feedback distributions. Human-centric Computing and Information Sciences 2(1):15. doi:10.1186/2192-1962-2-15

    Article  Google Scholar 

Download references

Acknowledgements

The described research activities were funded by Ghent University, iMinds, the Institute for the Promotion of Innovation by Science and Technology in Flanders (IWT), the Fund for Scientific Research Flanders (FWO Flanders), and the European Union.

Author information

Authors and Affiliations

  1. Ghent University - iMinds - Multimedia Lab, Gaston Crommenlaan 8 bus 201, 9050, Ledeberg-Ghent, Belgium

    Ben De Meester, Ruben Verborgh, Erik Mannens & Rik Van de Walle

  2. Ghent University - Department of Architecture and Urban Planning, Jozef Plateaustraat 22, 9000, Ghent, Belgium

    Pieter Pauwels

  3. Multimedia Lab, Ghent University – iMinds, Gaston Crommenlaan 8 bus 201, 9050, Ledeberg-Ghent, Belgium

    Wesley De Neve

  4. Image and Video Systems Lab, KAIST, 335 Ghawak-ro (373-1 Guseong-dong), Yuseong-gu, Daejeon, 305-701, Republic of Korea

    Wesley De Neve

Authors
  1. Ben De Meester
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruben Verborgh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pieter Pauwels
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wesley De Neve
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Erik Mannens
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rik Van de Walle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ben De Meester.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

De Meester, B., Verborgh, R., Pauwels, P. et al. Towards robust and reliable multimedia analysis through semantic integration of services. Multimed Tools Appl 75, 14019–14038 (2016). https://doi.org/10.1007/s11042-014-2445-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2445-9

Keywords

Search

Navigation