Skip to main content
SpringerLink
Log in

Personalized video adaptation framework (PIAF): high-level semantic adaptation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Despite much work on Universal Multimedia Experience (UME), existing video adaptation approaches cannot yet be considered as truly user-centric, mostly due to their poor handling of semantic user preferences. Indeed, these works mainly concentrate on lower-level user preferences but do neither consider any fine-grained object-level adaptation nor evaluate different adaptation options based on predicted user expectations. Moreover, these works do not provide owners with property rights that enable them to place restrictions on the types of modifications to be made to the video content. To address these shortcomings, we propose the Personalized vIdeo Adaptation Framework (PIAF) for high-level semantic video adaptation. PIAF is a fully integrated framework providing all the requirements for a semantic video adaptation. It defines a video annotation model and a user profile model comprising semantic constraints that are delineated in a consistent way, based on the standards MPEG-7 and MPEG-21. At the heart of the framework, the Adaptation Decision Taking Engine (ADTE) computes utility values for different adaptation options, considering each shot separately. The corresponding utility function evaluates the possible choices by evaluating multiple parameters that capture different dimensions of a multimedia experience: amount of modified content, modifications to key objects and shots with respect to the semantic integrity of the original content, expected processing cost of the adaptation, and the anticipated visual and temporal quality of the adapted content. Furthermore, the ADTE can deal with intellectual property issues by selecting an adaptation plan of good quality that also satisfies constraints specified by the content owner. This paper places a significant emphasis on theoretical details of the utility function and the computation of the adaptation plan. It also presents the results and evaluation of the adaptation process both in simulation and user study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. http://www.commercialalert.org/

  2. http://liris.cnrs.fr/advene/

  3. http://www.joanneum.at/en/digital/products-solutions/semantic-video-annotation.html

  4. http://www.research.ibm.com/VideoAnnEx/index.html

  5. http://mklab.iti.gr/via/

  6. http://www.avidemux.org/

  7. http://www.avidemux.org/

References

  1. Adzic V, Kalva H, Fuhrt B (2011) A survey of multimedia content adaptation for mobile devices. J Multimed Tools Appl 51(1):379–396. doi:10.1007/s11042-010-0669-x

    Article  Google Scholar 

  2. Bayou GA, Coquil D, Kosch H (2011) Enhancing Semantic Video Adaptation Speed in a Proxy Based Delivery System. In: Workshop on Multimedia on the Web in conjunction with the 12th International Conference on Knowledge Management and Knowledge Technologies

  3. Bruyne SD et al (2011) Annotation based personalized adaptation and presentation of videos for mobile applications. J Multimed Tools Appl 55(2):307–331. doi:10.1007/s11042-010-0575-2

    Article  Google Scholar 

  4. Chan TF, Kang SH (2006) Error analysis for image inpainting. J Math Imaging Vis 26(1–2):85–103

    Article  MathSciNet  Google Scholar 

  5. Chan T, Vese L (1999) An active contour model without edges. In Nielsen M, Johansen P, Olsen OF, Weickert J eds. Scale-space theories in computer vision, Lecture Notes in Comput. Sci. 1682, Springer, Berlin, Heidelberg: 141–151

  6. Cock JD, Notebaert S, Vermeirsch K et al (2010) Dyadic spatial resolution reduction transcoding for H.264/AVC. In: Journal of. Multimedia Systems 16(2):139–149

    Article  Google Scholar 

  7. Dasiopoulou S, Giannakidou E, Litos G, Malasioti P, Kompatsiaris Y (2011) A survey of semantic image and video annotation tools. In: Paliouras G, Spyropoulos CD, Tsatsaronis G Eds. Knowledge-driven multimedia information extraction and ontology evolution, Springer Verlag: 196–239

  8. El-Khoury V, Bennani N, Coquil D (2010) Utility function for semantic video content adaptation. In: iiWAS’2010 - The 12th international conference on information integration and Web-based applications and services: 921–924

  9. El-Khoury V, Jergler M, Coquil D, Kosch H (2012) Semantic Video Content Annotation at the Object Level. In: MoMM 2012- The 10th international Conference on Advances in Mobile Computing & Multimedia

  10. Hadhoud MM, Moustafa A, Shenoda SZ (2001) Digital Images Inpainting using Modified Convolution Based Method. In: journal of Signal Processing, Image Processing and Pattern Recognition

  11. ISO/IEC 21000-7 (2004) Information technology - multimedia framework (MPEG-21)-Part7: Digital Item Adaptation

  12. Kellerer H, Pferschy U, Pisinger D (2010) Knapsack problems. Springer, Berlin

    Google Scholar 

  13. Lankton S (2009) Sparse field methods – Technical report, Georgia institute of technology

  14. Lee WS, Bailer W, Bürger T, Champin PA et al. (2012) Ontology for media resources 1.0, W3C recommendation

  15. Lopez F, Martinez JM, Garcia N (2009) CAIN-21: an extensible and metadata-driven multimedia adaptation engine in the MPEG-21 framework. In: proceedings of the 4th international conference on semantic and digital media technologies: semantic multimedia (SAMT ’09), Graz, Austria, Springer-Verlag, Berlin, Heidelberg: 114–125. doi:10.1007/978-3-642-10543-2-12

  16. Magalhães J, Pereira F (2004) Using MPEG standards for multimedia customization. Signal Process Image Commun 19(5):437–456

    Article  Google Scholar 

  17. Mahalingam VK (2010) Digital inpainting algorithms and evaluation. Doctoral dissertation, University of Kentucky. http://uknowledge.uky.edu/gradschool_diss/55 Accessed 16.10.2011

  18. Martínez JM (2004) MPEG-7 overview (version 10). ISO/IEC JTC1/SC29/WG11/N5525

  19. MoCA project (2010) Movie content analysis. http://pi4.informatik.uni-mannheim.de/pi4.data/content/projects/moca/Project-ResolutionAdaptation.html. Accessed 15.10.2011

  20. Nielsen F, Nock R (2005) ClickRemoval: interactive pinpoint image object removal. In ACM Multimedia 2005:315–318

    Google Scholar 

  21. Pereira F, Burnett I (2003) Universal multimedia experiences for tomorrow. IEEE Signal Proc Mag 20(2):63–73

    Article  Google Scholar 

  22. Pisinger D (1994) A minimal Algorithm for Multiple-Choice Knapsack problem. Eur J Oper Res 83:394–410

    Article  Google Scholar 

  23. Prangl M, Kofler I, Hellwagner H (2008) An MPEG-21-driven utility-based multimedia adaptation decision taking Web service. In: proceedings of the 1st international conference on ambient media and systems (Ambi-sys’08), Quebec, Canada

  24. Rijsselbergen DV, Poppe C, Verwaest M, Mannens E, Walle RVD (2011) Semantic Mastering: content adaptation in the creative drama production workflow. In: Journal of Multimedia Tools and Applications: 1–34 doi: 10.1007/s11042-010-0710-0

  25. Shi Y, Karl WC (2005) Real-time tracking using level sets. In: proceedings of the IEEE Computer Society conference on computer vision and pattern recognition: 34–41

  26. Shih TK, Rong-Chi C (2005) Digital Inpainting – survey and multilayer image inpainting algorithms. In: proceedings of the 3rd international conference in Information Technology and Applications (ICITA’05), Taipei, Taiwan: 15–24 vol. 1

  27. Sinha A, Zoltners AA (1979) The multiple-choice knapsack problem. J Oper Res 27(3):503–515

    Article  MATH  MathSciNet  Google Scholar 

  28. Sofokleous AA, Angelides MC (2008) DCAF: An MPEG-21 dynamic content adaptation framework. Multimed Tools Appl 40(2):151–182. doi:10.1007/s11042-008-0198-z

    Article  Google Scholar 

  29. Stampou G, Van Ossenbruggen J, Pan JZ, Schreiber G (2006) Multimedia annotations and the semantic Web. IEEE Multimedia 13(1):86–90

    Article  Google Scholar 

  30. Stegmaier F, Döller M, Coquil D, El-Khoury V, Kosch H (2010) VAnalyzer: a MPEG-7 based semantic video annotation tool. In: proceedings of the Workshop on Interoperable Social Multimedia applications (WISMA 2010): 85–86

  31. Subjective video quality assessment methods for multimedia applications, ITU-T Recommendation P. 910, 2008. [Online]. Available: http://www.itu.int/rec/T-REC-P.910/en

  32. Troncy R, Mannens E, Pfeiffer S, Van Deursen D et al. (2011) Media fragments URI 1.0. W3C candidate recommendation

  33. Van DD, Lancker WV, Neve WD, Paridaens T, Mannens E, Walle RVD (2010) NinSuna: a fully integrated platform for format-independent multimedia content adaptation and delivery using semantic Web technologies. Multimed Tools Appl 46(2–3):371–398. doi:10.1007/s11042-009-0354-0

    Google Scholar 

  34. Zhang HY, Peng QC (2007) A Survey on Digital Image Inpainting. In: Journal of Image and Graphics. doi: http://en.cnki.com.cn/Article_en/CJFDTOTAL-ZGTB200701001.htm

  35. Zhongkang Lu et al. (2005) Integrated Evaluation of Temporal and Spatial Distortions for Low Bit-rate Videos. In: Proceedings of 7th IEEE Workshop on Multimedia Signal Processing

  36. Zufferey M, Kosch H (2006) Semantic Adaptation of Multimedia Content. In: Proceedings of the 48th International Symposium ELMAR-2006 focused on Multimedia Signal Processing and Communications. IEEE CS Press. Zadar, Croatia

Download references

Acknowledgments

This work was conducted in the framework of the Multimedia Distributed and Pervasive Secure Systems (MDPS) doctoral college. The MDPS is a doctoral college supported by the “Université Franco-Allemande” (CDFA-05-08).

Author information

Authors and Affiliations

  1. Lyon University, CNRS, INSA-Lyon, LIRIS, UMR5205, 7 Avenue Jean Capelle, 69621, Villeurbanne Cedex, France

    Vanessa El-Khoury, Nadia Bennani & Lionel Brunie

  2. Faculty of Informatics and Mathematics, University of Passau, Innstrasse 43, 94032, Passau, Germany

    David Coquil

Authors
  1. Vanessa El-Khoury
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Coquil
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nadia Bennani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lionel Brunie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vanessa El-Khoury.

Rights and permissions

Reprints and permissions

About this article

Cite this article

El-Khoury, V., Coquil, D., Bennani, N. et al. Personalized video adaptation framework (PIAF): high-level semantic adaptation. Multimed Tools Appl 70, 1099–1140 (2014). https://doi.org/10.1007/s11042-012-1225-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-012-1225-7

Keywords

Search

Navigation