Skip to main content

Advertisement

Springer Nature Link
Log in

Semantic Mastering: content adaptation in the creative drama production workflow

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In order to provide audiences with a proper universal multimedia experience, all classes of media consumption devices, from high definition displays to mobile media players, must receive a product that is not only adapted to their capabilities and usage environments, but also conveys the semantics and cinematography behind the narrative in an optimal way. This paper introduces a semantic video adaptation system that incorporates the media adaptation process in the center of the drama production process. Producers, directors and other creative staff instruct the semantic adaptation system using common cinematographic terminology and vocabulary, thereby seamlessly extending the drama production process into the realm of content adaptation. The multitude of production metadata obtained from various steps in the production process provides a valuable context of narrative semantics that is exploited by the adaptation process. As such, high definition imagery can be intelligently adapted to smaller resolutions while optimally fulfilling the filmmaker’s dramatic intentions with respect to the original narrative and obeying various rules of cinematographic grammar.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Arijon D (1976) Grammar of the film language. Silman-James Press, Los Angeles, CA

    Google Scholar 

  2. Avidan S, Shamir A (2007) Seam Carving for Content-Aware Image Resizing. ACM Trans Graph 26(3):1–9

    Article  Google Scholar 

  3. Bares WH, Lester JC (1999) Intelligent multi-shot visualization interfaces for dynamic 3D worlds. In: Proceedings of the 4th international conference on intelligent user interfaces, pp 119–126

  4. Bares W, McDermott S, Boudreaux C, Thainimit S (2000) Virtual 3D camera composition from frame constraints. In: Proceedings of the eighth ACM international conference on multimedia, pp 177–186

  5. Bertini M, Cucchiara R, Del Bimbo A, Prati A (2006) Semantic adaptation of sport videos with user-centred performance analysis. IEEE Trans Multimedia 8(3):433–443

    Article  Google Scholar 

  6. Cardinaels M, Frederix K, Nulens J, Van Rijsselbergen D, Verwaest M, Bekaert P (2008) A multi-touch 3D set modeler for drama production. In: Proc. of the IBC conf., pp 330–335

  7. Catmull E, Rom R (1974) A class of local interpolating splines. In: Barnhill RE, Reisenfeld RF (eds) Computer aided geometric design. Academic Press, New York, pp 317–326

    Google Scholar 

  8. Chen F, De Vleeschouwer C (2009) Autonomous production of basketball videos from multi-sensored data with personalized viewpoints. In: International workshop on image analysis for multimedia interactive services, pp 81–84

  9. Cheng W-H, Wang C-W, Wu J-L (2007) Video adaptation for small display based on content recomposition. IEEE Trans Circuits Syst Video Technol 17(1):43–58

    Article  MathSciNet  Google Scholar 

  10. Christianson DB, Anderson SE, He L, Salesin DH, Weld DS, Cohen MF (1996) Declarative camera control for automatic cinematography. In: Proc. of the AAAI-96, pp 148–155

  11. Christie M, Machap R, Normand J-M, Olivier P, Pickering J (2005) Virtual Camera Planning: A survey. In: Proceedings of smart graphics. Springer, pp 40–52

  12. De Geyter M, Oorts N, Overmeire L (2008) Integration demands on MAM systems: a proof of concept solution. SMPTE Motion Imaging Journal 118(8):38–46

    Article  Google Scholar 

  13. Deselaers T, Dreuw P, Ney H (2008) Pan, zoom, scan—time-coherent, trained automatic video cropping. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 1–8

  14. Dorai C, Venkatesh S (2001) Computational media aesthetics: finding meaning beautiful. IEEE Multimed 8(4):10–12

    Article  Google Scholar 

  15. Halper N, Helbing R, Strothotte T (2001) A camera engine for computer games: managing the trade-off between constraint satisfaction and frame coherence. Comput Graph Forum 20(3):174–183

    Article  Google Scholar 

  16. Jannach D, Leopold K (2007) Knowledge-based multimedia adaptation for ubiquitous multimedia consumption. J Netw Comput Appl 30(3):958–982

    Article  Google Scholar 

  17. Knoche H, Sasse A (2009) The big picture on small screens delivering acceptable video quality in mobile TV. ACM Trans Multimedia Comput Commun Appl 5(3):1–27

    Article  Google Scholar 

  18. Kopf S, Kiess J, Lemelson H, Effelsberg W (2009) FSCAV: fast seam carving for size adaptation of videos. In: ACM MM ’09: proceedings of the 17th ACM international conference on multimedia, pp 321–330

  19. Kovalick A (2006) Video systems in an IT environment. Elsevier Inc, Oxford

    Google Scholar 

  20. Krähenbühl P, Lang M, Hornung A, Gross M (2009) A system for retargeting of streaming video. In: ACM SIGGRAPH Asia ’09: international conference on computer graphics and interactive techniques

  21. Liu F, Gleicher M (2005) Automatic image retargeting with fisheye-view warping. In: UIST ’05: proceedings of the 18th annual ACM symposium on user interface software and technology, pp 153–162

  22. Liu F, Gleicher M (2006) Video retargeting: automating pan and scan. In: Proceedings of the ACM multimedia conference, PP 241–250

  23. López F, Martínez JM, García N (2009) CAIN-21: an extensible and metadata-driven multimedia adaptation engine in the MPEG-21 framework. In: Proceedings of the 4th international conference on semantic and digital media technologies, pp 114–125

  24. Magalhães J, Pereira F (2004) Using MPEG standards for multimedia customization. Signal Process Image Commun 19(5):437–456

    Article  Google Scholar 

  25. Mascelli JV (1998) The Five C’s of Cinematography: motion picture filming techniques. Silman-James Press, Los Angeles, CA

    Google Scholar 

  26. Parr T (2008) The reuse of grammars with embedded semantic actions. In: Proceedings of the 16th IEEE international conference on program comprehension, pp 5–10

  27. Pellan B, Concolato C (2009) Summarization of scalable multimedia documents. In: Proc. of the intl. workshop on image analysis for multimedia interactive services 2009, pp 304–307

  28. Pereira F, Burnett I (2003) Universal multimedia experiences for tomorrow. IEEE Signal Process Mag 20(2):63–73

    Article  Google Scholar 

  29. Rochart G (2008) The CHOCO constraint programming solver. In: Proceedings of the fifth international conference on integration of AI and OR techniques in constraint programming for combinatorial optimization problems

  30. Rubinstein M, Shamir A, Avidan S (2008) Improved seam carving for video retargeting. ACM Trans Graph 27(3):1–9

    Article  Google Scholar 

  31. Rubinstein M, Shamir A, Avidan S (2009) Multi-operator media retargeting. ACM Trans Graph 28(3):1–11

    Article  Google Scholar 

  32. Schwitter R (2002) English as a formal specification language. In: Proceedings of the 13th international workshop on database and expert systems applications, pp 228–232

  33. Seo K, Ko J, Ahn I, Kim C (2007) An intelligent display scheme of soccer video on mobile devices. IEEE Trans Circuits Syst Video Technol 17(10):1395–1401

    Article  Google Scholar 

  34. Setlur V, Takagi S, Raskar R, Gleicher M, Gooch B (2005) Automatic image retargeting. In: Proceedings of the 4th international conference on Mobile and Ubiquitous Multimedia, MUM ’05, pp 59–68

  35. Sofokleous AA, Angelides MC (2008) DCAF: an MPEG-21 dynamic content adaptation framework. Multimedia Tools & Applications 40(2):151–182

    Article  Google Scholar 

  36. Tseng BL, Lin C-Y, Smith JR (2004) Using MPEG-7 and MPEG-21 for Personalizing video. IEEE Multimed 11(1):42–53

    Article  Google Scholar 

  37. van Beek P, Smith JR, Ebrahimi T, Suzuki T, Askelof J (2003) Metadata-driven multimedia access. IEEE Signal Process Mag 20(2):40–52

    Article  Google Scholar 

  38. Van Deursen D, Van Lancker W, De Neve W, Paridaens T, Mannens E, Van de Walle R (2010) NinSuna: a fully integrated platform for format-independent multimedia content adaptation and delivery using Semantic Web technologies. Multimedia Tools & Applications 46(2–3):371–398

    Article  Google Scholar 

  39. van Harmelen F, Lifschitz V, Porter B (eds) (2008) Handbook of Knowledge Representation. Elsevier B.V., Amsterdam

    MATH  Google Scholar 

  40. Van Rijsselbergen D, Van De Keer B, Van de Walle R (2008) The canonical expression of the drama product manufacturing processes. Multimedia Syst 14(6):395–403

    Article  Google Scholar 

  41. Van Rijsselbergen D, Van De Keer B, Verwaest M, Mannens E, Van de Walle R (2009) On the implementation of semantic content adaptation in the drama manufacturing process. In: Proceedings of the 2009 IEEE international conference on multimedia and expo, pp 822–825

  42. Van Rijsselbergen D, Van De Keer B, Verwaest M, Mannens E, Van de Walle R (2009) Movie script markup language. In: Proceedings of the 9th ACM symposium on document engineering, pp 161–170

  43. Wolf L, Guttmann M, Cohen-Or D (2007) Non-homogeneous content-driven video-retargeting. In: IEEE International Conference on Computer Vision, pp 1–6

Download references

Acknowledgements

The research activities that have been described in this paper were funded by Ghent University, VRT, IBBT, the Institute for the Promotion of Innovation by Science and Technology in Flanders (IWT), the Fund for Scientific Research-Flanders (FWO-Flanders), the Belgian Federal Science Policy Office (BFSPO), and the European Union.

The image in Fig. 2 was taken from “Thuis”, © 1995–2010 Vlaamse Radio & Televisie (V.R.T.); the images in Fig. 21 were taken from “No Country for Old Men”, © 2007 Miramax Film Corp. and Paramount Vantage, A Division of Paramount Pictures Corporation; the images in Fig. 18–20 were taken from “Casino Royale”, © 2006 Danjaq LLC, United Artists Corporation and Columbia Pictures Industries Inc.

Author information

Authors and Affiliations

  1. Department of Electronics and Information Systems (ELIS)—Multimedia Lab, Ghent University—IBBT, Gaston Crommenlaan 8/201, 9050, Ghent, Belgium

    Dieter Van Rijsselbergen, Chris Poppe, Erik Mannens & Rik Van de Walle

  2. VRT-medialab, Gaston Crommenlaan 10/101, 9050, Ghent, Belgium

    Maarten Verwaest

Authors
  1. Dieter Van Rijsselbergen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Chris Poppe
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Maarten Verwaest
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Erik Mannens
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Rik Van de Walle
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Dieter Van Rijsselbergen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Van Rijsselbergen, D., Poppe, C., Verwaest, M. et al. Semantic Mastering: content adaptation in the creative drama production workflow. Multimed Tools Appl 59, 307–340 (2012). https://doi.org/10.1007/s11042-010-0710-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-010-0710-0

Keywords