Abstract
The current multimedia landscape is characterized by a significant heterogeneity in terms of coding and delivery formats, usage environments, and user preferences. The main contribution of this paper is a discussion of the design and functioning of a fully integrated platform for multimedia adaptation and delivery, called NinSuna. This platform is able to efficiently deal with the aforementioned heterogeneity in the present-day multimedia ecosystem, thanks to the use of format-agnostic adaptation engines (i.e., engines independent of the underlying coding format) and format-agnostic packaging engines (i.e., engines independent of the underlying delivery format). Moreover, NinSuna also provides a seamless integration between metadata standards and adaptation processes. Both our format-independent adaptation and packaging techniques rely on a model for multimedia bitstreams, describing the structural, semantic, and scalability properties of these multimedia streams. News sequences were used as a test case for our platform, enabling the user to select news fragments matching his/her specific interests and usage environment characteristics.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Fig1_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Fig2_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Figg_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Fig3_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Fig4_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Figh_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Fig5_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Fig6_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Fig7_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Fig8_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-009-0354-0/MediaObjects/11042_2009_354_Fig9_HTML.gif)
Similar content being viewed by others
Notes
In the remainder of this paper, ‘XML-driven content adaptation’ is used as a collective term to refer to these techniques.
NinSuna is short for “The NinSuna INtelligent Search framework for UNiversal multimedia Access”.
The mime-type class is defined in the Core Ontology for MultiMedia (COMM, [3]).
To express the unit of measure, we use concepts of the Suggested Upper Merged Ontology (SUMO, [39]). Note that we use the OWL DL implementation of SUMO, which is available at http://www.stuarthendren.net/sumodl.
A website containing information regarding the NinSuna platform and an online demo is available on http://multimedialab.elis.ugent.be/NinSuna.
See http://www.w3.org/2008/01/media-annotations-wg.html for more information.
References
Amielh M, Devillers S (2001) Multimedia content adaptation with XML. In: Proceedings of 8th international conference on multimedia modeling, Amsterdam, pp 127–145
Amielh M, Devillers S (2002) Bitstream syntax description language: application of XML-schema to multimedia content adaptation. In: Proceedings of 11th international world wide web conference, Honolulu, Hawaii. http://wwwconf.ecs.soton.ac.uk/archive/00000185/01/index.html
Arndt R, Troncy R, Staab S, Hardman L, Vacura M (2007) COMM: designing a well-founded multimedia ontology for the web. In: 6th international semantic web conference (ISWC 2007), Busan
Christopoulos C, Skodras A, Ebrahimi T (2000) The JPEG2000 still image coding system: an overview. IEEE Trans Consumer Electron 46(4):1103–1127
Clark K, Feigenbaum L, Torres E (eds) (2008) SPARQL Protocol for RDF. W3C Recommendation. World Wide Web Consortium
De Neve W, Van Deursen D, De Schrijver D, De Wolf K, Van de Walle R (2005) Using bitstream structure descriptions for the exploitation of multi-layered temporal scalability in H.264/AVC‘s base specification. Lect Notes Comput Sci 3768:641–652
Decker S, Melnik S, van Harmelen F, Fensel D, Klein MCA, Broekstra J, Erdmann M, Horrocks I (2000) The semantic web: the roles of XML and RDF. IEEE Internet Computing 4(5):63–74
Devillers S, Timmerer C, Heuer J, Hellwagner H (2005) Bitstream syntax description-based adaptation in streaming and constrained environments. IEEE Trans Multimedia 7(3):463–470
Dublin Core Metadata Initiative (2004) Dublin core metadata element set, version 1.1: reference description. Technical report, Dublin Core Metadata Initiative. http://www.dublincore.org/documents/dces/
Eleftheriadis A (1997) Flavor: a language for media representation. In: Proceedings of ACM multimedia conference, Seattle, WA, pp 1–9
Hannuksela MM, Wang Y-K, Gabbouj M (2004) Isolated regions in video coding. IEEE Trans Multimedia 6:259–267
Hong D, Eleftheriadis A (2008) XFlavor: providing XML features in media representation. Multimedia Tools and Applications 39(1):101–116
ISO/IEC (2003) 15938-5:2003 information technology—multimedia content description interface—part 5: multimedia description schemes
ISO/IEC (2004) 14496-14:2003 Information technology—Coding of Audio, Picture, Multimedia and Hypermedia Information—Part 14: MP4 file format
ISO/IEC (2004) 21000-7:2004 information technology—multimedia framework (MPEG-21)—part 7: digital item adaptation
ISO/IEC (2007) 21000-18:2007 information technology—multimedia framework (MPEG-21)—part 18: digital item streaming
ISO/IEC (2007) Information technology—multimedia framework (MPEG-21)—part 7: digital item adaptation, amendment 2: dynamic and distributed adaptation. ISO/IEC 21000-7:2007/FPDAmd 2
ISO/IEC (2008) 23000-5: information technology—MPEG-B systems technologies part 5: bitstream syntax description language (BSDL)
ISO/IEC JTC 1 (1993) Information technology—coding of moving pictures and associated audio for digital storage media at up to about 1,5 Mbit/s—part 3: audio. ISO/IEC 11172-3:1993
ISO/IEC JTC 1 (2000) Information technology—generic coding of moving pictures and associated audio information: video. ISO/IEC 13818-2:2000
ISO/IEC JTC 1 (2005) Information technology—coding of audio-visual objects—part 3: audio. ISO/IEC 14496-3:2005
ITU-T and ISO/IEC JTC 1 (2003) Advanced video coding for generic audiovisual services. ITU-T Rec. H.264 and ISO/IEC 14496-10 AVC
Klyne G, Carroll JJ (eds) (2004) Resource description framework (RDF): concepts and abstract syntax. W3C Recommendation. World Wide Web Consortium
McGuinness D, van Harmelen F (eds) (2004) OWL web ontology language: overview. W3C Recommendation. World Wide Web Consortium
Mukherjee D, Delfosse E, Kim J-G, Wang Y (2005) Optimal adaptation decision-taking for terminal and network quality-of-service. IEEE Trans Multimedia 7(3):454–462
Ohm J-R (2005) Advances in scalable video coding. Proc IEEE 93(1):42–56
Panis G, Hutter A, Heuer J, Hellwagner H, Kosch H, Timmerer C, Devillers S, Amielh M (2003) Bitstream syntax description: a tool for multimedia resource adaptation within MPEG-21. Signal Process, Image Commun 18(8):721–747
Pfeiffer S, Parker C, Schremmer C (2003) Annodex: a simple architecture to enable hyperlinking, search & retrieval of time continuous data on the web. In: MIR ’03: proceedings of the 5th ACM SIGMM international workshop on multimedia information retrieval, Berkeley, pp 87–93
Prud’hommeaux E, Seaborne A (eds) (2007) SPARQL query language for RDF. W3C Recommendation. World Wide Web Consortium
Ransburg M, Devillers S, Timmerer C, Hellwagner H (2007) Processing and delivery of multimedia metadata for multimedia content streaming. In: Proceedings of 6th workshop on multimedia semantics—the role of metadata. Aachen, Germany
RFC 3550 (2003) RTP: a transport protocol for real-time applications. http://www.ietf.org/rfc/rfc3550.txt
RFC 2326 (1998) Real time streaming protocol. http://www.ietf.org/rfc/rfc2326.txt
RFC 2616 (1999) Hypertext transfer protocol—HTTP/1.1. http://www.w3.org/Protocols/rfc2616/rfc2616.html
RFC 3984 (2005) RTP payload format for H.264 video. http://www.ietf.org/rfc/rfc3984.txt
RFC 2327 (1998) SDP: session description protocol. http://www.ietf.org/rfc/rfc2327.txt
Schwarz H, Marpe D, Wiegand T (2007) Overview of the scalable video coding extension of the H.264/AVC standard. IEEE Trans Circuits Syst Video Technol 17(9):1103–1120
SMPTE (2004) Material exchange format (MXF)—file format specification (Standard). SMPTE 377M-2004
Srinivasan S, Tu C, Regunathan SL, Sullivan GJ (2007) HD photo: a new image coding technology for digital photography. In: Proceedings of the SPIE, vol 6696, San Diego
SUMO (2009) Suggested upper merged ontology. http://www.ontologyportal.org/
Thomas-Kerr J, Burnett I, Ritz C (2008) Format-independent rich media delivery using the bitstream binding language. IEEE Trans Multimedia 10(3):514–522
Van Deursen D, De Bruyne S, Van Lancker W, De Neve W, Hellwagner H, Van de Walle R (2007) MuMiVA: a multimedia delivery platform using format-agnostic, XML-driven content adaptation. In: Proceedings of the 9th international symposium on multimedia, Taichung, pp 131–138
Vetro A, Christopoulos C, Ebrahimi T (2003) Universal multimedia access. IEEE Signal Process Mag 20(2):16
W3C Multimedia Semantics Incubator Group (2006) Incubator Activity > W3C Multimedia Semantics Incubator Group. http://www.w3.org/2005/Incubator/mmsem/
Acknowledgements
The research activities as described in this paper were funded by Ghent University, the Interdisciplinary Institute for Broadband Technology (IBBT), the Institute for the Promotion of Innovation by Science and Technology in Flanders (IWT), the Fund for Scientific Research-Flanders (FWO-Flanders), and the European Union.
Author information
Authors and Affiliations
Corresponding author
Additional information
At the time of writing, Wesley De Neve was also with the Image and Video Systems Lab of the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.
Rights and permissions
About this article
Cite this article
Van Deursen, D., Van Lancker, W., De Neve, W. et al. NinSuna: a fully integrated platform for format-independent multimedia content adaptation and delivery using Semantic Web technologies. Multimed Tools Appl 46, 371–398 (2010). https://doi.org/10.1007/s11042-009-0354-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-009-0354-0