Abstract
Querying live media streams is a challenging problem that is becoming an essential requirement in a growing number of applications. Research in multimedia information systems has addressed and made good progress in dealing with archived data. Meanwhile, research in stream databases has received significant attention for querying alphanumeric symbolic streams. The lack of a data model capable of representing different multimedia data in a declarative way, hiding the media heterogeneity and providing reasonable abstractions for querying live multimedia streams poses the challenge of how to make the best use of data in video, audio and other media sources for various applications. In this paper we propose a system that enables directly capturing media streams from sensors and automatically generating more meaningful feature streams that can be queried by a data stream processor. The system provides an effective combination between extendible digital processing techniques and general data stream management research. Together with other query techniques developed in related data stream management streams, our system can be used in those application areas where multifarious live media senors are deployed for surveillance, disaster response, live conferencing, telepresence, etc.
Similar content being viewed by others
References
Abadi D, Carney D, Cetintemel U et al (2003) Aurora: a new model and architecture for data stream management. VLDB J 12(2):120–139, Aug
Arasu A, Babu S, Widom J (2003) The cql continuous query language: semantic foundations and query execution. Technical report, Stanford University, Oct
Aref WG, Catlin AC et al. (2004) Vdbms: a testbed facility for research in video database benchmarking. ACM Multimed Syst J, Special Issue on Multimed Doc Manag Syst 9(6):575–585, June
Babcock B, Babu S, Datar M, Motwani R, Widom J (2002) Models and issues in data stream systems. In: Symposium on principles of database systems, Madison, Wisconsin, pp 1–16, May
Bonnet P, Gehrke J, Seshadri P (2001) Towards sensor database systems. In: Proc. 2nd int. conf. on mobile data management, pp 3–14
Chen J, DeWitt DJ, Tian F, Wang Y (2000) Niagaracq: a scalable continuous query system for internet databases. In: International conference on management of data, Dallas, Texas, pp 379–390, May
Elmasri R, Wuu GTJ, Kim YJ (1990) The time index: an access structure for temporal data. VLDB, pp 1–12
Enderle J, Hampel M, Seidl T (2004) Joining interval data in relational databases. In: Proc. ACM SIGMOD
Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image and video content: the qbic system. IEEE Comput 28(9):23–32, Sept
Golab L, Ozsu MT (2003) Issues in data stream management. ACM SIGMOD Record 32(2):5–14, June
Hongeng S, Nevatia R, Bremond F (2004) Video-based event recognition: activity representation and probabilistic recognition methods. Comput Vis Image Underst (96)2:129–162, November
Intel (2007) Open source computer vision library. Opencv. web site at http://www.intel.com/research/mrl/research/opencv/
Jain R (2003) Experiential computing. Communications of the ACM 46(7):48–55, July
Lillethun D, Hilley D, Horrigan S, Ramachandran U (2007) MB++: an integrated architecture for pervasive computing and high-performance computing. In: Proceedings of the 13th IEEE international conference on embedded and real-time computing systems and applications (RTCSA ’07), pp 241–248, Aug
Liu B, Gupta A, Jain R (2005) Medsman: a streaming data management system over live multimedia. In: Proceedings of the 13th annual ACM international conference on multimedia, pp 171–180, Nov
Liu B, Gupta A, Jain R (2006) Using temporal semantics for live media stream queries. In: International Conference on Semantics of a Networked World, March
Liu L, Pu C, Tang W (1999) Continual queries for internet scale event-driven information delivery. IEEE Trans Knowl Data Eng 11(4):610–628
Liu X, Corner M, Shenoy P (2005) Seva: sensor-enhanced video annotation. In: Proceedings of the 13th annual ACM international conference on multimedia, pp 171–180, Nov
Madden S, Shah M, Hellerstein JM, Raman V (2002) Continously adaptive continous queries over streams. In: ACM SIGMOD international conference on management of data, Madison, Wisconsin
Ramachandran U, Modahl M, Bagrak I, Wolenetz M, Lillethun D, Liu B, Kim J, Hutto P, Jain R (2005) Media broker: a pervasive computing infrastructure for adaptive transformation and sharing of stream data. To appear in Pervasive and Mobile Computing (PMC) Journal, vol 2
Salemier P, Smith JR (2002) Introduction to MPEG-7: multimedia content description interface. Wiley
Seshadri P, Livny M, Ramakrishnan R (1994) Sequence query processing. ACM SIGMOD, pp 430–441
Seshadri P, Livny M, Ramakrishnan R (1995) Seq: a model for sequence databases. ICDE, pp 232–239
Soo MD, Snodgrass RT, Jensen CS (1994) Efficient evaluation of the valid-time natural join. ICDE, pp 282–292
Steinmetz R, Nahrstedt K (1995) Multimedia computing, communications and applications. Prentice Hall
Tansel AU, Clifford J, Gadia SK, Segev A, Snodgrass RT (1993) Temporal databases: theory, design, and implementation. Benjamin/Cummings
Vinod VV, Murase H (1997) Video shot analysis using efficient multiple object tracking. In: ICMCS ’97, Ottawa, Ontario, CANADA, June
Viola P, Jones MJ (2001) Rapid object detection using a boosted cascade of simple features. In: IEEE CVPR, vol 1, pp 511–518
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, B., Gupta, A. & Jain, R. MedSMan: a live multimedia stream querying system. Multimed Tools Appl 38, 209–232 (2008). https://doi.org/10.1007/s11042-007-0177-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-007-0177-9