Skip to main content
SpringerLink
Log in

Semantic video database system with semi-automatic secondary-content generation capability

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A semantic video database system, based on an interactive approach that maps low-level features to high-level concepts, is proposed. A database of ontological semantic object models allows the user to get information about specific semantic objects, such as certain actors or other features of a TV drama. The system searches the database for key frames within the video to detect similarities in detailed or “low-level” features, such as the color, area, and position of a specific part of the frame. Since image recognition techniques are limited in their ability to fully identify and compare images, we propose an additional function in which a coarse model is used to recover a greater number of similar key frames, thus providing more relevant results. From these results, the content provider can select relevant key frames interactively; the matched objects in them are then automatically annotated according to descriptions that are added into the model by content provider. Therefore, more complex content can be generated with greater accuracy by using a combination of application-oriented operations. The system has high potential for use in object-based interactive multimedia applications. We also present an object-based video content generation application called the Drama Characters’ Popularity Voting System.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Adali S, Candan KS, Chen S-S, Erol K, Subrahmanian VS (1996) Advanced video information system: data structures and query processing. ACM-Springer Multimedia System 4:172–186

    Article  Google Scholar 

  2. Aguierre-Smith TG, Davenport G (1992, November) The stratification system: a design environment for random access video. Proceeding of the Third International Workshop Network and Operating System Support for Digital Audio and Video

  3. Ardizzone E, Cascia ML (1997) Automatic video database indexing and retrieval. Multimed Tools Appl 4(1)

  4. Bargeron D, Gupta A, Grudin J, Sanocki E (1999, May 17) Annotation for streaming video on the web: system design and usage studies. Computer Networks (Netherlands), Elsevier Science, 31(11–16), pp 1139–1153

    Google Scholar 

  5. Chang S-F, Chen W, Meng H J, Sundaram H, Zhong D (1998, September) A fully automated content-based video search engine supporting spatiotemporal queries. IEEE Transactions on Circuits and Systems for Video Technology 8(5):602–615

    Google Scholar 

  6. Comaniciu D, Meer P (1997) Robust analysis of feature spaces: color image segmentation. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp 750–755

  7. Deng Y, Kenney C, Moore MS, Manjunath BS (1999) Peer group filtering and perceptual color image quantization. Proc of IEEE ISCAS 4:21–24

    Google Scholar 

  8. Deng Y, Manjunath BS, Shin H (1999) Color image segmentation. Proc. of IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, pp 446–451

  9. Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995, September) Query by image and video content: the QBIC system. IEEE Comput Mag 28:23–32

    Google Scholar 

  10. Hacid M-S, Decleir C, Kouloumdjian J (2000, September/October) A database approach for modeling and query video data. IEEE Trans on Knowledge and Data Engineering 12(5):729–750

    Google Scholar 

  11. Hauptmann A, Yan R, Qi Y, Jin R, Christel M, Derthick M, Chen M-Y, Baron R, Lin W-H, Ng TD (2002, November) Video classification and retrieval with the informedia digital video library system. Text retrieval conference (TREC’02), Gaithersburg, Maryland

  12. Hsieh JW, et al (2000) Region-based image retrieval. Proc. ICIP

  13. Kamijo S, Matsushita Y, Ikeuchi K, Sakauchi M (2000, June) Traffic monitoring and accident detection at intersections. IEEE Transaction on ITS 1(2):108–118

    Google Scholar 

  14. Liu Yan, Li Fei (2002) Semantic extration and semanties-based annotation and retrieval for video databases. Multimed Tools Appl 17:5–20

    Article  Google Scholar 

  15. Mehtre BM, Kankanhalli MS, Lee WF (1998) Content-based image retrieval using a composite color-shape approach. Inf Process Manag 34(1):109–120

    Article  Google Scholar 

  16. MPEG-7 Application Document V.3, ISO/IECJTC1/SC29/WG11 N2084, February 1998

  17. Nishida Tsunetoshi, Matsushita Takeshi, Kamijo Shunsuke, Sakauchi Masao (2002) Interactive system of analyzing traffic event statistics based on occlusion robust vehicle tracking method. Proc. of 9th world congress on ITS (CD-ROM)

  18. Oomoto E, Tanaka K (1993) OVID: design and implementation of a video-object database system. IEEE Trans Knowl Data Eng 5(4):62–643

    Article  Google Scholar 

  19. Osawa Y, Sakauchi M (1990,Feb) A new type data structure with homogeneous nodes suitable for a very large spatial database. Proc. of 6th International Conference on Data Engineering, pp 296–303

  20. Rui Y, Huang TS (1998, September) Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans CSVT 8(5):644–655

    Google Scholar 

  21. Smith J R, Chang S-F (1996, November) VisualSEEk: a fully automated content-based image query system. ACM Multimedia 96, Boston, Massachusetts

  22. Tran D A, Hua K A, Vu K (2000, September 4–8) Semantic reasoning based video database systems. In: Proc. of the 11th Int’l Conf. on Database and Expert Systems Applications, pp 41–50

  23. Yoon J, Jayant N (2001) Relevant feedback for semantics for semantics based image retrieval. Proc. ICIP

  24. Zhang W, Wu X, Kamijo S, Yaginuma Y, Sakauchi M (2002, December) Movie content retrieval and semi-automatic annotation based on low-level descriptions. The proceeding of third IEEE pacific rim conference on multimedia, pp 261–269, Springer LNCS 2532

  25. http://www.informedia.cs.cmu.edu/

Download references

Author information

Authors and Affiliations

  1. Sakauchi Lab, The 3rd Department, Institute of Industrial Science, University of Tokyo, Tokyo, Japan

    Wenli Zhang, Xiaomeng Wu, Shunsuke Kamijo & Masao Sakauchi

Authors
  1. Wenli Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaomeng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shunsuke Kamijo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Masao Sakauchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenli Zhang.

Additional information

W. Zhang graduated in 2004 from Sakauchi Lab, The 3rd Department, Institute of Industrial Science, University of Tokyo, Tokyo, Japan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, W., Wu, X., Kamijo, S. et al. Semantic video database system with semi-automatic secondary-content generation capability. Multimed Tools Appl 30, 27–54 (2006). https://doi.org/10.1007/s11042-006-0007-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-006-0007-5

Keywords

Search

Navigation