Skip to main content
SpringerLink
Log in

Automatic Video Database Indexing and Retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The increasing development of advanced multimedia applications requires new technologies for organizing and retrieving by content databases of still digital images or digital video sequences. To this aim image and image sequence contents must be described and adequately coded. In this paper we describe a system allowing content-based annotation and querying in video databases. No user action is required during the database population step. The system automatically splits a video into a sequence of shots, extracts a few representative frames (said r-frames) from each shot and computes r-frame descriptors based on color, texture and motion. Queries based on one or more features are possible. Very interesting results obtained during the severe tests the system was subjected to are reported and discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. P. Aigrain and P. Joly, “Medium knowledge-based macro-segmentation of video into sequences,” in Intelligent Multimedia Information Retrieval, M. Maybury (Ed.), 1996.

  2. E. Ardizzone and M. La Cascia, “Motion labeling in video databases,” Journal of Visual Languages and Computing: Special Issue on Image and Video Database Visual Browsing Querying and Retrieval, 1996, (submitted).

  3. E. Ardizzone, A. Chella, and S. Gaglio, “A hybrid architecture for shape reconstruction and object recognition,” in International Journal of Intelligent Systems, 1996, (in press).

  4. E. Ardizzone, G.A.M. Gioiello, M. La Cascia, and D. Molinelli, “A real-time neural approach to scene cut detection,” in Proc. of IS &T/SPIE—Storage & Retrieval for Image and Video Databases IV, San Jose, 1996.

  5. E. Ardizzone, M. La Cascia, and D. Molinelli, “Motion and color based video indexing and retrieval,” in Proc. of Int. Conf. on Pattern Recognition, ICPR, Wien, Austria, 1996.

    Google Scholar 

  6. F. Arman, A. Hsu, and M.Y. Chiu, “Image processing on compressed data for large video databases,” in Proc. First ACM Int. Conf. Multimedia, 1993.

  7. J.R. Bach, S. Paul, and R. Jain, “A visual information management system for the interactive retrieval of faces,” IEEE Transaction on Knowledge and Data Engineering, Vol. 5, 1993.

  8. J.L. Barron, D.J. Fleet, and S.S. Beauchemin, “Performances of optical flow techniques,” Int. Journal ofComputer Vision, Vol. 12, p. 43, 1994.

    Google Scholar 

  9. M.G. Christel, “Addressing the contents of video in a digital library,” in Proc. of ACM Workshop on Effective Abstractions in Multimedia: Layout, Presentation and Interaction, San Francisco, 1995.

  10. J.M. Corridoni and A. Del Bimbo, “Automatic video segmentation through editing analysis,” in Proc. of 8th Int. Conf. on Image Analysis and Processing, ICIAP, Sanremo, Italy, 1995.

    Google Scholar 

  11. J.M. Corridoni and A. DelBimbo, “Film semantic analysis,” in Proc. of Computer Architecture for MachinePerception, CAMP, Como, Italy, 1995.

    Google Scholar 

  12. J.M. Corridoni, A. DelBimbo, and D. Lucarella, “Navigation and visualization of movies content,” in Proc. Int. Conf. on Visual Languages, VL'95, Darmstadt, Germany, 1995.

    Google Scholar 

  13. C. Faloutsos, R. Barber, M. Flickner, J. Hafner, W. Niblack, D. Pektovic, and W. Equitz, “Efficient and effective querying by image content,” Journal of Intelligent Information Systems, Vol. 3, p. 231, 1994.

    Google Scholar 

  14. M. Flickner et al. “Query by image and video content: The qbic system,” IEEE Computer, Sept. 1995.

  15. W.I. Grosky, “Multimedia information systems,” IEEE Multimedia, Vol. 2, 1994.

  16. V.N. Guditava and V.V. Raghavan, “Content-based image retrieval systems,” IEEE Computer, Sept. 1995.

  17. A. Hampapur, T. Weymouth, and R. Jain, “Digital video segmentation,” in ACM Multimedia ’94 Proceedings, ACM Press, 1994.

  18. A. Hampapur, R. Jain, and T. Weymouth, “Production model based digital video segmentation,” Journal of Multimedia Tools and Applications, Vol. 1, No. 1, pp. 9–46, March 1995.

    Google Scholar 

  19. B.K.P. Horn and B.G. Schunk, “Determining optical flow,” Artificial Intelligence, p. 17, 1981.

  20. R.W.G. Hunt, Measuring Color, John Wiley & Sons, 1989.

  21. P.M. Kelly and T.M. Cannon, “Candid: Comparison algorithm for navigating digital image databases,” in Proc. of the Seventh International Working Conference on Scientific and Statistical Database Management, Charlottesville VA, 1994.

  22. P.M. Kelly, T.M. Cannon, and D.R. Hush, “Query by image example: The candid approach,” in Proc. of SPIE: Storage and Retrieval for Image and Video Database III, 1995.

  23. M. La Cascia and E. Ardizzone, “Jacob: Just a content-based query system for video databases,” in Proc. of IEEE Int. Conf. on Acoustics, Speech and Signal Processing, ICASSP-96, Atlanta, May, 1996, pp. 7–10.

  24. D. Lee, R. Barber, W. Niblack, M. Flickner, J. Hafner, and D. Petkovic, “Indexing for complex queries on a query by content image database,” in Proceedings of the ICPR, Vol. 1, pp. 142–146, 1994.

    Google Scholar 

  25. D. Lee, R. Barber, W. Niblack, M. Flickner, J. Hafner, and D. Petkovic, “Query by image content using multiple objects and multiple features: User interface issue,” in Proceeding of the ICIP, 1994.

  26. W. Li, S. Gauch, J. Gauch, and K.M. Pua, “Vision: A digital video library,” in ACM Digital Libraries, Bethseda, MD, 1996.

  27. F. Liu and R.W. Picard, “Periodocity, directionality and randomness,” Technical Report TR-320, MIT Media Lab.

  28. T.P. Minka and R.W. Picard, “Interactive learning using a society of models,” Technical Report TR-349, MIT Media Lab.

  29. A. Nagasaka and Y. Tanaka, “Automatic video indexing and full-motion search for object appearances,” in Proc. IFIP TC2/WG2.6 Second Working Conference on Visual Database System, 1991.

  30. H.H. Nagel, “Displacement vectors derived from second-order intensity variations in image sequences,” Computer Vision, Graphics, Image Processing, Vol. 21, 1983.

  31. H.H. Nagel, “On the estimation of optical flow: Relations between different approaches and some new results,” Artificial Intelligence, Vol. 33, 1987.

  32. W. Niblack et al., “The qbic project: Querying images by content using color, texture and shape,” in IS&T/SPIE Symposium on Electronic Imaging: Science and Technology—Storage & Retrieval for Image and Video Databases I, San Jose, CA, 1993.

  33. V.E. Ogle and M. Stonebraker, “Chabot: Retrieval from a relational database of images,” IEEE Computer, Sept. 1995.

  34. E. Oomoto and K. Tanaka, “Ovid: Design and implementation of a video-object database system,” IEEE Transaction on Knowledge and Data Engineering, Vol. 5, Aug. 1994.

  35. A. Pentland, R.W. Picard, and S. Sclaroff, “Photobook: Tools for content-based manipulation of image databases,” in Proc. of SPIE: Storage and Retrieval Image and Video Database II, San Jose, Feb. 6–10, 1994.

  36. R.W. Picard, “Computer learning of subjectivity,” Technical Report TR-359, MIT Media Lab.

  37. R.W. Picard, “Toward a visual thesaurus,” in Proc. of Springer Werlag Workshops in Computing, MIRO, Glasgow, 1995.

    Google Scholar 

  38. R.W. Picard and T. Kabir, “Finding similar patterns in large image databases,” in Proc. of IEEE Int. Conf. on Acoustics, Speech and Signal Processing, ICASSP, Minneapolis, MN, 1993.

    Google Scholar 

  39. R.W. Picard, T. Kabir, and F. Liu, “Real-time recognition with the entire broadatz texture database,” in Proc. of IEEE Int. Conf. on Computer Vision and Pattern Recognition, CVPR, New York, NY. 1993.

    Google Scholar 

  40. R.W. Picard and T.P. Minka, “Vision texture for annotation,” Technical Report TR-302, MIT Media Lab.

  41. M.J.D. Powell, “Restart procedures for the conjugate gradient method,” Mathematical Programming, Vol. 12, pp. 241–254, 1968.

    Google Scholar 

  42. S. Sclaroff, “Deformable prototypes for encoding shape categories in image databases,” Pattern Recognition: Special issue on image databases, 1996, (to appear).

  43. J.R. Smith and S.-F. Chang, “Querying by color regions using the visualseek content-based visual query system,” in Intelligent Multimedia Information Retrieval, M. Maybury (Ed.), 1996.

  44. J.R. Smith and S.-F. Chang, “Tools and techniques for color image retrieval,” in Proc. of IS&T SPIE: Storage and Retrieval Image and Video Database IV, San Jose, CA, 1996.

  45. R.K. Srihari, “Automatic indexing and content-based retrieval of captioned images,” IEEE Computer, Sept. 1995.

  46. H.S. Stone and T.G. Shamoon, “The use of image content to control image retrieval and image processing,” Technical Report, Internal Report, NEC Research Institute, 1995.

  47. M. Stricker and A. Dimai, “Color indexing with weak spatial constraints,” in Proc. of IS&T SPIE: Storage and Retrieval Image and Video Database IV, San Jose, CA, 1996.

  48. M. Stricker and M. Orengo, “Similarity of color images,” in Proc. of IS&T SPIE: Storage and Retrieval Image and Video Database III, San Jose, CA, 1995.

  49. M. Swain and D. Ballard, “Color indexing,” Int. Journal of Computer Vision, Vol. 7, p. 11, 1991.

    Google Scholar 

  50. H. Tamura, S. Mori, and T. Yamawaki, “Textural features corresponding to visual perception,” IEEE Transaction on Systems, Man and Cybernetics, Vol. 8, No. 6, 1978.

  51. W. Wolf, “Key-frame selection by motion analysis,” in Proc. of IEEE Int. Conf. on Acoustics, Speech and Signal Processing, ICASSP, Atlanta, May 7–10, 1996.

    Google Scholar 

  52. B.L. Yeo and B. Liu, “Rapid scene analysis on compressed video,” IEEE Trans. on Circuits and Systems for Video Technology, Vol. 5, Dec. 1995.

  53. M.M. Yeung and B. Liu, “Efficient matching and clustering of video shot,” in Proc. of IEEE Int. Conf. on Image Processing, ICIP, Washington, DC, 1995.

    Google Scholar 

  54. M.M. Yeung, B.L. Yeo, and B. Liu, “Extracting story units from long programs for video browsing and navigation,” in Proc. of Int. Conf. on Multimedia Computing and Systems, 1996.

  55. M.M. Yeung, B.L. Yeo, W. Wolf, and B. Liu, “Video browsing using clustering and scene transitions on compressed sequences,” in Proc. of IS&T/SPIE Multimedia Computing and Networking, 1995.

  56. A. Yoshitaka, S. Kishida, M. Hirakawa, and T. Ichikawa, “Knowledge-assisted content-based retrieval for multimedia databases,” IEEE Multimedia, Vol. 1, 1994.

  57. H.J. Zhang, A. Kankanhalli, and S.W. Smoliar, “Automatic partitioning of full-motion video,” Multimedia Systems, Vol. 1, 1993.

  58. H.J. Zhang, C.Y. Low, S.W. Smoliar, and J.H. Wu, “Video parsing retrieval and browsing: An integrated and content-based solution,” in Proc. ACM Multimedia'95, pp. 15–24.

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, University of Palermo, Palermo, Italy

    Edoardo Ardizzone & Marco La Cascia

Authors
  1. Edoardo Ardizzone
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco La Cascia
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ardizzone, E., Cascia, M.L. Automatic Video Database Indexing and Retrieval. Multimedia Tools and Applications 4, 29–56 (1997). https://doi.org/10.1023/A:1009630331620

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1009630331620

Search

Navigation