Audio-guided audiovisual data segmentation, indexing, and retrieval

Tong Zhang; C.-C. Jay Kuo

doi:10.1117/12.333851

17 December 1998 Audio-guided audiovisual data segmentation, indexing, and retrieval

Tong Zhang, C.-C. Jay Kuo

Proceedings Volume 3656, Storage and Retrieval for Image and Video Databases VII; (1998) https://doi.org/10.1117/12.333851
Event: Electronic Imaging '99, 1999, San Jose, CA, United States

Abstract

While current approaches for video segmentation and indexing are mostly focused on visual information, audio signals may actually play a primary role in video content parsing. In this paper, we present an approach for automatic segmentation, indexing, and retrieval of audiovisual data, based on audio content analysis. The accompanying audio signal of audiovisual data is first segmented and classified into basic types, i.e., speech, music, environmental sound, and silence. This coarse-level segmentation and indexing step is based upon morphological and statistical analysis of several short-term features of the audio signals. Then, environmental sounds are classified into finer classes, such as applause, explosions, bird sounds, etc. This fine-level classification and indexing step is based upon time- frequency analysis of audio signals and the use of the hidden Markov model as the classifier. On top of this archiving scheme, an audiovisual data retrieval system is proposed. Experimental results show that the proposed approach has an accuracy rate higher than 90 percent for the coarse-level classification, and higher than 85 percent for the fine-level classification. Examples of audiovisual data segmentation and retrieval are also provided.

Citation Download Citation

Tong Zhang and C.-C. Jay Kuo "Audio-guided audiovisual data segmentation, indexing, and retrieval", Proc. SPIE 3656, Storage and Retrieval for Image and Video Databases VII, (17 December 1998); https://doi.org/10.1117/12.333851

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available