Abstract
Typical result of an automatic shot change detection algorithm expectedly includes a certain number of undetected shots as well as falsely detected shots. Even though automatic shot change detection algorithms are continuing to improve, the ultimate goal of automatically detecting all shot changes without false alarms may never be achieved. Thus, allowing a human operator to intervene—to review and verify the result of a shot change detection algorithm, to delete falsely detected shots as well as to find undetected shots—may be the most viable approach currently available for increasing the accuracy of the overall shot detection process. For this exact purpose, we propose a shot verifier based on the visual rhythm.
The visual rhythm, an abstraction of the video, is a single image, a sub-sampled version of a full video in which the sampling is performed in a pre-determined and in a systematic fashion. It is a representation of the video, which includes the overall content of the video. But most importantly, the visual rhythm contains patterns or visual features that allow the viewer/operator to distinguish and classify many different types of video effects (edits and otherwise): cuts, wipes, dissolves, fades, camera motions, object motions, flashlights, zooms, etc. The different video effects manifest themselves as different patterns on the visual rhythm. Using the visual rhythm, it becomes possible, without sequentially playing the entire video, to find false positive shots as well as undetected shots. Thus, inclusion of the visual rhythm in the shot verification process will aid the operator to verify detected shots as well as to find undetected shots fast and efficiently.
Our newly developed shot verifier based on the visual rhythm has been designed for operator efficiency. The design of our shot verifier presented and the usefulness of the visual rhythm during the shot verification process will be demonstrated.
Similar content being viewed by others
References
J.S. Boreczky and L.A. Rowe, “Comparison of video boundary detection techniques,” in Proc. of SPIE Storage and Retrieval for Image and Video Database IV, SPIE, 1996, Vol. 2670, pp. 170–179.
G. Davenport, T.A. Smith, and N. Pincever, “Cinematic primitives for multimedia,” IEEE Computer Graphics and Applications, pp. 67–74, July 1991.
A. Hampapur, R. Jain, and T. Weymouth, “Digital video segmentation,” in Proc. of ACM Multimedia '94, 1994, pp. 357–564.
H. Kim, S.-J. Park, J. Lee, W.M. Kim, and S.M. Song, “Processing of partial video data for detection of wipe,” in Proc. of SPIE Storage and Retrieval for Image and Video Databases VII, SPIE, 1999, Vol. 3656, pp. 280–289.
J. Meng, Y. Juan, and S. Chang, “Scene change detection in a MPEG compressed video sequence,” in Proc. of SPIE Digital Video Compression: Algorithms and Technologies, SPIE, 1995, Vol. 2419, pp. 14–25.
Y. Nakajima, K. Ujihara, and A. Yoneyama, “Universal scene change detection on MPEG-coded data domain,” in Proc. of SPIE Visual Communication and Image Processing, SPIE, 1997, Vol. 3024, pp. 992–1003.
J. Song and B.-L. Yeo, “Spatially reduced image extraction from MPEG-2 video: fast algorithms and application,” in Proc. of SPIE Storage and Retrieval for Image and Video Database VI, SPIE, 1998, Vol. 3312, pp. 93–107.
S.M. Song, T.-H. Kwon, W.M. Kim, H. Kim, and B.-D. Rhee, “On detection of gradual scene changes for parsing of video data,” in Proc. of SPIE Storage and Retrieval for Image and Video Database VI, SPIE, 1998, Vol. 3312, pp. 404–413.
B.-L. Yeo and B. Liu “Rapid scene analysis on compressed video,” IEEE Transactions on Circuits and Systems for Video Technology, Vol. 5, No. 6, pp. 533–544, 1995.
M.M. Yeung, B.-L. Yeo, W. Wolf, and B. Liu, “Video browsing using clustering and scene transitions on compressed sequences,” in Proc. of SPIE Multimedia Computing and Networking, SPIE, 1995, Vol. 2417, pp. 399–413.
H. Zhang, K. Kankanhalli, and S. Smoliar, “Automatic partitioning of full-motion video,” ACM Multimedia Systems, Vol. 1, No. 1, pp. 10–28, 1993.
H. Zhang and S.W. Smoliar, “Developing power tools for video indexing and retrieval,” in Proc. of SPIE Storage and Retrieval for Image and Video Database II, SPIE, 1994, Vol. 2185, pp. 140–149.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kim, H., Lee, J., Yang, JH. et al. Visual Rhythm and Shot Verification. Multimedia Tools and Applications 15, 227–245 (2001). https://doi.org/10.1023/A:1012452131892
Issue Date:
DOI: https://doi.org/10.1023/A:1012452131892