Victor Valdes
Abstract
This article describes a method for the automatic evaluation of video summaries based on the training of individual predictors for different quality measures from the TRECVid 2008 BBC Rushes Summarization Task. The obtained results demonstrate that, with a large set of evaluation data, it is possible to train fully automatic evaluation systems based on visual features automatically extracted from the summaries. The proposed approach will enable faster and easier estimation of the results of newly developed abstraction algorithms and the study of which summary characteristics influence their perceived quality.
- Bai, L., Lao, S., Smeaton, A. F., and O'Connor, N. E. 2008. Automatic summarization of rushes video using bipartite graphs. In Proceedings of the 3rd International Conference on Semantic and Digital Media Technologies (SAMT '08). Springer-Verlag, 3--14. Google Scholar
Digital Library
- Bailer, W. and Thallinger, G. 2008. Comparison of content selection methods for skimming rushes video. In Proceedings of the 2nd ACM TRECVid Video Summarization Workshop (TVS '08). ACM, 85--89. Google ScholarDigital Library
- Bredin, H., Byrne, D., Lee, H., O'Connor, N. E., and Jones, G. J. 2008. Dublin city university at the trecvid 2008 BBC rushes summarisation task. In Proceedings of the 2nd ACM TRECVid Video Summarization Workshop (TVS '08). ACM, 45--49. Google ScholarDigital Library
- Breiman, L., Friedman, J., Stone, C. J., and Olshen, R. A. 1984. Classification and Regression Trees. Chapman and Hall.Google Scholar
- Christel, M. G., Hauptmann, A. G., Lin, W.-H., Chen, M.-Y., Yang, J., Maher, B., and Baron, R. V. 2008. Exploring the utility of fast-forward surrogates for BBC rushes. In Proceedings of the 2nd ACM TRECVid Video Summarization Workshop (TVS '08). ACM, 35--39. Google ScholarDigital Library
- Christel, M. G., Smith, M. A., Taylor, C. R., and Winkler, D. B. 1998. Evolving video skims into useful multimedia abstractions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '98). ACM, 171--178. Google ScholarDigital Library
- Ding, W., Marchionini, G., and Tse, T. 1997. Previewing video data: Browsing key frames at high rates using a video slide show. In Proceedings of the International Symposium on Research, Development and Practice in Digital Libraries (ISDL '97). 151--158.Google Scholar
- Dumont, E. and Merialdo, B. 2007. Split-screen dynamically accelerated video summaries. In Proceedings of the International Workshop on TRECVID Video Summarization (TVS '07). ACM, 55--59. Google ScholarDigital Library
- Dumont, E. and Merialdo, B. 2008a. Automatic evaluation method for rushes summarization: experimentation and analysis. In Proceedings of the 6th International Workshop on Content-Based Multimedia Indexing (CBMI '08). 518--525.Google Scholar
- Dumont, E. and Merialdo, B. 2008b. Sequence alignment for redundancy removal in video rushes summarization. In Proceedings of the 2nd ACM TRECVid Video Summarization Workshop (TVS '08). ACM, 55--59. Google ScholarDigital Library
- Dumont, E. and Merialdo, B. 2010. Rushes video summarization and evaluation. Multimed. Tools Appl. 48, 1, 51--68. Google ScholarDigital Library
- Ekin, A., Tekalp, A., and Mehrotra, R. 2003. Automatic soccer video analysis and summarization. IEEE Trans. Image Process. 12, 7, 796--807. Google ScholarDigital Library
- Ferman, A. and Tekalp, A. 2003. Two-stage hierarchical video summary extraction to match low-level user browsing preferences. IEEE Trans. Multimed. 5, 2, 244--256. Google ScholarDigital Library
- Hauptmann, A. G., Christel, M. G., Lin, W.-H., Maher, B., Yang, J., Baron, R. V., and Xiang, G. 2007. Clever clustering vs. simple speed-up for summarizing rushes. In Proceedings of the International Workshop on TRECVID Video Summarization (TVS '07). ACM, 20--24. Google ScholarDigital Library
- Kasutani, E. and Yamada, A. 2001. The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval. In Proceedings of the International Conference on Image Processing (ICIP '01). 674--677.Google Scholar
- Over, P., Smeaton, A. F., and Awad, G. 2008. The trecvid 2008 BBC rushes summarization evaluation. In Proceedings of the 2nd ACM TRECVid Video Summarization Workshop (TVS '08). ACM, 1--20. Google ScholarDigital Library
- Over, P., Smeaton, A. F., and Kelly, P. 2007. The trecvid 2007 bbc rushes summarization evaluation pilot. In Proceedings of the International Workshop on TRECVID Video Summarization (TVS '07). ACM, 1--15. Google ScholarDigital Library
- Ren, T., Liu, Y., and Wu, G. 2008. Full-reference quality assessment for video summary. In Proceedings of the IEEE International Conference on Data Mining Workshops. IEEE Computer Society, Los Alamitos, CA, 874--883. Google ScholarDigital Library
- Taskiran, C., Pizlo, Z., Amir, A., Ponceleon, D., and Delp, E. 2006. Automated video program summarization using speech transcripts. IEEE Trans. Multimed. 8, 4, 775--791. Google ScholarDigital Library
- Valdes, V. and Martinez, J. M. 2008. Binary tree based on-line video summarization. In Proceedings of the 2nd ACM TRECVid Video Summarization Workshop (TVS '08). ACM, 134--138. Google ScholarDigital Library
- Yamasaki, K., Shinoda, K., and Furui, S. 2008. Automatically estimating number of scenes for rushes summarization. In Proceedings of the 2nd ACM TRECVid Video Summarization Workshop (TVS '08). ACM, 129--133. Google ScholarDigital Library
Index Terms
- Automatic evaluation of video summaries
Recommendations
Discrepancy between automatic and manual evaluation of summaries
NAACL HLT '12: Proceedings of Workshop on Evaluation Metrics and System Comparison for Automatic SummarizationToday, automatic evaluation metrics such as ROUGE have become the de-facto mode of evaluating an automatic summarization system. However, based on the DUC and the TAC evaluation results, (Conroy and Schlesinger, 2008; Dang and Owczarzak, 2008) showed ...
Supervised automatic evaluation for summarization with voted regression model
The high quality evaluation of generated summaries is needed if we are to improve automatic summarization systems. Although human evaluation provides better results than automatic evaluation methods, its cost is huge and it is difficult to reproduce the ...
Analysis of Sentence Scoring Methods for Extractive Automatic Text Summarization
ICTCS '14: Proceedings of the 2014 International Conference on Information and Communication Technology for Competitive StrategiesAutomatic text summarization is a major area of research in the domain of information systems. Most of the methods requires domain knowledge in order to produce a coherent and meaningful summary. In Extractive text summarization, sentences are scored on ...
Comments