Abstract
This chapter details the value and methods for content augmentation and personalization among different media such as TV and Web. We illustrate how metadata extraction can aid in combining different media to produce a novel content consumption and interaction experience. We present two pilot content augmentation applications. The first, called MyInfo, combines automatically segmented and summarized TV news with information extracted from Web sources. Our news summarization and metadata extraction process employs text summarization, anchor detection and visual key element selection. Enhanced metadata allows matching against the user profile for personalization. Our second pilot application, called InfoSip, performs person identification and scene annotation based on actor presence. Person identification relies on visual, audio, text analysis and talking face detection. The InfoSip application links person identity information with filmographies and biographies extracted from the Web, improving the TV viewing experience by allowing users to easily query their TVs for information about actors in the current scene.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ahanger, G. and Little, T. D. C.: 1997, A System for Customized News Delivery from Video Archives, In: Proceedings of ICMCS’97, (June 3–6) IEEE Press.
Ardissono, L., Portis, F. and Torasso, P.: 2001, Architecture of a System for the Generation of Personalized Electronic Program Guides, Eighth International Conference on User Modeling: Workshop on Personalization in Future TV. Sonthofen, Germany.
Blum, D. W.: 1992, Method and Apparatus for Identifying and Eliminating Specific Material from Video Signals, US patent 5, 151, 788, September.
Boguraev, B. and Neff, M.: 2000, Lexical Cohesion, Discourse Segmentation, and Document Summarization, Proc. RIAO International Conference. April, Paris.
Bonner, E. L. and Faerber, N. A.: 1982, Editing System for Video Apparatus, US patent 4,314,285, February.
Boykin, S. and Merlino, A.: 1999, Improving Broadcast News Segmentation Processing, IEEE International Conference on Multimedia and Computing Systems. Florence, Italy, 7–11 June.
Boykin, S. and Merlino, A.: 2000, Machine Learning of Event Segmentation for News on Demand, Communications of the ACM 43(2), 35–41.
Brown, M. G., Foote, J. T., Jones, G. J. F., Jones, S. K. and Young, S. J.: 1995, Automatic Content-Based Retrieval of Broadcast News, In: Proceedings of ACM Multimedia 95. San Francisco, CA: ACM Press, pp. 35–43.
Brown, E. W. and Coden, A. R.: 2002, Capitalization Recovery for Text, In: A. R. Coden, E. W. Brown, and S. Srinivasan, (eds.): Information Retrieval Techniques for Speech Applications. Springer, pp. 11–22.
Brusilovsky, P.: 2003, Adaptive Navigation Support in Educational Hypermedia: The Role of Student Knowledge Level and the Case for Meta-Adaptation. British Journal of Educational Technology 34(4), 487–497.
Chen, L. and Faudemay, P.: 1997, Multi-Criteria Video Segmentation for TV News, In: Proceedings of IEEE First Workshop on Multimedia Signal Processing. Princeton, NJ.
Connell, J.: 2002, Face Finding, http://www.research.ibm.com/ecvg/jhc_proj/faces.html..
Cotter, P. and Smyth, B.: 2000, PTV: Intelligent Personalized TV Guides, Seventeenth National Conference on Artificial Intelligence. Austin, TX, USA, pp. 957–964.
Cardie, C.: 1997, Empirical Methods in Information Extraction, AI Magazine 18(4), 65–79.
Dakss, J., Agamanolis, S., Chalom, E., Bove, V. M., Brooks, K., Nemirovsky, P. and Westner, A.: Hyper Soap: http://www.media.mit.edu/hypersoap..
Das, D. and ter Horst H.: 1998, Recommender Systems for TV. Technical Report WS-98-08 Recommender Systems, Papers from the 1998 Workshop, Madison, WI. Menlo Park, CA: AAAI Press, pp. 35–36.
Dimitrova, N., Martino, J., Agnihotri, L. and Elenbaas, H.: 1999, Superhistograms for Video Representation, IEEE ICIP. Kobe, Japan.
Dimitrova, N., Agnihotri, L. and Jasinschi, R.: 2003, Temporal video boundaries, In: Video A. Rosenfeld, D. Doermann, and D. Dementhon (eds.): Mining Book. Kluwer, pp. 61–90.
Elenbaas, H., Dimitrova, N. and McGee, T.: 1999, PNRS-Personalized News Retrieval System, SPIE Multimedia Storage and Archiving Systems.
Haas, N., Bolle, R., Dimitrova, N., Janevski, A. and Zimmerman, J.: 2002, Personalized News Through Content Augmentation and Profiling, In: Proceedings of International Conference on Image Processing 2002. Rochester, NY: IEEE Press, September 22–25.
Hampapur, A., Jain, R. and Weymouth, T.: 1994, Digital Video Segmentation, In: Proceedings of the ACM International Conference on Multimedia. San Francisco, pp. 357–364.
Hanjalic, A., Lagendijk, R. L. and Biemond, J.: 1999, Semiautomatic News Analysis, Indexing and Classification System Based on Topic Preselection, SPIE Storage and Retrieval for Image and Video Databases VII 3656, January pp. 86–97.
IBM Intelligent Miner for Text™
Janevski, A. and Dimitrova, N.: Web Information Extraction for Content Augmentation, In: Proceedings of ICME’ 02. Lausanne, Switzerland: IEEE Press, August 26–29.
Janevski, A.: UniveristyIE: Extracting Information from University Web Pages, MS Thesis, University of Kentucky, Lexington.
Jasinschi, R., Dimitrova, N., McGee, T., Agnihotri, L. and Zimmerman, J.: 2001, Video Scouting: An Architecture and System for the Integration of Multimedia Information in Personal TV Applications, IEEE Conference on Acoustics, Speech, and Signal Processing (ICASSP). Salt Lake City, UT, USA, May 7–11, pp. 1405–1408.
Jiang, H. and Elmagarmid, A. K.: 1998, Spatial and Temporal Content-Based Access to Hypervideo Databases, VLDB Journal, 7(4), 226–238.
Li, D., Wei, G., Sethi, I. K. and Dimitrova, N.: 2001, Person Identification in TV Shows, Journal on Electronic Imaging, Special Issue on Storage, Processing and Retrieval of Digital Media, October.
Li, D., Dimitrova, N., Li, M. and Sethi, I. K.: 2003, Multimedia Content Processing Through Cross-modality Association, ACM Multimedia. November 2–5, Berkeley.
Kubey, R. and Csikszentmihaly, M.: 1990, Television and the Quality of Life: How Viewing Shapes Everyday Experiences, Lawrence Erlbaum Associates. Hillsdale NJ, USA.
Maybury, M. (ed.): February 2000, News On Demand, CACM 43(2): 33–34, 35–79.
Mani, I., House, D. et al.: 1998, Tipster SUMMAC Text Summarization Evaluation, Final Report, October 1998. Mitre Technical Report MTR W980000138 and Technical report, DARPA.
McGee, T. and Dimitrova, N.: 1999, Parsing TV Program Structures for Identification and Removal of Non-story Segments, SPIE Conference on Storage and Retrieval for Image and Video Databases VII (ei24).
Merlino, A., Morey, D. and Maybury, M.: 1997, Broadcast Navigation Using Story segmentation, In: Proceedings of ACM MM’ 97. Seattle, WA: ACM Press, November, pp. 381–388.
ABC Enhanced TV: http://heavy.etv.go.com/etvHome/..
Microsoft NAB demo of enhanced TV: http://www.microsoft.com/presspass/exec/craig/nab97.asp
Microsoft/CBS interactive TV: http://www.microsoft.com/presspass/press/2000/Sept00/CBSpr.asp
Naphade, M. R., Kozintsev, I. and Huang, T. S.: 2002, A Factor Graph Framework for Semantic Video Indexing, IEEE Transactions on Circuits and Systems for Video Technology 12(1), 40–52.
Zimmerman, J., Marmaropoulos, G. and van Heerden, C.: 2001, Interface Design of Video Scout: A Selection, Recording, and Segmentation System for TVs, In: Proceedings of Human Computer Interaction International (HCII) 1, New Orleans, LA, USA, August 5–10, pp. 277–281.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2004 Kluwer Academic Publishers
About this chapter
Cite this chapter
Dimitrova, N. et al. (2004). Media Augmentation and Personalization Through Multimedia Processing and Information Extraction. In: Personalized Digital Television. Human-Computer Interaction Series, vol 6. Springer, Dordrecht. https://doi.org/10.1007/1-4020-2164-X_8
Download citation
DOI: https://doi.org/10.1007/1-4020-2164-X_8
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-2163-3
Online ISBN: 978-1-4020-2164-0
eBook Packages: Springer Book Archive