Yong Wei
Abstract
The design and implementation of a client-centered multimedia content adaptation system suitable for a mobile environment comprising of resource-constrained handheld devices or clients is described. The primary contributions of this work are: (1) the overall architecture of the client-centered content adaptation system, (2) a data-driven multi-level Hidden Markov model (HMM)-based approach to perform both video segmentation and video indexing in a single pass, and (3) the formulation and implementation of a Multiple-choice Multidimensional Knapsack Problem (MMKP)-based video personalization strategy. In order to segment and index video data, a video stream is modeled at both the semantic unit level and video program level. These models are learned entirely from training data and no domain-dependent knowledge about the structure of video programs is used. This makes the system capable of handling various kinds of videos without having to manually redefine the program model. The proposed MMKP-based personalization strategy is shown to include more relevant video content in response to the client's request than the existing 0/1 knapsack problem and fractional knapsack problem-based strategies, and is capable of satisfying multiple client-side constraints simultaneously. Experimental results on CNN news videos and Major League Soccer (MLS) videos are presented and analyzed.
- Akbar, M. D., Manning, E. G., Shoja, G. C., and Khan, S. 2001. Heuristic solutions for the multiple-choice multidimension knapsack problem. In Proceedings of the International Conference on Computational Science, 659--668. Google Scholar
Digital Library
- Bartoli, A., Dalal, N. and Horaud, R. 2004. Motion panoramas. Comput. Anim. Virtual Worlds, 15, 501--517. Google ScholarDigital Library
- Baum, L. E., Peterie, T., Souled, G., and Weiss, N. 1970. A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Statist, 164--171.Google Scholar
- Bhandarkar, S. M., Warke, Y. S., Khombhadia, A. A. 1999. Integrated parsing of compressed video. Lecture Notes In Computer Science, vol. 1614, 269--276. Google ScholarDigital Library
- Boreczky, J. S. and Wilcox. L. D. 1998. A hidden Markov model framework for video segmentation using audio and image features. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).Google ScholarCross Ref
- Brown, P. F., Pietra, V. J., DeSouza, P. V., Lai, J. C., and Mercer, R. L. 1992. Class-based n-gram models of natural language. Comput. Linguist. 18, 4, 467--479. Google ScholarDigital Library
- Chen, M. J., Chu, M. C., and Pan, C. W. 2002. Efficient motion estimation algorithm for reduced frame-rate video transcoder. IEEE Trans. Circ. Syst. Video Technol. 12, 4, 269--275. Google ScholarDigital Library
- Eickeler, S. and Müller, S. 1999. Content-based video indexing of TV broadcast news using hidden Markov models. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2997--3000. Google ScholarDigital Library
- Eickeler, S. and Rigoll, G. 2000. A novel error measure for the evaluation of video indexing systems. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 4, 1991--1994. Google ScholarDigital Library
- Eleftheriadis, A. and Batra, P. 2006. Dynamic rate shaping of compressed digital video. IEEE Trans. Multimedia 8, 2, 297--314. Google ScholarDigital Library
- Fellbaum, C., Ed. 1998. WordNet—An Electronic Lexical Database. The MIT Press, Cambridge, MA.Google Scholar
- Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., and Yanker, P. 1995. Query by image and video content: The QBIC system. IEEE Comput. Mag. 23--32. Google ScholarDigital Library
- Forney, G. D. 1973. The Viterbi algorithm. Proceedings of the IEEE, vol. 61, No. 3, 268-278.Google ScholarCross Ref
- Hernandez, R. P. and Nikitas, N. J. 2005. A new heuristic for solving the multiple-choice multidimensional knapsack problem. IEEE Trans. Syst. Man Cybernetics, Part A 35, 5, 708--717. Google ScholarDigital Library
- Huang, J., Liu, Z., and Wang, Y. 2005. Joint scene classification and segmentation based on hidden Markov model. IEEE Trans. Multimedia 7, 3, 538--550. Google ScholarDigital Library
- Irani, M., Hsu, S., and Anandan, P. 1995. Mosaic-based video compression. In Proceedings of the SPIE Conference on Electronic Imaging, Digital Video Compression: Algorithms and Technologies, vol. 2419, 242--253.Google ScholarCross Ref
- Irani, M., Anandan, P., Bergen, J., Kumar, R., and Hsu, S. 1996. Efficient representations of video sequences and their applications. Signal Process. Image Commun. Special Issue on Image Video Semantics: Processing, Analysis, Appl. 8, 4, 327--351.Google Scholar
- Khan, S. 1998. Quality adaptation in a multi-session adaptive multimedia system: Model and architecture. Ph.D. thesis, Department of Electronical and Computer Engineering, University of Victoria. Google ScholarDigital Library
- Leacock, C. and Chodorow, M. 1998. Combining local context and wordnet similarity for word sense identification. In WordNet: An Electronic Lexical Database, Fellbaum C. (Ed.), MIT Press, Cambridge, MA, 265--283.Google Scholar
- Li, B. and Sezan, M. I. 2001. Event detection and summarization in sports video. In Proceedings of the IEEE Workshop on Content-Based Access of Image and Video Libraries 8, 132--138. Google ScholarDigital Library
- Li, C. S., Mohan, R., and Smith, J. R. 1998. Multimedia content description in the Info-Pyramid. In Proceedings of the ICASSP'98, Special Session on Signal Processing in Modern Multimedia Standards, vol.6, 3789--3792.Google Scholar
- Merialdo, B., Lee, K.T., Luparello, D., and Roudaire, J. 1999. Automatic construction of personalized TV news programs. In Proceedings of the ACM Conference on Multimedia, 323--331. Google ScholarDigital Library
- Nakajima, Y., Hori, H., and Kanoh, T. 1995. Rate conversion of MPEG coded video by requantization process. In Proceedings of the IEEE International Conference on Image Processing, 408--411. Google ScholarDigital Library
- Ney, H. and Ortmanns, S. 1999. Progress on dynamic programming search for continuous speech recognition. IEEE Signal Proc. Mag. 16, 5, 64--83.Google ScholarCross Ref
- Papoulis, A. 1984. Probability, Random Variables, and Stochastic Processes, 2nd Ed. McGraw-Hill, New York, 104, 148.Google Scholar
- Rabiner, L. R. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. In Proceedings of the IEEE 77, 2, 257--286.Google ScholarCross Ref
- Shinoda, K., Bach, N. H., Furui, S., and Kawai, N. 2005. Scene recognition using hidden Markov models for video database. In Proceedings of the Symposium on Large-Scale Knowledge Resources (LKR'05), 107--110.Google Scholar
- Snoek, C. G. M. and Worring, M. 2003. Time interval maximum entropy based event indexing in soccer video. In Proceedings of the IEEE International Conference on Multimedia & Expo, vol. 3, 481--484. Google ScholarDigital Library
- Sun, H., Kwok, W., and Zdepski, J. 1996. Architectures for MPEG compressed bitstream scaling. IEEE Trans. Circ. Syst. Video Technol. 6, 191--199. Google ScholarDigital Library
- Tamura, H., Mori, S., and Yamawaki, T. 1978. Textural features corresponding to visual perception. IEEE Trans. Syst. Man Cybernetics 8, 460--472.Google ScholarCross Ref
- Tseng, B. L., Lin, C. Y., and Smith, J. R. 2004. Using MPEG-7 and MPEG-21 for personalizing video. IEEE Multimedia, 11, 1, 42--52. Google ScholarDigital Library
- Tseng, B. L. and Smith, J. R. 2003. Hierarchical video summarization based on context clustering. In Proceedings of the SPIE, 5242, 14--25.Google ScholarCross Ref
- Tseng, B. L., Lin, C. Y., and Smith, J. R. 2002. Video personalization and summarization system. In Proceedings of the IEEE Workshop on Multimedia Signal Processing, 424--427.Google Scholar
- Uykan, Z. and Koivo, H. N. 2000. Unsupervised learning of sigmoid perceptron. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 6, 3486--3489. Google ScholarDigital Library
- Vanderbei, R. J. 1997. Linear Programming: Foundations and Extensions. Kluwer Academic, Norwell, MA.Google Scholar
- Viola, P. and Jones, M. J. 2004. Robust real-time face detection. Int. J. Comput. Vision 57, 2, 137--154. Google ScholarDigital Library
- Wei, Y., Wang, H., Bhandarkar, S. M., and Li, K. 2006. Parallel algorithms for motion panorama construction. In Proceedings of the ICPP Workshop on Parallel and Distributed Multimedia, 82--92. Google ScholarDigital Library
- Wheeler, E. S. 2002. Zipf's law and why it works everywhere. Glottometrics, 4, 45--48.Google Scholar
- Zhu, W., Yang, K., and Beacken, M. 1998. CIF-to-QCIF video bitstream down conversion in the DCT domain. Bell Labs Tech. J. 3, 3, 21--29.Google ScholarCross Ref
Index Terms
- Client-centered multimedia content adaptation
Recommendations
Video personalization in resource-constrained multimedia environments
MM '07: Proceedings of the 15th ACM international conference on MultimediaMultimedia data, especially video data, is being increasingly transmitted to, transmitted from and viewed on mobile devices such as PDA's, laptop PCs, pocket PCs and cell phones. One of the natural limitations of these multimedia-capable, mobile devices ...
Video personalization in heterogeneous and resource-constrained environments
Access to multimedia data and multimedia services is becoming increasingly common in networked mobile environments. In such environments, both the mobile client devices and multimedia servers are typically resource constrained. Moreover, the mobile ...
A survey of multimedia content adaptation for mobile devices
With continued increase in the use of smartphones, user expectations of content access have also increased. Most of the content that exists today is not designed for mobile devices. Mobile devices cannot directly access most of the content due to the ...
Comments