Romulus Grigoras
Vincent Charvillat
ABSTRACT
Interaction with hypermedia documents is a required feature for new sophisticated yet flexible multimedia applications. This paper presents an innovative adaptive technique to stream hypervideo that takes into account user behaviour. The objective is to optimize hypervideo prefetching in order to reduce the latency caused by the network. This technique is based on a model provided by a Markov Decision Process approach. The problem is solved using two methods: classical stochastic dynamic programming algorithms and reinforcement learning. Experimental results under stochastic network conditions are very promising.
- MPEG-21, MPEG Group, Multimedia Framework, http://www.cselt.it/mpegGoogle Scholar
- B. Trousse, Evaluation of the Prediction Capability of a User behaviour Mining Approach for Adapative Web Sites, Proc. RIAO 2000, 6th Conference on Content-Based Multimedia Information Access, Paris, France, 2000.Google Scholar
- Apple, QuickTimeVR, http://quicktime.apple.comGoogle Scholar
- N. Sawhney, D. Balcom, I. Smith, Authoring and navigating video in space and time: A Framework and Approach towards Hypervideo, IEEE Multimedia, 4, 4, 1997 http://www.lcc.gatech.edu/gallery/hypercafe Google ScholarDigital Library
- J. Tolva, MediaLoom, An Interactive Authoring Tool for HyperVideo, M.S. Project Paper, Georgia Institute of Technology, 1998.Google Scholar
- I. Zukerman, D. Albrecht, Predictive Statistical Models for User Modeling User Modeling and User-Adapted Interaction, 2001. Google ScholarDigital Library
- M. Crovella, P. Barford, The Network Effects of Prefetching, Proc. of Infocom'98, IEEE, April 1998.Google Scholar
- Y. Aumann et al., Predicting Event Sequences: Data Mining for Prefetching Web-pages, Proc. of KDD'98.Google Scholar
- K. Aberer, S. Hollfelder, Resource Prediction and Admission Control for Interactive Video Browsing Scenarios Using Application Semantics, GMD Report No. 40, September 1998. Google ScholarDigital Library
- D. Albrecht, I. Zukerman, A. Nicholson, Pre-sending Documents on the WWW: A comparative Study, Proc. IJCAI'99, August 1999. Google ScholarDigital Library
- T. Syeda-Mahmood, Learning and Tracking Browsing Behavior of Users using Hidden Markov Models, IBM Make It Easy Conference, 2001.Google Scholar
- T. Syeda-Mahmood, D. Ponceleon, Learning video browsing behavior and its application in the generation of video previews, Proc. ACM Multimedia 20001. Google ScholarDigital Library
- A. Kohrs, B. Merialdo, Creating user-adapted Websites by the use of collaborative filtering, Interacting with Computers, Volume 13, Issue 6, August 2001.Google Scholar
- R. Grigoraş, V. Charvillat, M. Douze, Self-adaptive multimedia content, Proc. ECMCS, September 2001.Google Scholar
- R. E. Bellman, Dynamic programming, Princeton University Press, 1957. Google ScholarDigital Library
- D. P. Bertsekas, J. N. Tsitsikis, Neurodynamic programming, Athena Scientific, Belmond, Massachusetts, 1996. Google ScholarDigital Library
- M. L. Puterman, Markov decision processes: discrete stochastic dynamic programming, Wiley-Interscience, New York, 1994. Google ScholarDigital Library
- R. S. Sutton, A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, Massachusetts, 1998. Google ScholarDigital Library
Index Terms
- Optimizing hypervideo navigation using a Markov decision process approach
Recommendations
Approximate stochastic annealing for online control of infinite horizon Markov decision processes
We present an online simulation-based algorithm called Approximate Stochastic Annealing (ASA) for solving infinite-horizon finite state-action space Markov decision processes (MDPs). The algorithm estimates the optimal policy by sampling at each ...
Solving Semi-Markov Decision Problems Using Average Reward Reinforcement Learning
<P>A large class of problems of sequential decision making under uncertainty, of which the underlying probability structure is a Markov process, can be modeled as stochastic dynamic programs referred to, in general, as Markov decision problems or MDPs. ...
When are the value iteration maximizers close to an optimal stationary policy of a discounted Markov decision process?: closing the gap between the Borel space theory and actual computations
Markov Decision Processes [MDPs] have been repeteadly used in Economy and Engineering but apparently are still far from achieving its full potential due to the computational difficulties inherent to the subject due to the usual impossibility of finding ...
Comments