Abstract
Video retargeting aims at transforming an existing video in order to display it appropriately on a target device, often in a lower resolution, such as a mobile phone. To preserve a viewer’s experience, it is desired to keep the important regions in their original aspect ratio, i.e., to maintain them distortion-free. Most previous methods are susceptible to geometric distortions due to the anisotropic manipulation of image pixels. In this paper, we propose a novel approach to distortion-free video retargeting by scale-space spatiotemporal saliency tracking. An optimal source cropping window with the target aspect ratio is smoothly tracked over time, and then isotropically resized to the retargeted display. The problem is cast as the task of finding the most spatiotemporally salient cropping window with minimal information loss due to resizing. We conduct the spatiotemporal saliency analysis in scale-space to better account for the effect of resizing. By leveraging integral images, we develop an efficient coarse-to-fine solution that combines exhaustive coarse and gradient-based fine search, which we term scale-space spatiotemporal saliency tracking. Experiments on real-world videos and our user study demonstrate the efficacy of the proposed approach.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Liu, F., Gleicher, M.: Video retargeting: automating pan and scan. In: Proc. ACM international conference on Multimedia, pp. 241–250. ACM, New York (2006)
Wolf, L., Guttmann, M., Cohen-Or, D.: Non-homogeneous content-driven video-retargeting. In: Proceedings IEEE International Conference on Computer Vision (2007)
Setlur, V., Takagi, S., Raskar, R., Gleicher, M., Gooch, B.: Automatic image retargeting. In: Proc. International Conference on Mobile and Ubiquitous Multimedia (2005)
Chen, L.Q., Xie, X., Fan, X., Ma, W.Y., Zhang, H.J., Zhou, H.Q.: A visual attention model for adapting images on small displays. ACM Multimedia Systems Journal 9, 353–364 (2003)
Luis Herranz, J.M.M.: Adapting surneillance video to small displays via object-based cropping. In: Proc. International Workshop on Image Analysis for Multimedia Interactive Services, pp. 72–75 (2007)
Liu, H., Xie, X., Ma, W.Y., Zhang, H.J.: Automatic browsing of large pictures on mobile devices. In: Proc. ACM international conference on Multimedia. ACM, New York (2003)
Rui, Y., Gupta, A., Grudin, J., He, L.: Automating lecture capture and broadcast: technology and videography. ACM Multimedia Systems Journal 10, 3–15 (2004)
Kang, H.W., Matsushita, Y., Tang, X., Chen, X.Q.: Space-time video montage. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 1331–1338 (2006)
Gal, R., Sorkine, O., Cohen-Or, D.: Feature-aware texturing. In: Proceedings of Eurographics Symposium on Rendering, pp. 297–303 (2006)
He, L., Cohen, M.F., Salesin, D.: The virtual cinematographer: A paradigm for automatic real-time camera control and directing. In: Proc. Annual Conference on Computer Graphics (SIGGRAPH), pp. 217–224. ACM, New York (1996)
Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. ACM Transaction on Graphics, Proc. of SIGGRAPH 2007 26, 10 (2007)
Rubinstein, M., Shamir, A., Avidan, S.: Improved seam carving for video retargeting. In: ACM Transaction on Graphics, Proc. of SIGGRAPH 2008 (2008)
Hou, X., Zhang, L.: Saliency detection: A spectral residual approach. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition (2007)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 511–518 (2001)
Roth, S., Black, M.J.: On the spatial statistics of optical flow. In: Proc. IEEE International Conference on Computer Vision, vol. 1, pp. 42–49 (2005)
Guo, C., Ma, Q., Zhang, L.: Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 1–8 (2008)
Hua, G., Zhang, C., Liu, Z., Zhang, Z., Shan, Y.: Efficient scale-space spatiotemporal saliency tracking for distortion-free video retargeting. Technical Report MSR-TR-2009-87, Microsoft Research (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hua, G., Zhang, C., Liu, Z., Zhang, Z., Shan, Y. (2010). Efficient Scale-Space Spatiotemporal Saliency Tracking for Distortion-Free Video Retargeting. In: Zha, H., Taniguchi, Ri., Maybank, S. (eds) Computer Vision – ACCV 2009. ACCV 2009. Lecture Notes in Computer Science, vol 5995. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12304-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-12304-7_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12303-0
Online ISBN: 978-3-642-12304-7
eBook Packages: Computer ScienceComputer Science (R0)