Michael Rubinstein
Olga Sorkine
Abstract
The numerous works on media retargeting call for a methodological approach for evaluating retargeting results. We present the first comprehensive perceptual study and analysis of image retargeting. First, we create a benchmark of images and conduct a large scale user study to compare a representative number of state-of-the-art retargeting methods. Second, we present analysis of the users' responses, where we find that humans in general agree on the evaluation of the results and show that some retargeting methods are consistently more favorable than others. Third, we examine whether computational image distance metrics can predict human retargeting perception. We show that current measures used in this context are not necessarily consistent with human rankings, and demonstrate that better results can be achieved using image features that were not previously considered for this task. We also reveal specific qualities in retargeted media that are more important for viewers. The importance of our work lies in promoting better measures to assess and guide retargeting algorithms in the future. The full benchmark we collected, including all images, retargeted results, and the collected user data, are available to the research community for further investigation at http://people.csail.mit.edu/mrub/retargetme.
- Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., and Szeliski, R. 2007. A database and evaluation methodology for optical flow. In Proc. ICCV, vol. 5.Google Scholar
- Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM TOG 28, 3. Google ScholarDigital Library
- Bose, R. C. 1955. Paired comparison designs for testing concordance between judges. Biometrika, 42, 113--121.Google Scholar
- Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A benchmark for 3D mesh segmentation. ACM TOG 28, 3. Google ScholarDigital Library
- Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H., Finkelstein, A., Funkhouser, T., and Rusinkiewicz, S. 2008. Where do people draw lines? ACM TOG 27, 3. Google ScholarDigital Library
- Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., and Singh, M. 2009. How well do line drawings depict shape? ACM TOG 28, 3. Google ScholarDigital Library
- David, H. 1963. The Method of Paired Comparison. Charles Griffin and Company.Google Scholar
- Dong, W., Zhou, N., Paul, J.-C., and Zhang, X. 2009. Optimized image resizing using seam carving and scaling. ACM TOG 28, 5. Google ScholarDigital Library
- van der Helm, P. 2000. Principles of Symmetry Perception. International Congress of Psychology.Google Scholar
- Karni, Z., Freedman, D., and Gotsman, C. 2009. Energy-based image deformation. CGF 28, 5, 1257--1268. Google ScholarDigital Library
- Kasutani, E., and Yamada, A. 2001. The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval. In International Conference on Image Processing, 674--677.Google Scholar
- Kendall, M. G., and Babington-Smith, B. 1940. On the method of paired comparisons. Biometrica 31, 324--345.Google ScholarCross Ref
- Kendall, M. G. 1938. A new measure of rank correlation. Biometrika, 1/2 (June), 81--93.Google Scholar
- Kendall, M. G. 1955. Reviews. Biometrika 42.Google Scholar
- Krähenbühl, P., Lang, M., Hornung, A., and Gross, M. 2009. A system for retargeting of streaming video. ACM TOG 28, 5. Google ScholarDigital Library
- Liu, F., and Gleicher, M. 2006. Video retargeting: automating pan and scan. In MULTIMEDIA, ACM, 241--250. Google ScholarDigital Library
- Liu, C., Yuen, J., Torralba, A., Sivic, J., and Freeman, W. T. 2008. SIFT Flow: Dense correspondence across different scenes. In ECCV, 28--42. Google ScholarDigital Library
- Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91--110. Google ScholarDigital Library
- Manjunath, B. S., Ohm, J. R., Vasudevan, V. V., and Yamada, A. 2001. Color and texture descriptors. IEEE Trans. Circuits and Systems for Video Technology 11, 703--715. Google ScholarDigital Library
- Martin, D., Fowlkes, C., Tal, D., and Malik, J. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings ICCV, vol. 2.Google Scholar
- MPEG-7. 2002. ISO/IEC 15938: Multimedia Content Description Interface.Google Scholar
- Pearson, E. S., and Hartley, H. O. 1966. Biometrika Tables for Statisticians, 3rd ed., vol. 1. Cambridge University Press.Google Scholar
- Pele, O., and Werman, M. 2009. Fast and robust earth mover's distances. In ICCV '09.Google Scholar
- Pritch, Y., Kav-Venaki, E., and Peleg, S. 2009. Shift-map image editing. In ICCV, 151--158.Google Scholar
- Rubinstein, M., Shamir, A., and Avidan, S. 2008. Improved seam carving for video retargeting. ACM TOG 27, 3. Google ScholarDigital Library
- Rubinstein, M., Shamir, A., and Avidan, S. 2009. Multioperator media retargeting. ACM Trans. Graph. 28, 3. Google ScholarDigital Library
- Setyawan, I., and Lagendijk, R. L. 2004. Human perception of geometric distortions in images. In Proceedings of SPIE, Security, Steganography and Watermarking of Multimedia Contents VI, 256--267.Google Scholar
- Shamir, A., and Sorkine, O. 2009. Visual media retargeting. In ACM SIGGRAPH Asia Courses. Google ScholarDigital Library
- Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. 2008. Summarizing visual data using bidirectional similarity. In CVPR, 1--8.Google Scholar
- Tyler, C. W., Ed. 1996. Human Symmetry Perception and its Computational Analysis. VSP International Science Publishers, Utrecht.Google Scholar
- Wang, Y.-S., Tai, C.-L., Sorkine, O., and Lee, T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM TOG 27, 5. Google ScholarDigital Library
- Wolf, L., Guttmann, M., and Cohen-Or, D. 2007. Non-homogeneous content-driven video-retargeting. In ICCV.Google Scholar
Recommendations
Visual media retargeting
SIGGRAPH ASIA '09: ACM SIGGRAPH ASIA 2009 CoursesThe increasing variety of commonly used display devices, especially mobile devices, requires adapting visual media to different resolutions and aspect ratios - a process called "retargeting." The media retargeting problem is further accentuated by the ...
A comparative study of image retargeting
SIGGRAPH ASIA '10: ACM SIGGRAPH Asia 2010 papersThe numerous works on media retargeting call for a methodological approach for evaluating retargeting results. We present the first comprehensive perceptual study and analysis of image retargeting. First, we create a benchmark of images and conduct a ...
Using eye-tracking to assess different image retargeting methods
APGV '11: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and VisualizationAssessing media retargeting results is not a trivial issue. When resizing one image to a particular percentage of its original size, some content has to be removed, which may affect the image's original meaning and/or composition. We examine the impact ...
Comments