Abstract
This paper presents a new objective retargeting quality assessment method based on the integration of global and local features by a Gaussian regression model. In the proposed method, first, a graph-based visual saliency algorithm and deep learning model are employed to extract an importance map of the input image. At the same time, the SIFT-Flow method is used for estimating the displacement of different pixels in the retargeted image. Then, the importance map and the displacement results are used for computing four features, called local aspect ratio similarity, local geometric distortion, global geometric distortion, and global preserved information, from the images. Finally, a Gaussian process regression model is employed to integrate these features and compute the final criterion for retargeting quality assessment. The proposed method was tested on the images of the MIT-RetargetMe and CUHK datasets and the results demonstrated its excellent performance compared to state-of-the-art methods.
Similar content being viewed by others
Data availability
To evaluate the proposed method, we have used two popular databases those links are below:MIT RetargetMe:https://people.csail.mit.edu/mrub/retargetme/CUHK:https://ivp.ee.cuhk.edu.hk/projects/demo/retargeting/database.html
References
Rubinstein, M., Shamir, A., Avidan, S.: Improved seam carving for video retargeting. ACM Trans. Graph. 27 (2008)
Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. SIGGRAPH 26 (2007)
Pritch, Y., Kav-Venaki, E., Peleg, S.: Shift-map image editing 151–158 (2009)
Wolf, L., Guttmann, M., Cohen-Or, D.: Non-homogeneous content-driven video-retargeting. Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on 1, 1–6 (2007)
Krähenbühl, P., Lang, M., Sorkine-Hornung, A., Gross, M.: A system for retargeting of streaming video. ACM Trans. Graph. 28 (2009)
Wang, Y.-S., Tai, C.-L., Sorkine, O., Lee, T.-Y.: Optimized scale-and-stretch for image resizing. ACM Trans. Graph. (TOG) 27, 118 (2008)
Rubinstein, M., Shamir, A., Avidan, S.: Multi-operator media retargeting. ACM Trans. Graph. 28, 1–11 (2009)
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)
Simakov, D., Caspi, Y., Shechtman, E., Irani, M.: Summarizing visual data using bidirectional similarity. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition 1 – 8 (2008)
Rubinstein, M., Gutiérrez, D., Sorkine, O., Shamir, A.: A comparative study of image retargeting. ACM Trans. Graph. 29, 160 (2010)
Kasutani, E., Yamada, A.: The mpeg-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval 1, 674 – 677 vol.1 (2001)
Manjunath, B., Ohm, J., Vasudevan, V., Yamada, A.: Color and texture descriptors. IEEE Trans. Circuits Syst. Video Technol. 11, 703–715 (2001)
Liu, C., Yuen, J., Torralba, A.: Sift flow: Dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33, 978–94 (2011)
Boltz, S., Nielsen, F., Soatto, S.: Earth mover distance on superpixels. Proceedings / ICIP ... International Conference on Image Processing 4597 – 4600 (2010)
Hsu, C.-C., Lin, C.-W., Fang, Y., Lin, W.: Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE J. Sel. Topics Signal Process. 8, 377–389 (2014)
Fang, Y., et al.: Objective quality assessment for image retargeting based on structural similarity. IEEE J. Emerg. Sel. Topics Circuits Syst. 4, 95–105 (2014)
Zhang, Y., Fang, Y., Lin, W., Zhang, X., Li, L.: Backward registration based aspect ratio similarity (ars) for image retargeting quality assessment. IEEE Trans. Image Process. 25, 1–1 (2016)
Liang, Y., Liu, Y.-J., Gutiérrez, D.: Objective quality prediction of image retargeting algorithms. IEEE Trans. Visual. Comput. Graph. 23 (2016)
Zhang, Y., Lin, W., Li, Q., Cheng, W., Zhang, X.: Multiple-level feature based measure for retargeted image quality. IEEE Trans. Image Process. PP, 1–1 (2017)
Fu, Z., Shao, F., Qiuping, J., Fua, R., Ho, Y.-S.: Quality assessment of retargeted images using hand-crafted and deep-learned features. IEEE Access PP, 1–1 (2018)
Zhang, Y., Ngan, K., Ma, L., Li, H.: Objective quality assessment of image retargeting by incorporating fidelity measures and inconsistency detection. IEEE Trans. Image Process. PP, 1–1 (2017)
Shao, F., Fu, Z., Qiuping, J., Jiang, G., Ho, Y.-S.: Transformation-aware similarity measurement for image retargeting quality assessment via bidirectional rewarping. IEEE Trans. Syst. Man Cybern. Syst. PP, 1–15 (2019)
Absetan, A., Fathi, A.: Integration of deep learned and handcrafted features for image retargeting quality assessment. Cybern. Syst. 54, 673–696 (2023)
Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. Adv. Neural Inform. Process. Syst 19, 545–552 (2006)
Cui, Y., Jiang, G., Yu, M., Chen, Y., Ho, Y.-S.: Stitched wide field of view light field image quality assessment: Benchmark database and objective metric. IEEE Trans. Multim. 1–15 (2023)
Fang, Z., et al.: Blind 3d-synthesized image quality measurement by analysis of local and global statistical properties. IEEE Trans. Instrum. Meas. 72, 1–15 (2023)
Zheng, S., et al.: Conditional random fields as recurrent neural networks 1529–1537 (2015)
Ma, L., Lin, W., Deng, C., Ngan, K.: Image retargeting quality assessment: a study of subjective scores and objective metrics. IEEE J. Sel. Topics Signal Process. 6, 626–639 (2012)
Dong, W., Zhou, N., Paul, J.-C., Zhang, X.: Optimized image resizing using seam carving and scaling. ACM Trans. Graph. 28 (2009)
Karni, Z., Freedman, D., Gotsman, C.: Energy-based image deformation. Comput. Graph. Forum 28, 1257–1268 (2009)
Lin, W., Kuo, C.-C.J.: Perceptual visual quality metrics: a survey. J. Visual Commun. Image Rep. 22, 297–312 (2011)
Jiang, B., Yang, J., Meng, Q., Li, B., Lu, W.A.: deep evaluator for image retargeting quality by geometrical and contextual interaction. IEEE Trans. Cybern. PP, 1–13 (2018)
Peng, Z., Qiuping, J., Shao, F., Gao, W., Lin, W.: Lggd+: Image retargeting quality assessment by measuring local and global geometric distortions. IEEE Trans. Circuits Syst. Video Technol. PP, 1–1 (2021)
Sheikh, H., Sabir, M., Bovik, A.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. Publicat. IEEE Signal Process. Soc. 15, 3440–51 (2006)
Kendall, M.A.: New measure of rank correlation. Biometrika 30 (1938)
Funding
No funding was received to assist with the preparation of this manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Absetan, A., Fathi, A. Integration of local and global features for image retargeting quality assessment. SIViP 18, 3577–3586 (2024). https://doi.org/10.1007/s11760-024-03022-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-024-03022-6