skip to main content
10.1145/3343031.3350973acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Cross-Reference Stitching Quality Assessment for 360° Omnidirectional Images

Published: 15 October 2019 Publication History

Abstract

Along with the development of virtual reality (VR), omnidirectional images play an important role in producing multimedia content with an immersive experience. However, despite various existing approaches for omnidirectional image stitching, how to quantitatively assess the quality of stitched images is still insufficiently explored. To address this problem, we first establish a novel omnidirectional image dataset containing stitched images as well as dual-fisheye images captured from standard quarters of 0$^\circ$, 90$^\circ$, 180$^\circ$, and 270$^\circ$. In this manner, when evaluating the quality of an image stitched from a pair of fisheye images (\eg, 0$^\circ$ and 180$^\circ$), the other pair of fisheye images (\eg, 90$^\circ$ and 270$^\circ$) can be used as the cross-reference to provide ground-truth observations of the stitching regions. Based on this dataset, we propose a set of Omnidirectional Stitching Image Quality Assessment (OS-IQA) metrics. In these metrics, the stitching regions are assessed by exploring the local relationships between the stitched image and its cross-reference with histogram statistics, perceptual hash and sparse reconstruction, while the whole stitched images are assessed by the global indicators of color difference and fitness of blind zones.Qualitative and quantitative experiments show our method outperforms the classic IQA metrics and is highly consistent with human subjective evaluations. To the best of our knowledge, it is the first attempt that assesses the stitching quality of omnidirectional images by using cross-references.

References

[1]
Marc Assens Reina, Xavier Giro-i Nieto, Kevin McGuinness, and Noel E O'Connor. 2017. Saltinet: Scan-path prediction on 360 degree images using saliency volumes. In Proceedings of the IEEE International Conference on Computer Vision. 2331--2338.
[2]
Jonathan Baldwin and Anup Basu. 2000. 3D estimation using panoramic stereo. In Proceedings 15th International Conference on Pattern Recognition. ICPR-2000. 97--100.
[3]
Sebastian Bosse, Dominique Maniry, Klaus-Robert Müller, Thomas Wiegand, and Wojciech Samek. 2018. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Transactions on Image Processing (2018), 206--219.
[4]
Deng Cai, Xiaofei He, and Jiawei Han. 2007. Isometric Projection. In Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, July 22--26, 2007, Vancouver, British Columbia, Canada. 528--533.
[5]
Gene Cheung, Luyu Yang, Zhigang Tan, and Zhe Huang. 2017. A Content-Aware Metric for Stitched Panoramic Image Quality Assessment. In 2017 IEEE International Conference on Computer Vision Workshops, ICCV Workshops 2017, Venice, Italy, October 22--29, 2017. 2487--2494.
[6]
Huiyu Duan, Guangtao Zhai, Xiongkuo Min, Yucheng Zhu, Yi Fang, and Xiaokang Yang. 2018. Perceptual quality assessment of omnidirectional images. In 2018 IEEE International Symposium on Circuits and Systems (ISCAS). 1--5.
[7]
Github.com. 2016. DualFisheye. https://github.com/ooterness/DualFisheye .
[8]
Samsung Group. 2017. Samsung Gear360. https://www.samsung.com/global/galaxy/gear-360 .
[9]
Mingkai Huang, Qiu Shen, Zhan Ma, Alan Conrad Bovik, Praful Gupta, Rongbing Zhou, and Xun Cao. 2018. Modeling the Perceptual Quality of Immersive Images Rendered on Head Mounted Displays: Resolution and Compression. IEEE Trans. Image Processing (2018), 6039--6050.
[10]
Le Kang, Peng Ye, Yi Li, and David S. Doermann. 2014. Convolutional Neural Networks for No-Reference Image Quality Assessment. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, OH, USA, June 23--28, 2014. 1733--1740.
[11]
Le Kang, Peng Ye, Yi Li, and David S. Doermann. 2015. Simultaneous estimation of image quality and distortion via multi-task convolutional neural networks. In 2015 IEEE International Conference on Image Processing, ICIP 2015, Quebec City, QC, Canada, September 27--30, 2015 . 2791--2795.
[12]
Jari Korhonen. 2019. Assessing Personally Perceived Image Quality via Image Features and Collaborative Filtering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019. 8169--8177.
[13]
Chaofeng Li, Alan Conrad Bovik, and Xiaojun Wu. 2011. Blind image quality assessment using a general regression neural network. IEEE Transactions on Neural Networks (2011), 793--799.
[14]
Jing Li, Marcus Barkowsky, and Patrick Le Callet. 2013. Boosting paired comparison methodology in measuring visual discomfort of 3DTV: performances of three different designs. In Stereoscopic Displays and Applications XXIV . 86481V.
[15]
Suiyi Ling, Gene Cheung, and Patrick Le Callet. 2018. No-Reference Quality Assessment for Stitched Panoramic Images Using Convolutional Sparse Coding and Compound Feature Selection. In 2018 IEEE International Conference on Multimedia and Expo, ICME 2018, San Diego, CA, USA, July 23--27, 2018 . 1--6.
[16]
Lixiong Liu, Hongping Dong, Hua Huang, and Alan C. Bovik. 2014. No-reference image quality assessment in curvelet domain. Sig. Proc.: Image Comm. (2014), 494--505.
[17]
Xialei Liu, Joost van de Weijer, and Andrew D. Bagdanov. 2017. RankIQA: Learning from Rankings for No-Reference Image Quality Assessment. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22--29, 2017. 1040--1049.
[18]
David G Lowe et almbox. 1999. Object recognition from local scale-invariant features. In ICCV. 1150--1157.
[19]
Behnam Maneshgar, Leila Sujir, Sudhir P. Mudur, and Charalambos Poullis. 2017. A Long-range Vision System for Projection Mapping of Stereoscopic Content in Outdoor Areas. In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017) - Volume 1: GRAPP, Porto, Portugal, February 27 - March 1, 2017. 290--297.
[20]
Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. 2012. No-Reference Image Quality Assessment in the Spatial Domain. IEEE Trans. Image Processing (2012), 4695--4708.
[21]
Anish Mittal, Rajiv Soundararajan, and Alan C. Bovik. 2013. Making a "Completely Blind" Image Quality Analyzer. IEEE Signal Process. Lett. (2013), 209--212.
[22]
Anush K. Moorthy and Alan Conrad Bovik. 2011. Blind Image Quality Assessment: From Natural Scene Statistics to PerceptualQuality. IEEE Trans. Image Processing (2011), 3350--3364.
[23]
Venkatanath N., Praneeth D., Maruthi Chandrasekhar Bh., Sumohana S. Channappayya, and Swarup S. Medasani. 2015. Blind image quality evaluation using perception based features. In Twenty First National Conference on Communications, NCC 2015, Mumbai, India, February 27 - March 1, 2015. 1--6.
[24]
Yuzhen Niu, Haifeng Zhang, Wenzhong Guo, and Rongrong Ji. 2018. Image Quality Assessment for Color Correction Based on Color Contrast Similarity and Color Value Difference. IEEE Trans. Circuits Syst. Video Techn. (2018), 849--862.
[25]
Kristopher J Preacher, Patrick J Curran, and Daniel J Bauer. 2006. Computational tools for probing interactions in multiple linear regression, multilevel modeling, and latent curve analysis. Journal of educational and behavioral statistics (2006), 437--448.
[26]
Yuntao Qian, Danping Liao, and Jun Zhou. 2013. Manifold alignment based color transfer for multiview image stitching. In IEEE International Conference on Image Processing, ICIP 2013, Melbourne, Australia, September 15--18, 2013. 1341--1345.
[27]
HS Qureshi, MM Khan, R Hafiz, Y Cho, and J Cha. 2012. Quantitative quality assessment of stitched panoramic images. IET image processing (2012), 1348--1358.
[28]
Yashas Rai, Jesús Gutiérrez, and Patrick Le Callet. 2017a. A dataset of head and eye movements for 360 degree images. In Proceedings of the 8th ACM on Multimedia Systems Conference. 205--210.
[29]
Yashas Rai, Patrick Le Callet, and Philippe Guillotel. 2017b. Which saliency weighting for omni directional image quality assessment?. In 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX) . 1--6.
[30]
Christian Richardt, Yael Pritch, Henning Zimmer, and Alexander Sorkine-Hornung. 2013. Megastereo: Constructing high-resolution stereo panoramas. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1256--1263.
[31]
Hamid R. Sheikh, Muhammad F. Sabir, and Alan C. Bovik. 2006. A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms. IEEE Trans. Image Processing (2006), 3440--3451.
[32]
Mikhail Startsev and Michael Dorr. 2018. 360-aware saliency estimation with conventional image saliency predictors. Signal Processing: Image Communication (2018), 43--52.
[33]
Alexander Tanchenko. 2014. Visual-PSNR measure of image quality. Journal of Visual Communication and Image Representation (2014), 874--878.
[34]
Evgeniy Upenik, Martin Rerá bek, and Touradj Ebrahimi. 2016. Testbed for subjective evaluation of omnidirectional visual content. In 2016 Picture Coding Symposium, PCS 2016, Nuremberg, Germany, December 4--7, 2016 . 1--5.
[35]
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Processing (2004), 600--612.
[36]
Zhou Wang and Qiang Li. 2011. Information Content Weighting for Perceptual Image Quality Assessment. IEEE Trans. Image Processing (2011), 1185--1198.
[37]
Wufeng Xue, Lei Zhang, Xuanqin Mou, and Alan C. Bovik. 2014. Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index. IEEE Trans. Image Processing (2014), 684--695.
[38]
Peng Ye, Jayant Kumar, Le Kang, and David Doermann. 2012. Unsupervised feature learning framework for no-reference image quality assessment. In 2012 IEEE Conference on Computer Vision and Pattern Recognition . 1098--1105.
[39]
Weihua Ye, Kaiwen Yu, Yang Yu, and Jia Li. 2018. Logical Stitching: A Panoramic Image Stitching Method Based on Color Calibration Box. In 2018 14th IEEE International Conference on Signal Processing (ICSP) . 1139--1143.
[40]
Yuan Yuan, Qun Guo, and Xiaoqiang Lu. 2015. Image quality assessment: a sparse learning way. Neurocomputing (2015), 227--241.
[41]
Julio Zaragoza, Tat-Jun Chin, Michael S Brown, and David Suter. 2013. As-projective-as-possible image stitching with moving DLT. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2339--2346.
[42]
Chuang Zhang, Jing Pan, Suting Chen, Tingting Wang, and Dongjiao Sun. 2016. No reference image quality assessment using sparse feature representation in two dimensions spatial correlation. Neurocomputing (2016), 462--470.
[43]
Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. 2011. FSIM: A Feature Similarity Index for Image Quality Assessment. IEEE Trans. Image Processing (2011), 2378--2386.

Cited By

View all
  • (2025)Quality of Experience-Oriented Cloud-Edge Dynamic Adaptive Streaming: Recent Advances, Challenges, and OpportunitiesSymmetry10.3390/sym1702019417:2(194)Online publication date: 26-Jan-2025
  • (2024)OmniStitch: Depth-Aware Stitching Framework for Omnidirectional Vision with Multiple CamerasProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681208(10210-10219)Online publication date: 28-Oct-2024
  • (2024)Saliency and Depth-Aware Full Reference 360-Degree Image Quality AssessmentInternational Journal of Pattern Recognition and Artificial Intelligence10.1142/S021800142351022938:01Online publication date: 9-Feb-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '19: Proceedings of the 27th ACM International Conference on Multimedia
October 2019
2794 pages
ISBN:9781450368896
DOI:10.1145/3343031
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. cross-reference
  2. omnidirectional stitching
  3. quality assessment

Qualifiers

  • Research-article

Funding Sources

  • National Natural Science Foundation of China
  • Beijing Nova Program

Conference

MM '19
Sponsor:

Acceptance Rates

MM '19 Paper Acceptance Rate 252 of 936 submissions, 27%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)97
  • Downloads (Last 6 weeks)7
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Quality of Experience-Oriented Cloud-Edge Dynamic Adaptive Streaming: Recent Advances, Challenges, and OpportunitiesSymmetry10.3390/sym1702019417:2(194)Online publication date: 26-Jan-2025
  • (2024)OmniStitch: Depth-Aware Stitching Framework for Omnidirectional Vision with Multiple CamerasProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681208(10210-10219)Online publication date: 28-Oct-2024
  • (2024)Saliency and Depth-Aware Full Reference 360-Degree Image Quality AssessmentInternational Journal of Pattern Recognition and Artificial Intelligence10.1142/S021800142351022938:01Online publication date: 9-Feb-2024
  • (2024)Eye Scanpath Prediction-Based No-Reference Quality Assessment of Omnidirectional ImagesIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.347023973(1-15)Online publication date: 2024
  • (2024)Spatio-Temporal Feature Integration for Quality Assessment of Stitched Omnidirectional ImagesIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2023.33413338:2(1484-1499)Online publication date: Apr-2024
  • (2024)Perceptual Depth Quality Assessment of Stereoscopic Omnidirectional ImagesIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.344969634:12(13452-13462)Online publication date: Dec-2024
  • (2024)Diffusion-Based Wireless Semantic Communication for VR Image2024 IEEE/CIC International Conference on Communications in China (ICCC Workshops)10.1109/ICCCWorkshops62562.2024.10693684(639-644)Online publication date: 7-Aug-2024
  • (2024)Generating a full spherical view by modeling the relation between two fisheye imagesThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-024-03293-740:10(7107-7132)Online publication date: 1-Oct-2024
  • (2023)Attention-Aware Patch-Based CNN for Blind 360-Degree Image Quality AssessmentSensors10.3390/s2321867623:21(8676)Online publication date: 24-Oct-2023
  • (2023)Learning-based Homography Matrix Optimization for Dual-fisheye Video StitchingProceedings of the 2023 Workshop on Emerging Multimedia Systems10.1145/3609395.3610600(48-53)Online publication date: 10-Sep-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media