Skip to main content
SpringerLink
Log in

A new closed loop method of super-resolution for multi-view images

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose a closed loop method to resolve the multi-view super-resolution problem. For the mixed-resolution multi-view case, where the input is one high-resolution view along with its neighboring low-resolution views, our method can give the super-resolution results and obtain a high-quality depth map simultaneously. The closed loop method consists of two parts: part I, stereo matching and depth maps fusion; and part II, super-resolution. Under the guidance of the estimated depth information, the super-resolution problem can be formulated as an optimization problem. It can be solved approximately by a three-step method, which involves disparity-based pixel mapping, nonlocal construction and final fusion. Based on the super-resolution results, we can update the disparity maps and fuse them into a more reliable depth map. We repeat the loop several times until obtaining stable super-resolution results and depth maps simultaneously. The experimental results on public dataset show that the proposed method can achieve high-quality performance at different scale factors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. http://vision.middlebury.edu/stereo/data/.

  2. http://www.ifp.illinois.edu/~jyang29/ScSR.htm.

References

  1. Allebach, J., Wong, P.: Edge-directed interpolation. In: Proceedings of IEEE International Conference on Image Processing (ICIP’ 1996) (1996)

  2. Aly, H.A., Dubois, E.: Image up-sampling using total-variation regularization with a new observation model. IEEE Trans. Image Process. 14(10), 1647–1659 (2005)

    Article  MathSciNet  Google Scholar 

  3. Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002)

    Article  Google Scholar 

  4. Brust, H., Tech, G., Mller, K.: Report on generation of mixed spatial resolution stereo data base. Technical report, MOBILE3DTV project (2009)

  5. Buades, A., Coll, B., Morel, J.: A non-local algorithm for image denoising. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition (CVPR’ 2005) (2005)

  6. Buades, A., Coll, B., Morel, J.: A review of image denoising algorithms with a new one. Multiscale Model Simul. 4, 490–530 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  7. Caselles, V., Morel, J., Sbert, C.: An axiomatic approach to image interpolation. IEEE Trans. Image Process. 7(3), 376–386 (Mar. 1998)

    Google Scholar 

  8. Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition (CVPR’ 2004) (2004)

  9. Dai, S., Han, M., et al.: Soft edge smoothness prior for alpha channel super resolution. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition (CVPR’ 2007) (2007)

  10. Gao, S., Tsang, I., Chia, L.: Laplacian sparse coding, hypergraph laplacian sparse coding, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 92–104 (2013)

    Article  Google Scholar 

  11. Garcia, D., Dorea, C., de Queiroz, R.: Super resolution for multiview images using depth information. IEEE Trans. Circ. Syst. Video Technol. 22(9), 1249–1256 (2012)

    Article  Google Scholar 

  12. He, H., Siu, W.C.: Single image super-resolution using gaussian processing regression. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’ 2011) (2011)

  13. Hong, R., Tang, J., Tan, H., et al.: Beyond search: event-driven summarization for web videos. ACM Trans. Multimedia Comput. Commun. Appl. (TOMCCAP) 7(4), 35 (2011)

    Google Scholar 

  14. Hong, R., Wang, M., Li, G., Nie, L., Zha, Z., Chua, T.: Multimedia question answering. IEEE Trans. Multimedia 19(4), 72–78 (2012)

    Article  Google Scholar 

  15. Kaick, O., Zhang, H., Hamarneh, G.: Bilateral maps for partial matching. In: Computer Graphics Forum (2013)

  16. Li, X., Orchard, M.: New edge-directed interpolation. In: Proceedings of IEEE International Conference on Image Processing (ICIP’ 2000) (2000)

  17. Wang, M., Hong, R., Li, G., Zha, Z., Yan, S., Chua, T.: Event driven web video summarization by tag localization and key-shot identification. IEEE Trans. Multimedia 14(4), 975–985 (2012)

    Article  Google Scholar 

  18. Wang, M., Hua, X., Tang, J., Hong, R.: Beyond distance measurement: constructing neighborhood similarity for video annotation. IEEE Trans. Multimedia 11(3), 465–476 (2009)

    Article  Google Scholar 

  19. Protter, M., Elad, M., Takeda, H., Milanfar, P.: Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans. Image Process. 18(1), 36–51 (Jan. 2009)

    Google Scholar 

  20. Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’ 2007) (2007)

  21. Shan, Q., Li, Z., Jia, J., Tang, C.K.: Fast image/video upsampling. In: ACM Transactions on Grapihcs (SIGGRAPH ASIA’ 2008) (2008)

  22. Sun, J., Xu, Z.,, Shum, H.Y.: Image super-resolution using gradient profile prior. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’ 2008) (2008)

  23. Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, Berlin (2011)

  24. Tai, Y.W., Liu, S.,, Brown, M., Lin, S.: Super resolution using edge prior and single image detail synthesis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’ 2010) (2010)

  25. Wang, C., Yan, S., Zhang, L., Zhang, H.: Multi-label sparse coding for automatic image annotation. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition (CVPR’ 2009) (2009)

  26. Wang, M., Hong, R., Li, G., Zha, Z., Yan, S., Chua, T.: Event driven web video summarization by tag localization and key-shot identification. IEEE Trans. Multimedia 14(4), 975–985 (2012)

    Article  Google Scholar 

  27. Wang, Q., Tang, X., Shum, H.Y.: Patch based blind image super resolution. In: Proceedings of International Conference on Computer Vision (ICCV’ 2005) (2005)

  28. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  29. Wang, Z.F., Zheng, Z.G.: A region based stereo matching algorithm using cooperative optimization. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’ 2008) (2008)

  30. Xia, T., Tao, D., Mei, T., Zhang, Y.: Multiview spectral embedding. IEEE Trans. Syst. Man Cybern. Part B Cybern. 40(6), 1438–1446 (2010)

    Article  Google Scholar 

  31. Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  Google Scholar 

  32. Yu, J., Liu, D., Tao, D., Seah, H.: Complex object correspondence construction in two-dimensional animation. IEEE Trans. Image Process. 20(11), 3257–3269 (2011)

    Article  MathSciNet  Google Scholar 

  33. Yu, J., Wang, M., Tao, D.: Semisupervised multiview distance metric learning for cartoon synthesis. IEEE Trans. Image Process. 21(11), 4636–4648 (2012)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This paper is supported by the Fundamental Research Funds for the Central Universities of China (No. WK2100100009), NSFC (No.61175033), NSFY (No.BJ2100100018) and STP (No.11010202192) of Anhui.

Author information

Authors and Affiliations

  1. Department of Automation, University of Science and Technology of China, Hefei, Anhui, People’s Republic of China

    Jing Zhang, Yang Cao, Zhigang Zheng & Zengfu Wang

  2. Department of Computer Science and Engineering, University at Buffalo, State University of New York, Buffalo, NY, 14260-2000, USA

    Changwen Chen

Authors
  1. Jing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhigang Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Changwen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zengfu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang Cao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, J., Cao, Y., Zheng, Z. et al. A new closed loop method of super-resolution for multi-view images. Machine Vision and Applications 25, 1685–1695 (2014). https://doi.org/10.1007/s00138-013-0536-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-013-0536-7

Keywords

Search

Navigation