Skip to main content
SpringerLink
Log in

Continuous digital zooming of asymmetric dual camera images using registration and variational image restoration

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

This paper presents a theoretical basis to realize a high-quality digital zooming using two camera modules with different focal lengths. First, we describe an image degradation model of the asymmetric dual camera system to analyze the characteristic of the wide- and tele-view images. In an asymmetric dual camera system, we assume that the shorter focal length module produces the wide-view image with the low-resolution. On the other hand, the longer focal length module produces the tele-view image by an optical zooming. To reconstruct a wide-view image of a continuous digital zooming, the proposed method first estimates the point spread function (PSF) between the wide- and tele-view images. Next, the proposed method performs variational-based image restoration using the estimated PSF. In addition, since the tele-view image inserted into appropriate region of the wide-view image, the proposed method can provide significantly improved wide-view image.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  • Battiato, S., Gallo, G., & Stanco, F. (2002). A locally adaptive zooming algorithm for digital images. Image and Vision Computing, 20(11), 805–812.

    Article  Google Scholar 

  • Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. In Proceedings of European conference on computer vision (pp. 404–417). Berlin: Springer.

  • Bednar, J., & Watt, T. (1984). Alpha-trimmed means and their relationship to median filters. IEEE Transactions on Acoustics, Speech, and Signal Processing, 32(1), 145–153.

    Article  Google Scholar 

  • Bishop, T. E., Lindskog, A., Molgaard, C., & Doepke, F. (2015). Photo-realistic shallow depth-of-field rendering from focal stacks. US Patent App. 14/864,650.

  • Chang, Y., Yan, L., Fang, H., & Liu, H. (2014). Simultaneous destriping and denoising for remote sensing images with unidirectional total variation and sparse representation. IEEE Geoscience and Remote Sensing Letters, 11(6), 1051–1055.

    Article  Google Scholar 

  • Dong, C., Loy, C. C., He, K., & Tang, X. (2016). Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 295–307.

    Article  Google Scholar 

  • Du, C., Deng, B., & Luo, W. (2013). Photographing method of dual-lens device, and dual-lens device. US Patent App. 15/101,719.

  • Farsiu, S., Robinson, M. D., Elad, M., & Milanfar, P. (2004). Fast and robust multiframe super resolution. IEEE Transactions on Image Processing, 13(10), 1327–1344.

    Article  Google Scholar 

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    Article  MathSciNet  Google Scholar 

  • Freedman, G., & Fattal, R. (2011). Image and video upscaling from local self-examples. ACM Transactions on Graphics, 30(2), 12:1–12:11.

    Article  Google Scholar 

  • Freeman, W. T., Jones, T. R., & Pasztor, E. C. (2002). Example-based super-resolution. IEEE Computer Graphics and Applications Magazine, 22(2), 56–65.

    Article  Google Scholar 

  • Glasner, D., Bagon, S., & Irani, M. (2009). Super-resolution from a single image. In Proceedings of IEEE international conference on computer vision (pp. 349–356).

  • Goldstein, T., & Osher, S. (2009). The split Bregman method for L1-regularized problems. SIAM Journal on Imaging Sciences, 2(2), 323–343.

    Article  MathSciNet  MATH  Google Scholar 

  • Gonzalez, R., & Woods, R. (2008). Digital image processing. Englewood Cliffs, NJ: Pearson Prentice Hall.

    Google Scholar 

  • Hicks, R. (2015). Infrared and visible light dual sensor imaging system. US Patent App. 14/864,717.

  • Hong, K., Paik, J., Kim, H., & Lee, C. (1996). An edge-preserving image interpolation system for a digital camcorder. IEEE Transactions on Consumer Electronics, 42(3), 279–284.

    Article  Google Scholar 

  • Hu, Z., & Yang, M. H. (2012). Good regions to deblur. In Proceedings of European conference on computer vision (pp. 59–72). Berlin: Springer.

  • Hughes, C., Denny, P., Jones, E., & Glavin, M. (2010). Accuracy of fish-eye lens models. Applied Optics, 49(17), 3338–3347.

    Article  Google Scholar 

  • Hunt, B. R. (1973). The application of constrained least squares estimation to image restoration by digital computer. IEEE Transactions on Computers, 22(9), 805–812.

    Article  Google Scholar 

  • James, S., Maik, V., Karibassappa, K., & Paik, J. (2015). Regularized multichannel blind deconvolution using alternating minimization. IEIE Transactions on Smart Processing and Computing, 4(6), 413–421.

    Article  Google Scholar 

  • Kang, W., Jeon, J., Yu, S., & Paik, J. (2014). Fast digital zooming system using directionally adaptive image interpolation and restoration. SpringerPlus, 3(1), 1–9.

    Article  Google Scholar 

  • Keys, R. (1981). Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing, 29(6), 1153–1160.

    Article  MathSciNet  MATH  Google Scholar 

  • Kim, D., Park, J., & Paik, J. (2014a). Extended fisheye lens model for practical geometric correction and image enhancement. Optics Letters, 39(21), 6261–6264.

    Article  Google Scholar 

  • Kim, D., Shin, J., & Paik, J. (2014b). Real-time digital auto-focusing using prior PSF estimation. TECHART: Journal of Arts and Imaging Science, 1(3), 39–41.

    Google Scholar 

  • Kim, J.G., Lim, S. H., & Jeon, S. R. (2016). Digital photographing apparatus and method of operating the same. US Patent App. 15/058,319.

  • Lee, H., Jeon, S., Yoon, I., & Paik, J. (2016a). Recent advances in feature detectors and descriptors. IEIE Transactions on Smart Processing and Computing, 5(3), 153–163.

    Article  Google Scholar 

  • Lee, O., Park, S., Kim, J., & Kim, J. (2014). Multi-frame super-resolution of high frequency with spatially weighted bilateral total variance regularization. IEIE Transactions on Smart Processing and Computing, 3(5), 271–274.

    Article  Google Scholar 

  • Lee, S., Jeong, S., Yu, H., Kim, G., Kwak, H., Kang, E., et al. (2016b). Efficient image transformation and camera registration for the multi-projector image calibration. TECHART: Journal of Arts and Imaging Science, 3(1), 38–42.

    Article  Google Scholar 

  • Lee, S., Zhang, D., & Ko, S. (2015). Image contrast enhancement based on a multi-cue histogram. IEIE Transactions on Smart Processing and Computing, 4(5), 349–353.

    Article  Google Scholar 

  • Li, X., Hu, Y., Gao, X., Tao, D., & Ning, B. (2010). A multi-frame image super-resolution method. Signal Processing, 90(2), 405–414.

    Article  MATH  Google Scholar 

  • Paik, J., Park, S., & Kim, H. (1993). Combined digital zooming and digital effects system utilizing CCD signal characteristics. IEEE Transactions on Consumer Electronics, 39(3), 398–406.

    Article  Google Scholar 

  • Park, S., Park, M., & Kang, M. (2003). Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine, 20(3), 21–36.

    Article  Google Scholar 

  • Rudin, L. I., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60(1), 259–268.

    Article  MathSciNet  MATH  Google Scholar 

  • Shin, J. H., Jung, J. H., & Paik, J. K. (1998). Regularized iterative image interpolation and its application to spatially scalable coding. IEEE Transactions on Consumer Electronics, 44(3), 1042–1047.

    Article  Google Scholar 

  • Timofte, R., De Smet, V., & Van Gool, L. (2014). A+: Adjusted anchored neighborhood regression for fast super-resolution. In Proceedings of Asian conference on computer vision (pp. 111–126). Berlin: Springer.

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.

    Article  Google Scholar 

  • Yang, J., Wright, J., Huang, T., & Ma, Y. (2008). Image super-resolution as sparse representation of raw image patches. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1–8).

  • Yang, J., Wright, J., Huang, T., & Ma, Y. (2010). Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 19(11), 2861–2873.

    Article  MathSciNet  MATH  Google Scholar 

  • Yang, X. D., Xiao, Q., & Raafat, H. (1991). Direct mapping between histograms: An improved interactive image enhancement method. In Proceedings of IEEE international conference on systems, man, and cybernetics (Vol. 1, pp. 243–247).

  • Yoo, Y., Jang, J., Shin, J., & Paik, J. (2015). Optimal PSF selection using second-order frequency analysis for digital autofocusing. TECHART: Journal of Arts and Imaging Science, 2(1), 81–86.

    Article  Google Scholar 

  • Yu, S., Kang, W., Ko, S., & Paik, J. (2015). Single image super-resolution using locally adaptive multiple linear regression. Journal of the Optical Society of America A, 32(12), 2264–2275.

    Article  Google Scholar 

  • Zhang, K., Gao, X., Tao, D., Li, X. (2012). Multi-scale dictionary for single image super-resolution. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1114–1121).

  • Zhu, Y., Zhang, Y., & Yuille, A. L. (2014). Single image super-resolution using deformable patches. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 2917–2924).

  • Zitov, B., & Flusser, J. (2003). Image registration methods: A survey. Image and Vision Computing, 21(11), 977–1000.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Image, Chung-Ang University, Seoul, 06974, Korea

    Soohwan Yu, Byeongho Moon, Sangkeun Lee & Joonki Paik

  2. Convergence R&D Center, LG Innotek, Ansan-si, Gyeonggi-do, Korea

    Donggyun Kim

  3. DMC R&D Center, Samsung Electronics, Seoul, Korea

    Sehoon Kim & Wonhee Choe

Authors
  1. Soohwan Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Byeongho Moon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Donggyun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sehoon Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wonhee Choe
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sangkeun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Joonki Paik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joonki Paik.

Additional information

This research was partly funded and supported by Samsung Electronics Co., Ltd. and the ICT R&D program of MSIP/IITP (2017-0-00250, Intelligent Defense Boundary Surveillance Technology Using Collaborative Reinforced Learning of Embedded Edge Camera and Image Analysis).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, S., Moon, B., Kim, D. et al. Continuous digital zooming of asymmetric dual camera images using registration and variational image restoration. Multidim Syst Sign Process 29, 1959–1987 (2018). https://doi.org/10.1007/s11045-017-0534-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-017-0534-4

Keywords

Search

Navigation