Skip to main content
SpringerLink
Log in

Super-resolution in practice: the complete pipeline from image capture to super-resolved subimage creation using a novel frame selection method

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

We present a complete super-resolution system using a camera, that is assumed to be on a vibrating platform and continually capturing frames of a static scene, that have to be super-resolved in particular regions of interest. In a practical system the shutter of the camera is not synchronised with the vibrations it is subjected to. So, we propose a novel method for frame selection according to their degree of blurring and we combine a tracker with the sequence of selected frames to identify the subimages containing the region of interest. The extracted subimages are subsequently co-registered using a state of the art sub-pixel registration algorithm. Further selection of the co-registered subimages takes place, according to the confidence in the registration result. Finally, the subimage of interest is super-resolved using a state of the art super-resolution algorithm. The proposed frame selection method is of generic applicability and it is validated with the help of manual frame quality assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Baboulaz L., Dragotti P.L.: Exact feature extraction using finite rate of innovation principles with an application to image super-resolution. IEEE Trans. Image Process. 18(2), 281–298 (2009)

    Article  MathSciNet  Google Scholar 

  2. Bascle, B., Blake, A., Zisserman, A.: Motion deblurring and super-resolution from an image sequence. In: European Conference on Computer Vision. Lecture Notes in Computer Science, vol. 1065, pp 571–582 (1996)

  3. Black M., Anandan P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Comput. Vis. Image Understand. 63(1), 75–104 (1996)

    Article  Google Scholar 

  4. Blake A., Zisserman A.: Visual Reconstruction. MIT Press, Cambridge (1987)

    Google Scholar 

  5. Domke J., Aloimonos Y.: Image transformations and blurring. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 811–823 (2009)

    Article  Google Scholar 

  6. Esparragon, D.R., Gonzalez, L.M.M., Santana, J.G.V., Ortiz, J.C.: Automatic image quality measurement tool. In: 3rd International Conference on Information and Communication Technologies: From Theory to Applications, pp. 1–6, April (2008)

  7. Ferzli R., Karam L.J.: A No-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). IEEE Trans. Image Process. 18(4), 717–728 (2009)

    Article  MathSciNet  Google Scholar 

  8. Gunawan I.P., Ghanbari M.: Reduced-reference video quality assessment using discriminative local harmonic strength with motion consideration. IEEE Trans. Circ. Syst. Video Technol. 18(1), 71–83 (2008)

    Article  Google Scholar 

  9. Hardie R., Barnard K., Armstrong E.: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans. Image Process. 6(2), 1621–1633 (1997)

    Article  Google Scholar 

  10. Katartzis, A., Petrou, M.: Robust Bayesian estimation and normalised convolution for super-resolution image reconstruction. In: Workshop on Image Registration and Fusion, Computer Vision and Pattern Recognition, CVPR’07, Minneapolis, USA, 18–23 June 2007, pp. 1–7 (2007)

  11. Katartzis A., Petrou M.: Current trends in super-resolution image reconstruction. In: Stathaki, T. (ed.) Image Fusion: Algorithms and Applications, Academic Press, New York (2008)

    Google Scholar 

  12. Knutsson, H., Westin, C.: Normalised and differential convolution. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 93, pp. 515–523 (1993)

  13. Lee W.-H., Lai S.-H., Chen C.-L.: Iterative blind image motion deblurring via learning a No-reference image quality measure. IEEE Int. Conf. Image Process. 4, 405–408 (2007)

    Google Scholar 

  14. Li, B., Osberger, W.: Measurement of blurring in video sequences, United States Patent 7099518 (2006)

  15. Mei T., Hua X.-S., Zhu C.-Z., Zhou H.-Q., Li S.: Home video visual quality assessment with spatiotemporal factors. IEEE Trans. Circ. Syst. Video Technol. 17(6), 699–706 (2007)

    Article  Google Scholar 

  16. Nguyen N., Milanfar P., Golub G.: Efficient generalised cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Image Process. 10(9), 308–1299 (2001)

    Article  MathSciNet  Google Scholar 

  17. Park S.C., Park M.K., Kang M.G.: Super-resolution image reconstruction. IEEE Signal Process. Mag. 20(3), 21–36 (2003)

    Article  Google Scholar 

  18. Petrou, M., Petrou, C.: Image Processing, the Fundamentals, J Wiley. ISBN: 9780470745861 (2010)

  19. Pham, T., van Vliet, L., Schutte, K.: Robust fusion of irregularly sampled data using adaptive normalised convolution. EURASIP J. Appl. Signal Process. pp. 1–12 (2006)

  20. Ponomarenko, N., Battisti, F., Egiazarian, K., Astola, J., Lukin, V.: Metrics performance comparison for colour image databases. In: Fourth International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, Arizona, USA. January 14–16 (2009)

  21. Robinson D., Milanfer P.: Fundamental performance limits in image registration. IEEE Trans. Image Process. 13(9), 1185–1199 (2004)

    Article  Google Scholar 

  22. Segall C., Katsaggelos A., Molina R., Mateos J.: Bayesian resolution of enhancement of compressed video. IEEE Trans. Image Process. 13(7), 898–910 (2004)

    Article  Google Scholar 

  23. Sheikh H.R., Bovik A.C., Cormack L.: No-reference quality assessment using natural scene statistics: JPEG2000. IEEE Trans. Image Process. 14(11), 1918–1927 (2005)

    Article  Google Scholar 

  24. Sheikh H.R., Bovik A.C.: Image information and visual quality. IEEE Trans. Image Process. 15(2), 430–444 (2006)

    Article  Google Scholar 

  25. Shen, H., Ng, M.K., Li, P., Zhang, L.: Super-resolution reconstruction algorithm to MODIS remote sensing images. Comput J. doi:10.1093/compjnl/bxm028 (2007)

  26. Shen H., Zhang L., Huang B., Li P.: A MAP approach for joint motion estimation, segmentation and super resolution. IEEE Trans. Image Process. 16(2), 479–490 (2007)

    Article  MathSciNet  Google Scholar 

  27. Shnayderman A., Gusev A., Eskicioglu A.M.: An SVD-based gray scale image quality measure for local and global assessment. IEEE Trans. Image Process. 15(2), 422–429 (2006)

    Article  Google Scholar 

  28. Thevenaz P., Ruttimann U., Unser M.: A pyramid approach to subpixel registration based on intensity. IEEE Trans. Image Process. 7(1), 27–41 (1998)

    Article  Google Scholar 

  29. Wang X., Tian B., Liang C., Shi D.: Blind image quality assessment for measuring image blur. Congress Image Sig. Process. CISP’ 08(1), 467–470 (2008)

    Article  Google Scholar 

  30. Wee C.-Y., Paramesran R.: Measure of image sharpness using eigenvalues. Int. J. Inform. Sci. 177(12), 2533–2552 (2007)

    MATH  Google Scholar 

  31. Woods N., Galatsanos N., Katsaggelos A.: Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images. IEEE Trans. Image Process. 15(1), 201–213 (2006)

    Article  MathSciNet  Google Scholar 

  32. Zamani, A.N., Awang, M.K., Omar, N., Nazeer, S.A.: Image quality assessments and restoration for face detection and recognition system images. In: Second Asia International Conference on Modelling and Simulation, pp. 505–510, May (2008)

  33. Zheng H., Hellwich O.: VQ-based Bayesian Estimation for Blur Identification and Image Selection in Video Sequences. Int. J. Innov. Comput. Inform. Control (IJICIC) 2(2), 399–410 (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Communications and Signal Processing Group, Electrical and Electronic Engineering Department, Imperial College, London, SW7 2AZ, UK

    Maria Petrou, Mohamed H. Jaward & Shengyong Chen

  2. AT030, QinetiQ, Malvern, UK

    Mark Briers

Authors
  1. Maria Petrou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed H. Jaward
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shengyong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mark Briers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria Petrou.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Petrou, M., Jaward, M.H., Chen, S. et al. Super-resolution in practice: the complete pipeline from image capture to super-resolved subimage creation using a novel frame selection method. Machine Vision and Applications 23, 441–459 (2012). https://doi.org/10.1007/s00138-010-0315-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-010-0315-7

Keywords

Search

Navigation