Abstract
In this study, video super-resolution using particle swarm optimization (PSO) is proposed to super-resolve low-resolution (LR) frames. The proposed super-resolution method consists of three main modules, i.e., supersampling, spatio-temporal classification, and frame fusion using PSO. In the proposed method, the LR frames are super-resolved to high-resolution frames through the fusion of four full-resolution frames. One of four full-resolution frames is obtained using direct spatial interpolation, and the other three are obtained using motion compensation with given reference frames. The essence of the proposed method is the spatio-temporal classification mechanism that exploits the temporal variation between frames and the spatial energy inside the frame. Using the classification results, PSO is used to determine the optimal weights for frame fusion. Simulation results show that the proposed fusion method successfully improves the perceptual quality and the average peak signal-to-noise ratio (PSNR) in super-resolved frames.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Zhang, L., Zhang, H., Shen, H., & Li, P. (2010). A super-resolution reconstruction algorithm for surveillance images. Signal Processing, 90, 848–859.
Girod, B., Aaron, A., Rane, S., & Rebollo-Monedero, D. (2005). Distributed video coding. Proceedings of the IEEE, 93, 71–83.
Kennedy, J. A., Israel, O., Frenkel, A., BarShalom, R., & Azhari, H. (2006). Super-resolution in PET imaging. IEEE Trans. on Medical Imaging, 25, 137–147.
Kasetkasem, T., Arora, M. K., & Varshney, P. K. (2005). Super-resolution land cover mapping using a Markov random field based approach. Remote Sensing of Environment, 96, 302–314.
Park, S. C., Park, M. K., & Kang, M. G. (2003). Super-resolution image reconstruction: a technical overview. IEEE Signal Processing Magazine, 20, 21–36.
Ouwerkerk, J. D. V. (2006). Image super-resolution survey. Image and Vision Computing, 24, 1039–1052.
Chen, H. Y., & Leou, J. J. (2010). Saliency-directed image interpolation using particle swarm optimization. Signal Processing, 90(5), 1676–1692.
Freeman, W. T., Jones, T. R., & Pasztor, E. C. (2002). Example-based super-resolution. IEEE Computer Graphics and Applications, 22, 56–65.
Li, X., Lam, K. M., Qiu, G., Shen, L., & Wang, S. (2009). Example-based image super-resolution with class-specific predictors. Journal of Visual Communication and Image Representation, 20, 312–322.
Costa, G. H., & Bermudez, J. C. M. (2007). Statistical analysis of the LMS algorithm applied to super-resolution image reconstruction. IEEE Trans. on Signal Processing, 55(5), 2084–2095.
Protter, M., Elad, M., Takeda, H., & Milanfar, P. (2009). Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans. on Image Processing, 18, 36–51.
Brandi, F., Queiroz, R.D., Mukherjee, D. (2008) Super-resolution of video using key frames. in IEEE International Symposium on Circuits and Systems, 1608–1611.
Brandi, F., Queiroz, R.D., Mukherjee, D. (2008) Super-resolution of video using key frames and motion estimation. in IEEE International Conference on Image Processing, 321–324.
Simonyan, K., Grishin, S., Vatolin, D., Popov, D. (2008) Fast video super-resolution via classification. in IEEE International Conference on Image Processing, 349–352.
Li, X. (2007). Video processing via implicit and mixture motion models. IEEE Trans. on Circuits and Systems for Video Technology, 17, 953–963.
Dane, G., & Nguyen, T. Q. (2006). Optimal temporal interpolation filter for motion-compensated frame rate up conversion. IEEE Trans. on Image Processing, 15, 978–991.
Shechtman, E., Caspi, Y., & Irani, M. (2005). Space-time super-resolution. IEEE Trans. on Pattern Analysis and Machine Intelligence, 27, 531–545.
Shahar, O., Faktor, A., Irani, M. (2011) Space-time super-resolution from a single video. in IEEE International Conference on Computer Vision and Pattern Recognition, 3353–3360.
Kennedy, J., Eberhart, R.C. (1995) Particle swarm optimization. in IEEE International Conference on Neural Networks, 1942–1948.
Eberhart, R.C., Kennedy, J. (1995) A new optimizer using particle swarm theory. in IEEE International Symposium on Micro Machine and Human Science, 39–43.
Tseng, C. C., Hsieh, J. G., & Jeng, J. H. (2008). Fractal image compression using visual-based particle swarm optimization. Image and Vision Computing, 26, 1154–1162.
Gan, Z., Chow, T. W. S., & Huang, D. (2008). Effective gene selection method using bayesian discriminant based criterion and genetic algorithms. Journal of Signal Processing Systems, 50, 293–304.
Ali, Y. M. B. (2009). Edge-based segmentation using robust evolutionary algorithm applied to medical images. Journal of Signal Processing Systems, 54, 231–238.
Hung, W., Xie, Y., Vijaykrishnan, N., Kandemir, M., & Irwin, M. J. (2010). Total power optimization for combinational logic using genetic algorithms. Journal of Signal Processing Systems, 58, 145–160.
Rajavel, R., Sathidevi, P.S. (2011) Adaptive reliability measure and optimum integration weight for decision fusion audio-visual speech recognition. Journal of Signal Processing Systems, Online First, 2011.
Sullivan, G. J., & Wiegand, T. (2005). Video compression-from concepts to the H.264/AVC standard. Proceedings of the IEEE, 93, 18–31.
Gonzalez, R. C., & Woods, R. E. (2002). Digital image processing (2nd ed.). New Jersey: Prentice Hall.
Cheng, M. H., Chen, H. Y., & Leou, J. J. (2011). Video super-resolution reconstruction using a mobile search strategy and adaptive patch size. Signal Processing, 91, 1284–1297.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Cheng, MH., Hwang, KS., Jeng, JH. et al. PSO-based Fusion Method for Video Super-Resolution. J Sign Process Syst 73, 25–42 (2013). https://doi.org/10.1007/s11265-012-0725-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-012-0725-z