Jarrett Ethan Singian
ABSTRACT
With the increased use of closed-circuit television (CCTV) footage for security and surveillance purposes as well as for object or person recognition and efficiency monitoring, high-quality CCTV videos are necessary. In this paper, we propose Corgi Eye, a moving object removal + super-resolution framework for enhancing CCTV footages to remove ghosting artifacts caused by performing multi-frame super-resolution (MISR) on moving objects. Our method extends the framework of Eagle Eye, which is an existing MISR framework tailored for mobile devices. Our results demonstrate that the system can completely remove ghosting effects caused by moving objects while performing MISR on CCTV footage. Our proposed method demonstrates competitive performance when compared to Eagle Eye, achieving a 16% increase in terms of PSNR metric. Additionally, our method can produce clear images, on par with deep learning approaches such as ESPCN and SOF-VSR.
Supplemental Material
- Jiwoong Bang, Daewon Kim, and Hyeonsang Eom. 2012. Motion object and regional detection method using block-based background difference video frames. In 2012 IEEE International Conference on Embedded and Real-Time Computing Systems and Applications. IEEE, 350–357.Google Scholar
Digital Library
- Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte. 2021. Deep burst super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9209–9218.Google ScholarCross Ref
- S. Chiu, C. C. Chiu, and S. S. D. Xu. 2018. A background subtraction algorithm in complex environments based on category entropy analysis. Applied Sciences 8, 6 (2018), 885. https://doi.org/10.3390/app8060885Google ScholarCross Ref
- N Del Gallego and J. Ilao. 2017. Multiple-image super-resolution on mobile devices: An image warping approach. Journal of Imaging 8, 2017 (2017). https://doi.org/10.1186/s13640-016-0156-zGoogle Scholar
- N. Del Gallego and J. Ilao. 2018. Improving multiple-image super-resolution for mobile devices through image alignment selection. Journal of WSCG 26, 2 (2018), 122–131.Google ScholarCross Ref
- C. Ding, A. Kamal, G. Denina, H. Nguyen, A. Ivers, B. Varda, C. Ravishankar, B. Bhanu, and A. Roy-Chowdhury. 2010. Videoweb Activities Dataset, ICPR contest on Semantic Description of Human Activities (SDHA). http://cvrc.ece.utexas.edu/SDHA2010/Wide_Area_Activity.html.Google Scholar
- Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2015. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence 38, 2(2015), 295–307.Google Scholar
- Chao Dong, Chen Change Loy, and Xiaoou Tang. 2016. Accelerating the super-resolution convolutional neural network. In European conference on computer vision. Springer, 391–407.Google ScholarCross Ref
- A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. 2013. Vision meets robotics: The KITTI dataset. International Journal of Robotics Research (IJRR). http://www.cvlibs.net/datasets/kitti/raw_data.phpGoogle Scholar
- Daniel Glasner, Shai Bagon, and Michal Irani. 2009. Super-resolution from a single image. In 2009 IEEE 12th international conference on computer vision. IEEE, 349–356.Google ScholarCross Ref
- A. Godbehere, A. Matsukawa, and K. Goldberg. 2012. Visual tracking of human visitors under variable-lighting conditions for a responsive audio art installation. In 2012 American Control Conference (ACC). IEEE.Google Scholar
- N. Goyette, P.-M. Jodoin, J. Konrad, and P. Ishwar. 2012. Changedetection.net: A new change detection benchmark dataset. 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. http://jacarini.dinf.usherbrooke.ca/dataset2012/Google Scholar
- Wei Han, Shiyu Chang, Ding Liu, Mo Yu, Michael Witbrock, and Thomas S Huang. 2018. Image super-resolution via dual-state recurrent networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1654–1663.Google ScholarCross Ref
- Muhammad Haris, Gregory Shakhnarovich, and Norimichi Ukita. 2019. Recurrent back-projection network for video super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3897–3906.Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.Google ScholarCross Ref
- Michal Irani and Shmuel Peleg. 1991. Improving resolution by image registration. CVGIP: Graphical models and image processing 53, 3(1991), 231–239.Google Scholar
- Takashi Isobe, Xu Jia, Shuhang Gu, Songjiang Li, Shengjin Wang, and Qi Tian. 2020. Video super-resolution with recurrent structure-detail network. In European Conference on Computer Vision. Springer, 645–660.Google ScholarDigital Library
- Julio Cezar Silveira Jacques, Claudio Rosito Jung, and Soraia Raupp Musse. 2006. A background subtraction model adapted to illumination changes. In 2006 International Conference on Image Processing. IEEE, 1817–1820.Google ScholarCross Ref
- Reza Javadzadeh, Ehsan Banihashemi, and Javad Hamidzadeh. 2015. Fast vehicle detection and counting using background subtraction technique and prewitt edge detection. International Journal of Computer Science and Telecommunications 6, 10(2015), 8–12.Google Scholar
- Kyong Hwan Jin, Michael T McCann, Emmanuel Froustey, and Michael Unser. 2017. Deep convolutional neural network for inverse problems in imaging. IEEE Transactions on Image Processing 26, 9 (2017), 4509–4522.Google ScholarDigital Library
- S. Kaur. 2017. Background Subtraction in Video Surveillance. Electronic Theses and Dissertations(2017), 5944.Google Scholar
- Michal Kawulok, Pawel Benecki, Szymon Piechaczek, Krzysztof Hrynczenko, Daniel Kostrzewa, and Jakub Nalepa. 2019. Deep learning for multiple-image super-resolution. IEEE Geoscience and Remote Sensing Letters 17, 6 (2019), 1062–1066.Google ScholarCross Ref
- P. Li, L. Prieto, D. Mery, and P. J. Flynn. 2019. Face recognition in low quality images: A survey. ACM Comput. Surv 1, 1 (2019).Google Scholar
- Bee Lim and Kyoung Mu Lee. 2017. Deep recurrent ResNet for video super-resolution. In 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 1452–1455.Google ScholarCross Ref
- Ce Liu and Deqing Sun. 2011. A bayesian approach to adaptive video super resolution. In CVPR 2011. IEEE, 209–216.Google ScholarDigital Library
- LY Liu, N Sang, and R Huang. 2010. Background subtraction using shape and colour information. Electronics letters 46, 1 (2010), 41–43.Google Scholar
- Xiang Ma, Junping Zhang, and Chun Qi. 2010. Hallucinating face by position-patch. Pattern Recognition 43, 6 (2010), 2224–2236.Google ScholarDigital Library
- Dennis Mitzel, Thomas Pock, Thomas Schoenemann, and Daniel Cremers. 2009. Video super resolution using duality based tv-l 1 optical flow. In Joint Pattern Recognition Symposium. Springer, 432–441.Google ScholarCross Ref
- Anaswara S Mohan and R Resmi. 2014. Video image processing for moving object detection and segmentation using background subtraction. In 2014 First International Conference on Computational Systems and Communications (ICCSC). IEEE, 288–292.Google ScholarCross Ref
- SeungJong Noh and Moongu Jeon. 2012. A new framework for background subtraction using multiple cues. In Asian Conference on Computer Vision. Springer, 493–506.Google Scholar
- S. Oh, A. Hoogs, A. Perera, N. Cuntoor, C. Chen, J.T. Lee, S. Mukherjee, J.K. Aggarwal, H. Lee, L. Davis, E. Swears, X. Wang, Q. Ji, K. Reddy, Shah M., C. Vondrick, H. Pirsiavasha, D. Ramanan, J. Yuen, A. Torralba, B. Song, A. Fong, A. Roy-Chowdhury, and Desai M.2011. A Large-scale Benchmark Dataset for Event Recognition in Surveillance Video. Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR). https://viratdata.org/Google Scholar
- T. Qasim, R.B. Fisher, and N. Bhatti. 2021. Ground-truthing Large Human Behavior Monitoring Datasets.Google Scholar
- M. S. Ryoo and J. K. Aggarwal. 2010. UT-Interaction Dataset, ICPR contest on Semantic Description of Human Activities (SDHA). http://cvrc.ece.utexas.edu/SDHA2010/Human_Interaction.html.Google Scholar
- Mehdi SM Sajjadi, Raviteja Vemulapalli, and Matthew Brown. 2018. Frame-recurrent video super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6626–6634.Google ScholarCross Ref
- Francesco Salvetti, Vittorio Mazzia, Aleem Khaliq, and Marcello Chiaberge. 2020. Multi-Image Super Resolution of Remotely Sensed Images Using Residual Attention Deep Neural Networks. Remote Sensing 12, 14 (2020). https://doi.org/10.3390/rs12142207Google Scholar
- Wenzhe Shi, Jose Caballero, Ferenc Huszar, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. (2016). https://arxiv.org/pdf/1609.05158.pdfGoogle Scholar
- Pierre-Luc St-Charles, Guillaume-Alexandre Bilodeau, and Robert Bergevin. 2015. SuBSENSE: A Universal Change Detection Method With Local Adaptive Sensitivity. IEEE Transactions on Image Processing 24, 1 (2015).Google ScholarCross Ref
- Alexandru Telea. 2004. An image inpainting technique based on the fast marching method. Journal of graphics tools 9, 1 (2004), 23–34.Google ScholarCross Ref
- Longguang Wang, Yulan Guo, Zaiping Lin, Xinpu Deng, and Wei An. 2018. Learning for video super-resolution through HR optical flow estimation. In Asian Conference on Computer Vision. Springer, 514–529.Google Scholar
- Longguang Wang, Yulan Guo, Li Liu, Zaiping Lin, Xinpu Deng, and Wei An. 2019. Deep video super-resolution using HR optical flow estimation. (2019). https://arxiv.org/pdf/2001.02129.pdfGoogle Scholar
- L. Wang, Y. Guo, L. Liu, Z. Lin, X. Deng, and W. An. 2020. Deep video super-resolution using HR optical flow estimation. arXiv (6 Jan. 2020).Google Scholar
- Longguang Wang, Yulan Guo, Li Liu, Zaiping Lin, Xinpu Deng, and Wei An. 2020. Deep video super-resolution using HR optical flow estimation. IEEE Transactions on Image Processing 29 (2020), 4323–4336.Google ScholarCross Ref
- W. Wang, C. Ren, X. He, H. Chen, and L. Qing. 2018. Video super-resolution via residual learning. arXiv 6(2018), 23767–23777.Google Scholar
- Greg Ward. 2003. Fast, robust image registration for compositing high dynamic range photographs from hand-held exposures. Journal of graphics tools 8, 2 (2003), 17–30.Google ScholarCross Ref
- Chih-Yuan Yang, Sifei Liu, and Ming-Hsuan Yang. 2013. Structured face hallucination. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1099–1106.Google ScholarDigital Library
- Chih-Yuan Yang, Chao Ma, and Ming-Hsuan Yang. 2014. Single-Image Super-Resolution: A Benchmark. In Computer Vision – ECCV 2014, David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars (Eds.). Springer International Publishing, Cham, 372–386.Google ScholarCross Ref
- Xin Yu, Basura Fernando, Richard Hartley, and Fatih Porikli. 2018. Super-Resolving Very Low-Resolution Face Images with Supplementary Attributes. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 908–917.Google Scholar
- Qiangqiang Yuan, Liangpei Zhang, Huanfeng Shen, and Pingxiang Li. 2010. Adaptive multiple-frame image super-resolution based on U-curve. IEEE Transactions on Image Processing 19, 12 (2010), 3157–3170.Google ScholarDigital Library
- D. Zeng, X. Chen, M. Zhu, M. Goesele, and A. Kuijper. 2019. Background subtraction with real-time semantic segmentation. IEEE Access 7(2019), 153869–153884. https://doi.org/10.1109/ACCESS.2019.2899348Google ScholarCross Ref
Recommendations
Moving Object Detection Using Three-Frame Difference and Background Subtraction
ICECC '12: Proceedings of the 2012 International Conference on Electronics, Communications and ControlMoving object detection is an important research content in video processing. Frames subtraction and background subtraction are commonly used methods to detect moving objects. They are both simple and effective, but have some limitations. Background ...
Background subtraction based on logarithmic intensities
Background subtraction is widely used as the basis for moving object extraction from image sequences. For traditional background subtraction, the standard intensities in interested images are compared to those in the reference image. As standard ...
Background subtraction driven seeds selection for moving objects segmentation and matting
In this paper, we address the difficult task of moving objects segmentation and matting in dynamic scenes. Toward this end, we propose a new automatic way to integrate a background subtraction (BGS) and an alpha matting technique via a heuristic seeds ...
Comments