Tianxiao Han
ABSTRACT
With the more and more common use of infrared video in life, people have higher and higher requirements for infrared imaging frame rates. Better quality infrared images provide a better basis for subsequent target recognition, video compression and decompression operations. Therefore, it is of great significance to effectively obtain infrared high frame rate images and improve the quality of infrared images. As an effective means of video conversion, frame rate enhancement technology has become a research hotspot in the direction of computer vision. However, the existing video interpolation methods based on infrared images are all implemented based on block matching, which cannot well approximate the complex moving real world. In order to solve these problems, an optical flow method based on pixel motion compensation is proposed for infrared video interpolation. The second interpolation method can make better use of the motion information in the video. In the end, an optical flow optimization network is used for optimization, which can better optimize the artifacts in the optical flow estimation and improve the final image quality. Experiments show that our method has a better effect than existing methods on models on various infrared video data sets, and has lower computational complexity.
- Salton G, McGill M J. Introduction to modern information retrieval[J]. 1986.Google Scholar
Digital Library
- Sullivan G J, Ohm J, Han W-J, Overview of the high efficiency video coding (HEVC) standard[J]. IEEE Transactions on circuits and systems for video technology, 2012, 22(12) : 1649 –1668.Google ScholarDigital Library
- Bauer D , Bonacker M , Cavonius C R . Frame repetition rate for flicker-free viewing of bright VDU screens[J]. Displays, Technology and Applications, 1983, 4(1):31-33.Google Scholar
- Richard C, David D, Yadong L I. MOTION ADAPTIVE FRAME AVERAGING FOR ULTRASOUND DOPPLER COLOR FLOW IMAGING:, US 7153268 B2[P]. 2006.Google Scholar
- Yan N, Liu D, Li H, A convolutional neural network approach for halfpel interpolation in video coding[C] // Circuits and Systems (ISCAS), 2017 IEEE International Symposium on. 2017 : 1–4.Google Scholar
- Chen Z, He T, Jin X, Learning for Video Compression[J]. arXiv preprint arXiv:1804.09869, 2018]Google Scholar
- Dosovitskiy A, Fischery P, Ilg E, FlowNet: Learning Optical Flow with Convolutional Networks[C]// IEEE International Conference on Computer Vision. 2015.Google Scholar
- Ilg E, Mayer N, Saikia T, FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks[C]// IEEE Conference on Computer Vision & Pattern Recognition. 2017.Google Scholar
- Zhou T, Tulsiani S, Sun W, View synthesis by appearance flow[C] // European conference on computer vision. 2016 : 286–301.Google Scholar
- Liu Z, Yeh R, Tang X, Video frame synthesis using deep voxel flow[C]// International Conference on Computer Vision (ICCV) : Vol 2. 2017.Google Scholar
- Niklaus S, Mai L, Liu F. Video frame interpolation via adaptive separable convolution[J]. arXiv preprint arXiv:1708.01692, 2017.Google Scholar
- Niklaus S, Liu F. Context-aware Synthesis for Video Frame Interpolation[J]. 2018.Google ScholarCross Ref
- Jiang H, Sun D, Jampani V, Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation[J]. 2017.Google Scholar
- Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[J]. arXiv preprint arXiv:1502.03167, 2015.Google Scholar
- Santurkar S, Tsipras D, Ilyas A, How Does Batch Normalization Help Optimization?(No, It Is Not About Internal Covariate Shift)[J]. arXiv preprint arXiv:1805.11604, 2018.Google Scholar
- Horn B K P, Schunck B G. Determining optical flow[J]. Artificial Intelligence, 1980, 17(1–3):185-203.Google ScholarDigital Library
- Lucas B D, Kanade T. Optical navigation by the method of differences[C]// International Joint Conference on Artificial Intelligence. 1985.Google Scholar
- Fleet DJ, Jepson A D. Computation of component image velocity from local phase information[J]. International Journal of Computer Vision, 1990, 5(1):77-104.Google ScholarDigital Library
- Mahajan D, Huang F C, Matusik W, Moving gradients: a path-based method for plausible image interpolation[C]// Acm Siggraph. 2009.Google ScholarDigital Library
- N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman. Phase-based video motion processing. ACM Trans. Graph., 32(4):80:1–80:10, 2013.Google ScholarDigital Library
- P. Didyk, P. Sitthi-amorn, W. T. Freeman, F. Durand, and W. Matusik. Joint view expansion and filtering for automultiscopic 3D displays. ACM Trans. Graph., 32(6):221:1–221:8, 2013.Google ScholarDigital Library
- Meister S, Hur J, Roth S. UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss[J]. 2017.Google Scholar
- Liu P , Lyu M , King I , SelFlow: Self-Supervised Learning of Optical Flow[J]. 2019.Google Scholar
- Hur J , Roth S . Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation[J]. 2019.Google ScholarCross Ref
- Long G , Kneip L , Alvarez J M , Learning Image Matching by Simply Watching Video[C]// European Conference on Computer Vision. Springer, Cham, 2016.Google Scholar
- Frame Rate Up-Conversion Algorithm Based on Infrared Image
Recommendations
Adaptive frame rate up-conversion based on motion classification
In this paper, a new technique on video frame rate up-conversion (FRUC) is presented by combining the adaptive motion classification (AMC) for image sequences and the mixed motion estimation (ME). In the proposed FRUC scheme, the AMC classifies ...
Multiframe-based bilateral motion estimation with emphasis on stationary caption processing for frame rate up-conversion
In this paper, we present a new motion compensated frame rate up-conversion algorithm that uses multiframes to enhance the accuracy of motion estimation. We also develop adaptive motion vector smoothing to correct outliers in a motion vector field. In ...
Dual Motion Estimation for Frame Rate Up-Conversion
In this letter, we present a new motion estimation algorithm for frame rate up-conversion. The proposed dual motion estimation algorithm enhances the estimation accuracy of motion vectors by using the unidirectional and bidirectional matching ratios of ...
Comments