Skip to main content
SpringerLink
Log in

Multilevel Residual Motion Compensation for High Efficiency Video Coding

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

The motion estimation, which is combined with the motion compensation in the decoding stage, is a fundamental method to achive high compression efficiency in video coding. The method achieves high compression efficiency by reducing temporal redundancy between adjacent pictures. Temporarily adjacent residual pictures also contain some redundancy even though the motion estimation technique removes some of the redundant information. Based on this characteristic, this paper proposes a multilevel residual MC (MRMC) algorithm, which repeats ME and MC on the residual signals in order to incorporate the possibility of further reducing the redundancy in the residual information. This paper mainly deals with the trade-off between the number of bits for residual signals and their corresponding motion vectors (MVs) based on the rate-distortion optimization. In addition, a fast method is presented for the practical use of the proposed MRMC method. The proposed method is implemented on the High Efficiency Video Coding software test model version 16.0. Experimental results show that an average Bjøntegaard-delta rate savings of 9.16% can be achieved at a cost of increase of 17.0% in encoding time and 2.3% in decoding time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13

Similar content being viewed by others

References

  1. Sullivan, G. J., Ohm, J.-R., Han, W.-J., & Wiegand, T. (2012). Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology, 22(12), 1649–1668.

    Article  Google Scholar 

  2. Video Codec for Audiovisual Services at px64 kbit/s. ITU-T Recommendation H.261 (1990).

  3. Video Coding for Low Bit Rate Communication. ITU-T Recommendation H.263 (1995).

  4. Coding of Moving Pictures and Associated Audio for Digital Storage Media at up to about 1.5 Mbit/s—Part 2: Video. ISO/IEC 11172–2 (MPEG-1) (1993).

  5. Coding of Audio-Visual Objects—Part 2: Visual. ISO/IEC 14496–2 (MPEG-4 Visual version 1). (1999).

  6. Generic Coding of Moving Pictures and Associated Audio Information—Part 2: Video. ITU-T Recommendation H.262 and ISO/IEC 13818–2 (MPEG 2 Video) (1994).

  7. Advanced Video Coding for Generic Audio-Visual Services. ITU-T Recommendation H.264 and ISO/IEC 14496–10 (AVC) (2003).

  8. Wiegand, T., Sullivan, G. J., Bjøntegaard, G., & Luthra, A. (2003). Overview of the H.264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology, 13(7), 560–576.

    Article  Google Scholar 

  9. Ciuperca, G., Girardin, V., & Lhote, L. (2011). Computation and estimation of generalized entropy rates for denumerable Markov chains. IEEE Transactions on Information Theory, 57(7), 4026–4034.

    Article  MathSciNet  MATH  Google Scholar 

  10. Sullivan, G. J. & Baker, R. L. (1991). Rate-distortion optimized motion compensation for video compression using fixed or variable size blocks. International Conference on Global Telecommunications, (pp. 85–90).

  11. Girod, B. (1993). Motion-compensating prediction with fractional-pel accuracy. IEEE Transactions on Consumer Electronics, 41(4), 604–612.

    Google Scholar 

  12. Wiegand, T., Zhang, X., & Girod, B. (1999). Long-term memory motion compensated prediction. IEEE Transactions on Circuits and Systems for Video Technology, 9(1), 70–84.

    Article  Google Scholar 

  13. Wiegand, T., Schwarz, H., Joch, A., Kossentini, F., & Sullivan, G. J. (2003). Rate-constrained coder control and comparison of video coding standards. IEEE Transactions on Circuits and Systems for Video Technology, 13(7), 688–703.

    Article  Google Scholar 

  14. Schwarz, H., Marpe, D., & Wiegand, T. (2007). Overview of the scalable video coding extension of the H.264/AVC standard. IEEE Transactions on Circuits and Systems for Video Technology, 17(9), 1103–1120.

    Article  Google Scholar 

  15. Kim, H., Cosman, P. C., & Milstein, L. B. (2013). Motion- compensated scalable video transmission over MIMO wireless channels. IEEE Transactions on Circuits and Systems for Video Technology, 23(1), 116–127.

    Article  Google Scholar 

  16. Schwarz, H., Marpe, D., & Wiegand, T. (2005). Hierarchical B Pictures. Joint Video Team (JVT) Document JVT-P014.

  17. Schwarz, H., Hinz, T., Marpe, D., & Wiegand, T. (2005). Constrained interlayer prediction for single-loop decoding in spatial scalability. IEEE International Conference on Image Processing (pp. 870–873).

  18. Bross, B., Han, W.-J., Ohm, J.-R., Sullivan, G. J., Wang, Y.-K., & Wiegand, T. (2013). High Efficiency Video Coding (HEVC) Text Specification Draft 10. Joint Collaborative Team on Video Coding (JCT-VC) Document JCTVC-L1003.

  19. Ohm, J., Sullivan, G. J., Schwarz, H., Tan, T. K., & Wiegand, T. (2012). Comparison of the coding efficiency of video coding standards—including high efficiency video coding (HEVC). IEEE Transactions on Circuits and Systems for Video Technology, 22(12), 1669–1684.

    Article  Google Scholar 

  20. McCann, K., Han, W.-J., & Kim, I.-K. (2010). Samsung’s Response to the Call for Proposals on Video Compression Technology. Joint Collaborative Team on Video Coding (JCT-VC) Document JCTVC-A124.

  21. McCann K. (2010). Tool Experiment 12: Evaluation of TMuC Tools. Joint Collaborative Team on Video Coding (JCT-VC) Document JCTVC-B312.

  22. K. McCann, W.-J. Han, & I.-K. Kim (2010). Samsung’s Response to the Call for Proposals on Video Compression Technology. Joint Collaborative Team on Video Coding (JCT-VC) Document JCTVC-A124.

  23. Kuo, T.-Y., & Lu, H.-J. (2008). Efficient reference frame selector for H.264. IEEE Transactions on Circuits and Systems for Video Technology, 18(3), 400–405.

    Article  Google Scholar 

  24. K. Choi, S.-H. Park, & E. S. Jang (2011). Coding tree pruning based CU early termination. Joint Collaborative Team on Video Coding (JCT-VC) Document JCTVC-F092.

  25. J. Yang, J. Kim, K. Won, H. Lee, & B. Jeon (2011). Early SKIP Detection for HEVC. Joint Collaborative Team on Video Coding (JCT-VC) Document JCTVC-G543.

  26. R. H. Gweon, Y.-L. Lee, & J. Lim (2011). Early Termination of CU Encoding to Reduce HEVC Complexity. Joint Collaborative Team on Video Coding (JCT-VC) Document JCTVC-F045.

  27. Kamp, S., & Wien, M. (2012). Decoder-side motion vector derivation for block-based video coding. IEEE Transactions on Circuits and Systems for Video Technology, 22(12), 1732–1745.

    Article  Google Scholar 

  28. Peng, W.-H., & Chen, C.-C. (2013). An interframe prediction technique combining template matching prediction and block-motion compensation for high-efficiency video coding. IEEE Transactions on Circuits and Systems for Video Technology, 23(8), 1432–1446.

    Article  Google Scholar 

  29. Kim, H.-S., Lee, J.-H., Kim, C.-K., & Kim, B.-G. (2012). Zoom motion estimation using block-based fast local area scaling. IEEE Transactions on Circuits and Systems for Video Technology, 22(9), 1280–1291.

    Article  Google Scholar 

  30. Kordasiewicz, R. C., Gallant, M. D., & Shirani, S. (2007). Encoding of affine motion vectors. IEEE Transactions on Multimedia, 9(7), 1346–1356.

    Article  Google Scholar 

  31. Kim, D.-Y., Lim, H.-G., & Park, H.-W. (2013). Iterative true motion estimation for motion-compensated frame interpolation. IEEE Transactions on Circuits and Systems for Video Technology, 23(3), 445–454.

    Article  Google Scholar 

  32. Tsai, T.-H., & Lin, H.-Y. (2013). Hybrid frame rate upconversion method based on motion vector mapping. IEEE Transactions on Circuits and Systems for Video Technology, 23(11), 1901–1910.

    Article  Google Scholar 

  33. Kang, J.-W., Lou, C.-C., Kim, S.-H., & Kuo, C.-C. J. (2013). Efficient HD video coding with joint first-order-residual (FOR) and second-order-residual (SOR) coding technique. Journal of Visual Communication and Image Representation, 24(1), 1–11.

    Article  Google Scholar 

  34. Kamisli, F. & Lim, J. S. (2009). Transforms for the motion compensation residual. IEEE International Conference on Acoustics Speech and Signal Processing (pp. 789–792).

  35. Kang, J.-W., Gabbouj, M., & Kuo, C.-C. J. (2013). Sparse/DCT (S/DCT) two-layered representation of prediction residuals for video coding. IEEE Transactions on Image Processing, 22(7), 2711–2722.

    Article  MathSciNet  Google Scholar 

  36. Gu, Z., Lin, W., Lee, B.-S., & Lau, C. T. (2012). Rotated orthogonal transform (ROT) for motion-compensation residual coding. IEEE Transactions on Image Processing, 21(12), 4770–4781.

    Article  MathSciNet  Google Scholar 

  37. McCann, K., Rosewarne, C., Bross, B., Naccari, M., Sharman, K., & Sullivan, G. J. (2014). High Efficiency Video Coding (HEVC) Test Model 16 (HM 16) Improved Encoder Description. Joint Collaborative Team on Video Coding (JCT-VC) Document JCTVC-S1002.

  38. Bossen, F., Bross, B., Suhring, K., & Flynn, D. (2012). HEVC complexity and implementation analysis. IEEE Transactions on Circuits and Systems for Video Technology, 22(12), 1685–1696.

    Article  Google Scholar 

  39. Chen, M.-J., Li, G.-L., Chiang, Y.-Y., & Hsu, C.-T. (2006). Fast multiframe motion estimation algorithms by motion vector composition for the MPEG-4/AVC/H.264 standard. IEEE Transactions on Multimedia, 8(3), 478–487.

    Article  Google Scholar 

  40. Po, L.-M., Ting, C.-W., Wong, K.-M., & Ng, K.-H. (2007). Novel point-oriented inner searches for fast block motion estimation. IEEE Transactions on Multimedia, 9(1), 9–15.

    Article  Google Scholar 

  41. Jin, S.-J., Park, S.-J., & Jeong, J.-C. (2007). Adaptive fast full search algorithm using partitioned region and optimized search order. IEEE Transactions on Consumer Electronics, 53(4), 1703–1711.

    Article  Google Scholar 

  42. Kannangara, C. S., Richardson, I. E., Bystrom, M., & Zhao, Y. (2009). Complexity control of H.264/AVC based on mode-conditional cost probability distributions. IEEE Transactions on Multimedia, 11(3), 433–442.

    Article  Google Scholar 

  43. Shen, L., Liu, Z., Zhang, X., Zhao, W., & Zhang, Z. (2013). An effective CU size decision method for HEVC encoders. IEEE Transactions on Multimedia, 15(2), 465–470.

    Article  Google Scholar 

  44. Xiong, J., Li, H., Wu, Q., & Meng, F. (2014). A fast HEVC inter CU selection method based on pyramid motion divergence. IEEE Transactions on Multimedia, 16(2), 559–564.

    Article  Google Scholar 

  45. Tao, B., & Orchard, M. T. (2001). A parametric solution for optimal overlapped block motion compensation. IEEE Transactions on Image Processing, 10(3), 341–350.

    Article  MATH  Google Scholar 

  46. Tseng, Y.-C., Wu, C.-H., Chen, Y.-W., Wang, T.-W., & Peng W.-H. (2010). On the analysis and design of motion sampling structure for advanced motion-compensated prediction. IEEE International Conference on Image Processing (pp. 949–952).

  47. Chen, Y.-W., & Peng, W.-H. (2012). Parametric OBMC for pixel-adaptive temporal prediction on irregular motion sampling grids. IEEE Transactions on Circuits and Systems for Video Technology, 22(1), 113–127.

    Article  Google Scholar 

  48. Bossen, F. (2013). Common Test Conditions and Software Reference Configurations. Joint Collaborative Team on Video Coding (JCT-VC) Document JCTVC-L1100.

  49. Cho, S., & Kim, M. (2013). Fast CU splitting and pruning for suboptimal CU partitioning in HEVC intra coding. IEEE Transactions on Circuits and Systems for Video Technology, 23(9), 1555–1564.

    Article  Google Scholar 

  50. Bjøntegaard G. (2008). Improvements of the BD-PSNR model. Video Coding Experts Group (VCEG) Document VCEG-AI11.

Download references

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, Sogang University, Seoul, 121-742, South Korea

    Hyo-Song Kim & Rae-Hong Park

Authors
  1. Hyo-Song Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rae-Hong Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rae-Hong Park.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, HS., Park, RH. Multilevel Residual Motion Compensation for High Efficiency Video Coding. J Sign Process Syst 89, 363–377 (2017). https://doi.org/10.1007/s11265-016-1207-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-016-1207-5

Keywords

Search

Navigation