Skip to main content
Springer Nature Link
Log in

Low-energy motion estimation memory system with dynamic management

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The digital video coding process imposes severe pressure on memory traffic, leading to considerable power consumption related to frequent DRAM accesses. External off-chip memory demand needs to be minimized by clever architecture/algorithm co-design, thus saving energy and extending battery lifetime during video encoding. To exploit temporal redundancies among neighboring frames, the motion estimation (ME) algorithm searches for good matching between the current block and blocks within reference frames stored in external memory. To save energy during ME, this work performs memory accesses distribution analysis of the test zone search (TZS) ME algorithm and, based on this analysis, proposes both a multi-sector scratchpad memory design and dynamic management for the TZS memory access. Our dynamic memory management, called neighbor management, reduces both static consumption—by employing sector-level power gating—and dynamic consumption—by reducing the number of accesses for ME execution. Additionally, our dynamic management was integrated with two previously proposed solutions: a hardware reference frame compressor and the Level C data reuse scheme (using a scratchpad memory). This system achieves a memory energy consumption savings of \(99.8\%\) and, when compared to the baseline solution composed of a reference frame compressor and data reuse scheme, the memory energy consumption was reduced by \(44.1\%\) at a cost of just \(0.35\%\) loss in coding efficiency, on average. When compared with related works, our system presents better memory bandwidth/energy savings and coding efficiency results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Cisco: Cisco visual networking index: forecast and trends, 2017–2022 (2018). https://www.cisco.com/. Accessed 20 Dec 2019

  2. Nokia: Network traffic insights in the time of COVID-19 (2020). https://www.nokia.com/blog/network-traffic-insights-time-covid-19-april-9-update/. Accessed 6 July 2020

  3. Mutlu, O., Subramanian, L.: Research problems and opportunities in memory systems. Supercomput. Front. Innovat. 1(3), 19–55 (2015)

    Google Scholar 

  4. H.265: ITU-T recommendation H.265: high efficiency video coding, audiovisual and multimedia systems (2013). https://www.itu.int/rec/T-REC-H.265. Accessed 20 Dec 2019

  5. Sullivan, G.J., Ohm, J., Han, W.J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. Trans. Circ. Syst. Video Technol. 22(12), 1649–1668 (2012)

    Article  Google Scholar 

  6. Shafique, M., Garg, S., Henkel, J., Marculescu, D.: The EDA challenges in the dark silicon era: temperature, reliability, and variability perspectives. In: Design Automation Conference (2014). https://doi.org/10.1145/2593069.2593229

  7. Correa, G., Assuncao, P.A., Agostini, L.V., da Silva Cruz, L.A.: Fast HEVC encoding decisions using data mining. Trans. Circ. Syst. Video Technol. 25(4), 660–673 (2015). https://doi.org/10.1109/TCSVT.2014.2363753

    Article  Google Scholar 

  8. Tikekar, M., Huang, C., Juvekar, C., Sze, V., Chandrakasan, A.P.: A 249-Mpixel/s HEVC video-decoder chip for 4K ultra-HD applications. J. Solid-State Circ. 49(1), 61–72 (2014)

    Article  Google Scholar 

  9. Khan, M.U.K., Shafique, M., Henkel, J.: AMBER: adaptive energy management for on-chip hybrid video memories. In: International Conference on Computer-Aided Design (2013). https://doi.org/10.1109/ICCAD.2013.6691150

  10. Tang, X., Dai, S., Cai, C.: An analysis of TZSearch algorithm in JMVC. In: International Conference on Green Circuits and Systems (2010). https://doi.org/10.1109/ICGCS.2010.5543008

  11. Mativi, A., Monteiro, E., Bampi,S.: Memory access profiling for HEVC encoders. In: Latin American Symposium on Circuits Systems (2016). https://doi.org/10.1109/LASCAS.2016.7451055

  12. Zatt, B., Shafique, M., Sampaio, F., Agostini, L., Bampi, S., Henkel, J.: Run-time adaptive energy-aware motion and disparity estimation in multiview video coding. In: Design Automation Conference (2011). https://doi.org/10.1145/2024724.2024950

  13. Shafique, M., Zatt, B., Walter, F.L., Bampi, S., Henkel, J.: Adaptive power management of on-chip video memory for multiview video coding. In: Design Automation Conference (2012). https://doi.org/10.1145/2228360.2228516

  14. Tuan, J.-C., Chang, T.-S., Jen, C.-W.: On the data reuse and memory bandwidth analysis for full-search block-matching VLSI architecture. Trans. Circ. Syst. Video Technol. 12(1), 61–72 (2002). https://doi.org/10.1109/76.981846

    Article  Google Scholar 

  15. Silveira, D., Povala, G., Amaral, L., Zatt, B., Agostini, L., Porto, M.: Efficient reference frame compression scheme for video coding systems: algorithm and VLSI design. J. Real-Time Image Process. 16(2), 391–411 (2019)

    Article  Google Scholar 

  16. Lian, X., Liu, Z., Zhou, W., Duan, Z.: Parallel content-aware adaptive quantization-oriented lossy frame memory recompression for HEVC. Trans. Circ. Syst. Video Technol. 28(4), 958–971 (2018)

    Article  Google Scholar 

  17. Willème, A., Macq, B., Descampe, A., Rouvroy, G.: Power-aware HEVC compression through asymmetric JPEG XS frame buffer compression. In: International Conference on Image Processing (2018). https://doi.org/10.1109/ICIP.2018.8451539

  18. Lee, Y.-H., Chen, C.-C., You, Y.-L.: Design of VLSI architecture of autocorrelation-based lossless recompression engine for memory-efficient video coding systems. Springer Circ. Syst. Signal Process. 3(2), 459–482 (2014)

    Article  Google Scholar 

  19. Lian, X., Liu, Z., Zhou, W., Duan, Z.: Lossless frame memory compression using pixel-grain prediction and dynamic order entropy coding. Trans. Circ. Syst. Video Technol. 26(1), 223–235 (2016)

    Article  Google Scholar 

  20. Chen, C.-H., Huang, C.-T., Chen, Y.-H., Chen, L.-G.: Level C+ data reuse scheme for motion estimation with corresponding coding orders. Trans. Circ. Syst. Video Technol. 16(4), 553–558 (2006). https://doi.org/10.1109/TCSVT.2006.871388

    Article  Google Scholar 

  21. Dai, W., Au, O.C., Li, S., Sun, L., Zou, R.: Adaptive search range algorithm based on Cauchy distribution. In: Visual Communications and Image Processing (2012). https://doi.org/10.1109/VCIP.2012.6410741

  22. Du, L., Liu, Z., Ikenaga, T., Wang, D.: Linear adaptive search range model for uni-prediction and motion analysis for bi-prediction in HEVC. In: International Conference on Image Processing (2014). https://doi.org/10.1109/ICIP.2014.7025745

  23. Chien, W., Liao, K., Yang, J.: Enhanced AMVP mechanism based adaptive motion search range decision algorithm for fast HEVC coding. In: International Conference on Image Processing (2014). https://doi.org/10.1109/ICIP.2014.7025750

  24. Li, Y., Liu, Y., Yang, H., Yang, D.: An adaptive search range method for HEVC with the k-nearest neighbor algorithm. In: Visual Communications and Image Processing (2015). https://doi.org/10.1109/VCIP.2015.7457794

  25. Ji, X., Jia, H., Liu, J., Xie, X., Gao, W.: Computation-constrained dynamic search range control for real-time video encoder. Image Commun. 31, 134–150 (2015)

    Google Scholar 

  26. Pakdaman, F., Gabbouj, M., Hashemi, M.R., Ghanbari, M.: Fast motion estimation algorithm with efficient memory access for HEVC hardware encoders. In: European Workshop on Visual Information Processing (2018). https://doi.org/10.1109/EUVIP.2018.8611766

  27. Singh, K., Rafi Ahamed, S.: Low power motion estimation algorithm and architecture of HEVC/H.265 for consumer applications. Trans. Consumer Electron. 64(3), 267–275 (2018). https://doi.org/10.1109/TCE.2018.2867823

    Article  Google Scholar 

  28. Huang, Y.-W., Chen, C.-Y., Tsai, C.-H., Shen, C.-F., Chen, L.-G.: Survey on block matching motion estimation algorithms and architectures with new results. J. VLSI Signal Process. Syst. Signal Image Video Technol. 42, 297–320 (2006)

    Article  Google Scholar 

  29. High Efciency Video Coding (HEVC): Reference software (2018). https://hevc.hhi.fraunhofer.de. Accessed 15 Dec 2019

  30. Afonso, V., Conceição, R., Saldanha, M., Braatz, L., Perleberg, M., Corrêa, G., Porto, M., Agostini, L., Zatt, B., Susin, A.: Energy-aware motion and disparity estimation system for 3D-HEVC with run-time adaptive memory hierarchy. Trans. Circ. Syst. Video Technol. 29(6), 1878–1892 (2019)

    Article  Google Scholar 

  31. Bossen, F., Bross, B., Suhring, K., Flynn, D.: HEVC complexity and implementation analysis. Trans. Circ. Syst. Video Technol. 22(12), 1685–1696 (2012)

    Article  Google Scholar 

  32. Jeong, K., Kahng, A.B., Kang, S., Rosing, T.S., Strong, R.: MAPG: memory access power gating. In: Design, Automation Test in Europe Conference (2012). https://doi.org/10.1109/DATE.2012.6176651

  33. Bossen, F.: Common test conditions and software reference configurations. Document JCTVC-L1100, ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 Joint Collaborative Team on Video Coding (JCT-VC) (2013)

  34. Amaral, L., Povala, G., Porto, M., Silveira, D., Bampi, S.: Memory energy consumption analyzer for video encoder hardware architectures. In: International Conference on Electronics, Circuits and Systems (2016). https://doi.org/10.1109/ICECS.2016.7841203

  35. Sharman, K., Suhring, K.: Common test conditions for HM. Document JCTVC-Z1100, ITU-T SG16 and ISO/IEC/JTC1/SC29/WG11 Joint Collaborative Team on Video Coding (JCT-VC) (2017)

  36. Micron: Micron MT46H64M16LF: 1 Gb DDR SDRAM (2019). https://www.micron.com/. Accessed 15 Dec 2019

  37. CACTI: HP CACTI 6.5 (2018). https://www.hpl.hp.com/research/cacti/. Accessed 15 Dec 2019

  38. Bjontegaard, G.: Improvements of the BD-PSNR model. In: VCEGAI11, ITU-T SG16/Q6 VCEG 35th meeting, Berlin, Germany, pp. 16–18 (2008)

Download references

Acknowledgements

This study was supported by the Federal Institute of Education, Science and Technology of Rio Grande do Sul (IFRS), Fundação CAPES Finance Code 01, FAPERGS, and CNPq Brazilian agencies for R&D support.

Author information

Author notes

  1. Dieison Soares Silveira and Livia Amaral are equally-credited first authors.

Authors and Affiliations

  1. Federal Institute of Education, Science and Technology of Rio Grande do Sul and Graduate Program in Microelectronics, Federal University of Rio Grande do Sul, Porto Alegre, Brazil

    Dieison Soares Silveira

  2. Federal University of Pelotas, Pelotas, Brazil

    Lívia Amaral, Guilherme Povala, Bruno Zatt, Luciano Volcan Agostini & Marcelo Schiavon Porto

  3. Informatics Institute at the Federal University of Rio Grande do Sul, Porto Alegre, Brazil

    Sergio Bampi

Authors
  1. Dieison Soares Silveira
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Lívia Amaral
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Guilherme Povala
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Bruno Zatt
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Luciano Volcan Agostini
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Marcelo Schiavon Porto
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Sergio Bampi
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Dieison Soares Silveira.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Silveira, D.S., Amaral, L., Povala, G. et al. Low-energy motion estimation memory system with dynamic management. J Real-Time Image Proc 18, 2495–2510 (2021). https://doi.org/10.1007/s11554-021-01138-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-021-01138-3

Keywords