Skip to main content
Springer Nature Link
Log in

FPGA-based implementation of the VVC low-frequency non-separable transform

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The Versatile Video Coding (VVC) standard, released in July 2020, brings better coding performance than the High-Efficiency Video Coding (HEVC) thanks to the introduction of new coding tools. The transform module in the VVC standard incorporates the Multiple Transform Selection (MTS) concept, which relies on separable Discrete Cosine Transform (DCT)/Discrete Sine Transform (DST) kernels, and the recently introduced Low-Frequency Non-Separable Transform (LFNST). This latter serves as a secondary transform process, enhancing coding efficiency by further decorrelating residual samples. However, it introduces heightened computational complexity and substantial resource allocation demands, potentially complicating its hardware implementation. This paper introduces an effective and cost-efficient hardware architecture for LFNST. The proposed design employs additions and bit-shifting operations preserving hardware logic usage. The synthesis results for an Arria 10 10AX115N1F45E1SG FPGA device demonstrate that the logic cost is only of 26% of the available hardware resources. Additionally, the proposed design is working at 204 MHz and can process Ultra High Definition (UHD) 4K videos at up to 60 frames per second (fps).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

No datasets were generated or analyzed during the current study.

References

  1. Hamidouche, W., Biatek, T., Abdoli, M., et al.: Versatile video coding standard: a review from coding tools to consumers deployment. IEEE Consum. Electron. Mag. 11(5), 10–24 (2022)

    Article  Google Scholar 

  2. Akramullah, S.: Digital Video Concepts, Methods, and Metrics: Quality, Compression, Performance, and Power Trade-Off Analysis. Springer Nature, Berlin (2014)

    Book  Google Scholar 

  3. Shen, Y.-F., Li, J.-T., Zhu, Z.-M., et al.: High efficiency video coding. Chin. J. Comput. 11(36), 2340–2355 (2013)

    Google Scholar 

  4. Garrido, M.J., Pescador, F., Chavarrías, M., et al.: A high performance FPGA-based architecture for the future video coding adaptive multiple core transform. IEEE Trans. Consum. Electron. 64(1), 53–60 (2018)

    Article  Google Scholar 

  5. Bross, B.: Versatile video coding. https://www.digitalmedia.fraunhofer.de/content/dam/dcinema/en/documents/ibc2020/2020_Flyer_Versatile%20Video%20Coding_final.pdf (2020). Accessed Jan 2024

  6. Bossen, F., Sühring, K., Wieckowski, A., et al.: VVC complexity and software implementation analysis. IEEE Trans. Circuits Syst. Video Technol. 31(10), 3765–3778 (2021)

    Article  Google Scholar 

  7. Saldanha, M., Corrêa, M., Corrêa, G., et al.: An overview of dedicated hardware designs for state-of-the-art AV1 and H.266/VVC video codecs. In: 2020 27th IEEE International Conference on Electronics, Circuits and Systems (ICECS), pp. 1–4. IEEE (2020)

  8. Wiegand, T., Sullivan, G.J., Bjontegaard, G., et al.: Overview of the H.264/AVC video coding standard. IEEE Trans. Circuits Syst. Video Technol. 13(7), 560–576 (2003)

    Article  Google Scholar 

  9. Ahmed, N., Natarajan, T., Rao, K.: Discrete cosine transform. IEEE Trans. Comput. C–23(1), 90–93 (1974)

    Article  MathSciNet  Google Scholar 

  10. Wang, Z.: Fast discrete sine transform algorithms. Signal Process. 19(2), 91–102 (1990)

    Article  MathSciNet  Google Scholar 

  11. Sullivan, G.J., Ohm, J., Han, W., et al.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22, 1649–1668 (2012)

    Article  Google Scholar 

  12. Chen, J., Ye, Y., Kim, S.: Algorithm description for versatile video coding and test model 6 (VTM 6). In: JVET Document O2002 (JVET-O2002), 15th JVET Meeting: Gothenburg, SE (2019)

  13. Koo, M., Salehifar, M., Lim, J., et al.: Low frequency non-separable transform (LFNST). In: 2019 Picture Coding Symposium (PCS), pp. 1–5 (2019)

  14. Zhao, X., Chen, J., Karczewicz, M., et al.: Enhanced multiple transform for video coding. In: Data Compression Conference (DCC), pp. 73–82 (2016)

  15. De-Luxán-Hernández, S., George, V., Ma, J., et al.: An intra subpartition coding mode for VVC. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1203–1207. IEEE (2019)

  16. Zhao, X., Kim, S.-H., Zhao, Y., et al.: Transform coding in the VVC standard. IEEE Trans. Circuits Syst. Video Technol. 31(10), 3878–3890 (2021)

    Article  Google Scholar 

  17. Browne, A., Ye, Y., Kim, S.-H.: Algorithm description for versatile video coding and test model 17 (VTM 17). In: Draft Document: JVET-Z2002-v2, 26th JVET Meeting, by Teleconference (2022)

  18. Garrido, M.J., Pescador, F., Chavarrías, M., et al.: A 2-D multiple transform processor for the versatile video coding standard. IEEE Trans. Consum. Electron. 65, 274–283 (2019)

    Article  Google Scholar 

  19. Garrido, M.J., Pescador, F., Chavarrías, M., et al.: An FPGA-based architecture for the versatile video coding multiple transform selection core. IEEE Access 8, 81887–81903 (2020)

    Article  Google Scholar 

  20. Hao, Z., Zheng, Q., Fan, Y., et al.: An area-efficient unified transform architecture for VVC. In: 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2012–2016. IEEE (2022)

  21. Hao, Z., Sun, H., Xiang, G., et al.: A reconfigurable multiple transform selection architecture for VVC. IEEE Trans. Very Large Scale Integr. (VLSI) Syst. 31(5), 658–669 (2023)

    Article  Google Scholar 

  22. Ben Jdidia, S., Belghith, F., Masmoudi, N.: A high-performance two-dimensional transform architecture of variable block sizes for the VVC standard. J. Real-Time Image Process. 19(6), 1081–1090 (2022)

    Article  Google Scholar 

  23. Farhat, I., Hamidouche, W., Grill, A., et al.: Lightweight hardware transform design for the versatile video coding 4k ASIC decoders. IEEE Trans. Consum. Electron. 67(4), 329–340 (2021)

    Article  Google Scholar 

  24. Goebel, J., Costa, V., Agostini, L., et al,: A high-throughput design for the H. 266/VVC low-frequency non-separable transform. In: 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1798–1802. IEEE (2022)

  25. Goebel, J., Agostini, L., Zatt, B., et al.: Low-frequency non-separable transform hardware system design for the VVC encoder. In: 2022 35th SBC/SBMicro/IEEE/ACM Symposium on Integrated Circuits and Systems Design (SBCCI), pp. 1–6. IEEE (2022)

  26. Zatt, B., Agostini, L., Porto, M., et al.: Efficient dedicated hardware design system for the VVC low-frequency non-separable transform. J. Integr. Circuits Syst. 18(1), 1–11 (2023)

    Article  Google Scholar 

  27. VVC Test Model (VTM) software version 17.1. https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/tree/VTM-17.1?ref_type=tags

  28. Mentor-modelsim. functional-verification-tool-web. https://www.mentor.com/products/fv/modelsim (2019)

  29. Intel/Altera: Intel-arria-10-device-overview. https://www.altera.com/documentation/sam1403480274650.html (2018)

Download references

Author information

Authors and Affiliations

  1. Université de Sfax, ENIS, LETI, Sfax, 3038, Tunisie

    Fatma Belghith, Sonda Ben Jdidia, Bouthaina Abdallah & Nouri Masmoudi

Authors
  1. Fatma Belghith
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Sonda Ben Jdidia
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Bouthaina Abdallah
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Nouri Masmoudi
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

All authors listed on the submission have made equal contributions to the accomplishment of this work.

Corresponding author

Correspondence to Fatma Belghith.

Ethics declarations

Conflict of interest

The authors declare no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Belghith, F., Ben Jdidia, S., Abdallah, B. et al. FPGA-based implementation of the VVC low-frequency non-separable transform. J Real-Time Image Proc 21, 94 (2024). https://doi.org/10.1007/s11554-024-01471-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-024-01471-3

Keywords