Skip to main content
SpringerLink
Log in

On Predicting Bottlenecks in Wavefront Parallel Video Coding Using Deep Neural Networks

  • Conference paper
  • First Online:
Engineering Applications of Neural Networks (EANN 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1000))

  • 1944 Accesses

Abstract

Video coding incurs high computational complexity particularly at the encoder side. For this reason, parallelism is used at the various encoding steps. One of the popular coarse grained parallelization tools offered by many standards is wavefront parallelism. Under the scheme, each row of blocks is assigned to a separate thread for processing. A thread might commence encoding a particular block once certain precedence constraints are met, namely, it is required that the left block of the same row and the top and top-right block of the previous row have finished compression. Clearly, the imposed constraints result in processing delays. Therefore, in order to optimize performance, it is of paramount importance to properly identify potential bottlenecks before the compression of a frame starts, in order to alleviate them through better resource allocation. In this paper we present a simulation model that predicts bottlenecks based on the estimated block compression times produced from a regression neural network. Experiments with datasets obtained using the reference encoder of HEVC (High Efficiency Video Coding) illustrate the merits of the proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Wiegand, T., Sullivan, G.J., Bjontegaard, G., Luthra, A.: Overview of the H. 264/AVC video coding standard. IEEE Trans. Circ. Syst. Video Technol. 13, 560–576 (2003)

    Article  Google Scholar 

  2. YouTube, Recommended upload encoding settings. https://support.google.com/youtube/answer/1722171

  3. Sullivan, G.J., Ohm, J.R., Han, W.J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circ. Syst. Video Technol. 22, 1649–1668 (2012)

    Article  Google Scholar 

  4. AV1: Bitstream & Decoding Process Specification (2018). https://aomedia.org/av1-bitstream-and-decoding-process-specification/

  5. Topiwala, P., Krishnan, M., Dai, W.: Performance comparison of VVC, AV1 and HEVC on 8-bit and 10-bit content. In: Applications of Digital Image Processing XLI International Society for Optics and Photonics, vol. 10752, p. 107520 (2018)

    Google Scholar 

  6. VVC: Versatile Video Coding (2018). https://jvet.hhi.fraunhofer.de/

  7. Franche, J.F., Coulombe, S.: A multi-frame and multi-slice H. 264 parallel video encoding approach with simultaneous encoding of prediction frames. In: 2012 2nd International Conference on Consumer Electronics, Communications and Networks (CECNet), pp. 3034–3038. IEEE (2012)

    Google Scholar 

  8. Lemmetti, A., Koivula, A., Viitanen, M., Vanne, J., Hämäläinen, T.D.: AVX2-optimized Kvazaar HEVC intra encoder. In: IEEE International Conference on Image Processing (ICIP), pp. 549–553 (2016)

    Google Scholar 

  9. Koziri, M.G., Papadopoulos, P., Tziritas, N., Dadaliaris, A.N., Loukopoulos, T., Khan, S.U.: Slice-based parallelization in HEVC encoding: realizing the potential through efficient load balancing. In: 18th IEEE International Workshop on Multimedia Signal Processing (MMSP), pp. 1–6 (2016)

    Google Scholar 

  10. Misra, K.M., Segall, C.A., Horowitz, M., Xu, S., Fuldseth, A., Zhou, M.: An overview of tiles in HEVC. IEEE J. Sel. Top. Signal Process. 7(6), 969–977 (2013)

    Article  Google Scholar 

  11. Chi, C.C., et al.: Parallel scalability and efficiency of HEVC parallelization approaches. IEEE Trans. Circ. Syst. Video Technol. 22, 1827–1838 (2012)

    Article  Google Scholar 

  12. x265 HEVC encoder (2018). http://x265.org

  13. Qiu, X., Zhang, L., Ren, Y., Suganthan, P.N., Amaratunga, G.: Ensemble deep learning for regression and time series forecasting. In: 2014 IEEE Symposium on Computational Intelligence in Ensemble Learning, pp. 1–6 (2014)

    Google Scholar 

  14. HM reference software. http://hevc.hhi.fraunhofer.de

  15. Zhao, L., Xu, J., Zhou, Y., Ai, M.: A dynamic slice control scheme for slice-parallel video encoding. In: ICIP 2012, pp. 713–716 (2012)

    Google Scholar 

  16. Shafique, M., Khan, M.U.K., Henkel, J.: Power efficient and workload balanced tiling for parallelized high efficiency video coding. In: ICIP 2014, pp. 1253–1257 (2014)

    Google Scholar 

  17. Storch, I., Palomino, D., Zatt, B., Agostini, L.: Speedup-aware history-based tiling algorithm for the HEVC standard. In: ICIP 2016, pp. 824–828 (2016)

    Google Scholar 

  18. Koziri, M., et al.: Adaptive tile parallelization for fast video encoding in HEVC. In: Proceedings of the 12th International Conference on Green Computing and Communications (GreenCom), pp. 738–743 (2016)

    Google Scholar 

  19. Koziri, M., et al.: Heuristics for tile parallelism in HEVC. In: 25th European Signal Processing Conference (EUSIPCO), pp. 1514–1518 (2017)

    Google Scholar 

  20. Blumenberg, C., Palomino, D., Bampi, S., Zatt, B.: Adaptive content-based tile partitioning algorithm for the HEVC standard. In: PCS 2013, pp. 185–188 (2013)

    Google Scholar 

  21. Papadopoulos, P.K., Koziri, M.G., Loukopoulos, T.: A fast heuristic for tile partitioning and processor assignment in HEVC. In: Proceedings of the IEEE International Conference on Image Processing (2018)

    Google Scholar 

  22. Zhao, Z., Liang, P.: Data partition for wavefront parallelization of H. 264 video encoder. In: Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), p. 4 (2006)

    Google Scholar 

  23. Zhao, Y., Song, L., Wang, X., Chen, M., Wang, J.: Efficient realization of parallel HEVC intra encoding. In: Proceedings of the IEEE International Conference Multimedia and Expo Workshops (ICMEW), pp. 1–6 (2013)

    Google Scholar 

  24. Wang, Z.Y., Dong, S.F., Wang, R.G., Wang, W.M., Gao, W.: Dynamic macroblock wavefront parallelism for parallel video coding. J. Vis. Commun. Image Represent. 28, 36–43 (2015)

    Article  Google Scholar 

  25. Wen, Z., Guo, B., Liu, J., Li, J., Lu, Y., Wen, J.: Novel 3D-WPP algorithms for parallel HEVC encoding. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1471–1475 (2016)

    Google Scholar 

  26. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)

    Article  Google Scholar 

  27. Lv, Y., Duan, Y., Kang, W., Li, Z., Wang, F.Y.: Traffic flow prediction with big data: a deep learning approach. IEEE Trans. Intell. Transp. Syst. 16(2), 865–873 (2015)

    Google Scholar 

  28. Bossen, F.: Common test conditions and software reference configurations. In: Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 5th meeting (2011)

    Google Scholar 

  29. Frank, E., Hall, M.A., Witten, I.H.: The WEKA Workbench. Online Appendix for “Data Mining: Practical Machine Learning Tools and Techniques, 4th edn. Morgan Kaufmann, Burlington (2016)

    Google Scholar 

Download references

Acknowledgments

This research has been co-financed by the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH-CREATE-INNOVATE (project code: T1EDK-02070).

Author information

Authors and Affiliations

  1. Department of Computer Science and Telecommunications, University of Thessaly, Papasiopoulou 2-4, 35131, Lamia, Greece

    Natalia Panagou, Panagiotis Oikonomou & Maria Koziri

  2. Department of Computer Science and Biomedical Informatics, University of Thessaly, Papasiopoulou 2-4, 35131, Lamia, Greece

    Panos K. Papadopoulos, Thanasis Loukopoulos & Dimitris Iakovidis

Authors
  1. Natalia Panagou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Panagiotis Oikonomou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Panos K. Papadopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maria Koziri
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Thanasis Loukopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dimitris Iakovidis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thanasis Loukopoulos .

Editor information

Editors and Affiliations

  1. David Goldman Informatics Centre, University of Sunderland, Sunderland, UK

    John Macintyre

  2. Democritus University of Thrace, Xanthi, Greece

    Lazaros Iliadis

  3. University of Piraeus, Piraeus, Greece

    Ilias Maglogiannis

  4. Oxford Brookes University, Oxford, UK

    Chrisina Jayne

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Panagou, N., Oikonomou, P., Papadopoulos, P.K., Koziri, M., Loukopoulos, T., Iakovidis, D. (2019). On Predicting Bottlenecks in Wavefront Parallel Video Coding Using Deep Neural Networks. In: Macintyre, J., Iliadis, L., Maglogiannis, I., Jayne, C. (eds) Engineering Applications of Neural Networks. EANN 2019. Communications in Computer and Information Science, vol 1000. Springer, Cham. https://doi.org/10.1007/978-3-030-20257-6_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20257-6_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20256-9

  • Online ISBN: 978-3-030-20257-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation