Skip to main content
SpringerLink
Log in

Mode skipping for screen content coding based on Neural Network Classifier

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The Screen Content Coding Extension in High-Efficiency Video Coding standard (HEVC-SCC) promotes the capabilities of HEVC in coding screen content videos (SCVs) using new techniques, which improves coding efficiency dramatically. These new techniques depend on the distinguished features of SCV such as repeated patterns, limited number of colors, sharp edges, and non-noisy regions. Nonetheless, this coding efficiency comes at the cost of enormous computational complexity. In this paper, a new technique is proposed to save encoding time while conserving coding efficiency. The proposed algorithm selects the suitable mode for each Coding Unit (CU) and skips unhelpful modes by two methods. Two methods depend on skipping unwanted modes by Neural Network Classifiers. The first classifier is Neural Network Classifier Based on Current Depth Features (NNC_CF), which depends on the CU current depth features. The second one is Neural Network Classifier Based on Parent Depth Features (NNC_PF); the Parent depth features are considered the input of this classifier. The simulation results demonstrate the efficacy of the proposed scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Sullivan, G., Ohm, J., Han, W., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22, 1649–1668 (2012)

    Article  Google Scholar 

  2. Ma, Z., Wang, W., Xu, M., Yu, H.: Advanced screen content coding using color table and index map. IEEE Trans. Image Process. 23, 4399–4412 (2014)

    Article  MathSciNet  Google Scholar 

  3. Xu, J., Joshi, R., Cohen, R.: Overview of the emerging HEVC screen content coding extension. IEEE Trans. Circuits Syst. Video Technol. 26, 50–62 (2016)

    Article  Google Scholar 

  4. Joshi, R.: Screen Content Coding Test Model 4 Encoder Description (SCM 4), document JCTVC-T1014, Warsaw, PL, (2015).

  5. Jiang, Q., Shao, F., Lin, W., Gu, K., Jiang, G., Sun, H.: Optimizing multistage discriminative dictionaries for blind image quality assessment. IEEE Trans. Multimed. 20, 2035–2048 (2017)

    Article  Google Scholar 

  6. Jiang, Q., Shao, F., Lin, W., Jiang, G.: Learning a referenceless stereopair quality engine with deep nonnegativity constrained sparse autoencoder. Pattern Recognit 76, 242–255 (2018)

    Article  Google Scholar 

  7. Chang, T. S. RCE3: Results of Subtest B. 1 on Nx2N/2NxN Intra Block Copy, document JCTVC-P0176. San Jose, CA (2014).

  8. Pang, C., Sole, J., Guo, L., Karczewicz, M. RCE3: Subtest B. 3—Intra Block Copy With NxN PU, document JCTVC-P0145. San Jose, CA (2014).‏

  9. Chen, C. C., Xu, X., Liao, R. L., Peng, W. H., Liu, S., & Lei, S.: Screen content coding using non-square intra block copy for HEVC. In: IEEE International Conference on Multimedia and Expo (ICME), pp.1–6 (2014).

  10. Zhang, K., An, J., Zhang, X., Huang, H., Lei, S. Symmetric intra block copy in video coding. In: IEEE International Symposium on Circuits and Systems, pp. 521–524 (2015).

  11. Xu, X., Liu, S., Chuang, T. Der, Lei, S.: Block vector prediction for intra block copying in HEVC screen content coding. In: Data Compression Conference, pp. 273–282 (2015).

  12. Sun, Y. C., Chuang, T. D., Lai, P., Chen, Y. W., Liu, S., Huang, Y. W., Lei, S.: Palette mode—a new coding tool in screen content coding extensions of HEVC. In: IEEE International Conference on Image Process. ICIP, pp. 2409–2413 (2015).

  13. Guo, L., Pu, W., Zou, F., Sole, J., Karczewicz, M., Joshi, R.: Color palette for screen content coding. In: IEEE International Conference on Image Process, ICIP 2014, pp. 5556–5560 (2014). Doi:https://doi.org/10.1109/ICIP.2014.7026124

  14. Z hang, L., Chen, J., Sole, J., Karczewicz, M., Xiu, X., He, Y., Ye, Y.: SCCE5 test 3.2.1: in-loop color-space transform. document JCTVC-R0147.

  15. Zhang, Y., Kwong, S., Zhang, G., Pan, Z., Yuan, H., Jiang, G.: Low complexity HEVC INTRA coding for high-quality mobile video communication. IEEE Trans. Ind. Informatics 11, 1492–1504 (2015)

    Article  Google Scholar 

  16. Li, B., Xu, J., Sullivan, G. J., Zhou, Y., & Lin, B.: Adaptive motion vector resolution for screen content. document JCTVC-S0085 (2014).

  17. Li, X., Sole, J., & Karczewicz, M.: Adaptive MV Precision for Screen Content Coding. document JCTVC-P0283 (2014).

  18. Badry, E., Shalaby, A., Sayed, M. S.: Fast algorithm with palette mode skipping and splitting early termination for HEVC screen content coding. In: Midwest Symposium on Circuits Systems (MWSCAS), pp. 606–609 (2019).

  19. Tsang, S., Chan, Y., Siu, W.: Fast and efficient intra coding techniques for smooth regions in screen content coding based on boundary prediction samples. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1409–1413 (2015).

  20. Zhang, M., Zhang, Y., Bai, H.: Fast CU splitting in HEVC intra coding for screen content coding. IEICE Trans. Inf. Syst. 98, 467–470 (2015)

    Article  Google Scholar 

  21. Tsang, S., Kuang, W., Chan, Y., Siu, W.: Fast HEVC screen content coding by skipping unnecessary checking of intra block copy mode based on CU activity and gradient. In: Asia-Pacific Signal and Information Processing Association Annual Summit Conference APSIPA 2016 (2017). Doi:https://doi.org/10.1109/APSIPA.2016.7820900.

  22. Lei, J., Li, D., Pan, Z., Sun, Z., Kwong, S., Hou, C.: Fast intra prediction based on content property analysis for low complexity HEVC-based screen content coding. IEEE Trans. Broadcast. 63, 48–58 (2016)

    Article  Google Scholar 

  23. Badry, E., Shalaby, A. Sayed, M.: Intra mode decision acceleration for HEVC screen content coding. In: Proceedings of the International Japan-Africa Conference on Electronics, Communications and Computations, JAC-ECC 92–95 (2019). Doi:https://doi.org/10.1109/JAC-ECC48896.2019.9051167.

  24. Tsang, S.H., Chan, Y.L., Kuang, W., Siu, W.C.: Reduced-complexity intra block copy (IntraBC) mode with early CU splitting and pruning for HEVC screen content coding. IEEE Trans. Multimed. 21, 269–283 (2018)

    Article  Google Scholar 

  25. Zeng, H., Xiang, W., Chen, J., Cai, C., Ni, Z., Ma, K.K.: Unimodal model-based inter mode decision for high efficiency video coding. IEEE Access 7, 27936–27947 (2019)

    Article  Google Scholar 

  26. Li, T., Xu, M. Deng, X. A deep convolutional neural network approach for complexity reduction on intra-mode HEVC. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 1255–1260 (2017). Doi:https://doi.org/10.1109/ICME.2017.8019316.

  27. Liu, X., Li, Y., Liu, D., Wang, P., Yang, L.T.: An adaptive CU size decision algorithm for HEVC intra prediction based on complexity classification using machine learning. IEEE Trans. Circuits Syst. Video Technol. 29, 144–155 (2017)

    Article  Google Scholar 

  28. Laude, T. Ostermann, J. Deep learning-based intra prediction mode decision for HEVC. In: Picture Coding Symposium (PCS), pp. 1–5 (2017).. Doi:https://doi.org/10.1109/PCS.2016.7906399.

  29. Kuang, W., Chan, Y., Tsang, S., Siu, W.: DeepSCC: deep learning-based fast prediction network for screen content coding. IEEE Trans. Circuits Syst. Video Technol. 30, 1917–1932 (2019)

    Google Scholar 

  30. Kuang, W., Chan, Y., Tsang, S.: Efficient intra bitrate transcoding for screen content coding based on convolutional neural network. IEEE Access 7, 107211–107224 (2019)

    Article  Google Scholar 

  31. Peng, W., Walls, F.G., Cohen, R.A., Xu, J., Ostermann, J., MacInnis, A., Lin, T.: Overview of screen content video coding technologies standards and beyond. IEEE J. Emerg. Sel. Top. Circuits Syst. 6, 393–408 (2016)

    Article  Google Scholar 

  32. Huang, C., Peng, Z., Chen, F., Jiang, Q., Jiang, G., Hu, Q.: Efficient CU and PU decision based on neural network and gray level co-occurrence matrix for intra prediction of screen content coding. IEEE Access 6, 46643–46655 (2018)

    Article  Google Scholar 

  33. H. Yu, R. Cohen, K. Rapaka, J. X.: Common test conditions for screen content coding. document JCTVC-R1015, Sapporo, Japan (2015).

  34. Xu, Y., Zhu, K.: Cost sensitive learning based HEVC screen content intra coding for mobile devices. Mob. Networks Appl. 25, 2471–2481 (2020)

    Article  Google Scholar 

  35. Kuang, W., Chan, Y., Tsang, S., Siu, W.: Fast HEVC to SCC transcoder by early CU partitioning termination and decision tree-based flexible mode decision for intra-frame coding. IEEE Access 7, 8773–8788 (2019)

    Article  Google Scholar 

  36. Bjontegaard, G.: Calculation of average PSNE differences between R-D Curves, document VCEG-M33, ITU-T VCEG (2001).

  37. HEVC Test Model Version 16.18 Screen Content Model Version 8.7. [Online]. Available:https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.18+SCM-8.7/. Accessed 14 May 2021

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communications Engineering, Zagazig University, Zagazig, Egypt

    Nabila Elsawy, Mohammed S. Sayed & Fathi Farag

  2. Department of Electronics and Communications Engineering, Egypt-Japan University of Science and Technology, New Borg El Arab City, Egypt

    Mohammed S. Sayed

Authors
  1. Nabila Elsawy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammed S. Sayed
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fathi Farag
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nabila Elsawy.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Elsawy, N., Sayed, M.S. & Farag, F. Mode skipping for screen content coding based on Neural Network Classifier. J Real-Time Image Proc 18, 2453–2468 (2021). https://doi.org/10.1007/s11554-021-01137-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-021-01137-4

Keywords

Search

Navigation