Skip to main content
Springer Nature Link
Log in

PCA-based hierarchical clustering approach for motion vector estimation in H.265/HEVC video error concealment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video error concealment (EC) is a challenging issue, and one of the key challenges is accurately modeling motion vectors (MVs) in corrupted frames. To address this problem, we propose a novel Principal Component Analysis (PCA)-based model for EC that uses hierarchical clustering for MV estimation. Our approach has two main steps: first, we obtain the MV fields represented in the 2D plane, and second, we conceal the erroneous regions of the frame using a PCA-based hierarchical clustering algorithm. This algorithm selects the best MVs among the field candidates while maximizing the correlation of MVs in a cluster. Our approach outperforms recent related techniques in terms of PSNR and SSIM for the H.265/HEVC compression standard. Specifically, our method improves average PSNR by up to 4.99 dB and average SSIM by 0.029. Furthermore, our method has slightly lower computational complexity compared to the compared techniques, especially for videos with relatively uniform motion over the missing areas.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability Statement

All data generated or analyzed during this study are included in this published article, and its supplementary information files are available in the Zenodo repository, 10.5281/zenodo.7756564.

References

  1. Adeyemi-Ejeye AO, Alreshoodi M, Al-Jobouri L, Fleury M (2019) Impact of packet loss on 4K UHD video for portable devices. Multimed Tools Appl 78(22):31733–31755

    Article  Google Scholar 

  2. Balzano L, Chi Y, Lu YM (2018) Streaming PCA and Subspace Tracking: The Missing Data Case. Proc IEEE 106(8):1293–1310

    Article  Google Scholar 

  3. Chang YL, Reznik YA, Chen Z, Cosman PC (2013) Motion Compensated Error Concealment for HEVC Based on Block-Merging and Residual Energy. In 20th International Packet Video Workshop, pp 1–6

  4. Chen Y, Wang H, Wu H et al (2018) A video error concealment method using data hiding based on compressed sensing over lossy channel. Telecommun Syst 68(2):337–349

    Article  Google Scholar 

  5. Choe G, Nam C, Chu C (2018) An effective temporal error concealment in H.264 video sequences based on scene change detection-PCA model. Multimedia Tools and Applications 77(24):31953–31967

  6. Chung B, Yim C (2020) Bi-Sequential Video Error Concealment Method Using Adaptive Homography-Based Registration. IEEE Trans Circuits Syst Video Technol 30(6):1535–1549

    Article  Google Scholar 

  7. Coding HEV (2021) High Efficiency Video Coding: Recommendation ITU-T H.265. International Standard ISO/IEC 23008-23002. https://handle.itu.int/11.1002/1000/14660 Accessed 7 March 2023

  8. Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute(HHI). https://hevc.hhi.fraunhofer.de/ Accessed 18 Mar 2023

  9. Fujiwara T, Chou JK, Shilpika S, Xu P, Ren L, Ma KL (2020) An Incremental Dimensionality Reduction Method for Visualizing Streaming Multidimensional Data. IEEE Transactions on Visualization and Computer Graphics 26(1):418–428

    Article  PubMed  Google Scholar 

  10. Gutub A (2022) Boosting image watermarking authenticity spreading secrecy from counting-based secretsharing. CAAI Trans. on Intelligence Technology

  11. Hameed A, Dai R, Balas B (2016) A Decision-Tree-Based Perceptual Video Quality Prediction Model and Its Application in FEC for Wireless Multimedia Communications. IEEE Trans Multimed 18(4):764–774

    Article  Google Scholar 

  12. Hojati S, Kazemi M, Moallem P (2020) Error concealment with parallelogram partitioning of the lost area. Multimed Tools Appl 79(11):7449–7469

    Article  Google Scholar 

  13. Hsia SC, Hsiao CH (2016) Fast-efficient shape error concealment technique based on block classification. IET Image Process. 10(10):693–700

    Article  Google Scholar 

  14. Huang Z, Cai Q, Xiao X (2018) A Video Data Recovery Algorithm in Wireless Communication Networks. In 2018 IEEE 18th Int Conf Commun Technol (ICCT), pp 727–731

  15. Kazemi M (2021) In favor of fully intra coding for HEVC video transmission over lossy channels. Signal, Image and Video Processing 15(1):165–173

    Article  Google Scholar 

  16. Kazemi M, Ghanbari M, Shirmohammadi S (2020) The Performance of Quality Metrics in Assessing Error-Concealed Video Quality. IEEE Transactions on Image Processing 29:5937–5952

    Article  ADS  Google Scholar 

  17. Kazemi M, Ghanbari M, Shirmohammadi S (2021) A review of temporal video error concealment techniques and their suitability for HEVC and VVC. Multimed Tools Appl 80(8):12685–12730

    Article  Google Scholar 

  18. Korhonen J (2018) Study of the subjective visibility of packet loss artifacts in decoded video sequences. IEEE Trans. Broadcast 64(2):354–366

    Article  Google Scholar 

  19. Lei Y (2022) Research on microvideo character perception and recognition based on target detection technology. J Comput Cog Eng 1(2):83–87

    MathSciNet  Google Scholar 

  20. Li ZN, Drew MS, Liu J (2021) Multimedia Over Wireless and Mobile Networks. Fundamentals of Multimedia, pp 627–669

  21. Li Y, Chen R (2017) Motion vector recovery for video error concealment based on the plane fitting. Multimed. Tools Appl. 76(13):14993–15006

    Article  Google Scholar 

  22. Lin TL, Yang NC, Syu RH, Liao CC, Tsai WL (2013) Error concealment algorithm for HEVC coded video using block partition decisions. In 2013 IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC 2013), pp 1–5

  23. Lin T, Wei X, Su T, Chiang Y (2018) Novel pixel recovery method based on motion vector disparity and compensation difference. IEEE Access 6:44362–44375

    Article  Google Scholar 

  24. Ma R, Li T, Bo D, Wu Q, An P (2020) Error sensitivity model based on spatial and temporal features. Multimed Tools Appl 79(43–44):31913–31930

    Article  Google Scholar 

  25. Montgomery C (2023) Xiph.org. Derf’s Test Media Collection. Available at: https://media.xiph.org/video/derf/. Accessed 7 March 2023

  26. Murtagh F, Contreras P (2017) Algorithms for hierarchical clustering: an overview, II. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 7(6):e1219

    Google Scholar 

  27. Nam C, Chu C, Kim T, Han S (2020) A novel motion recovery using temporal and spatial correlation for fast temporal error concealment over H.264 video sequences. Multimedia Tools and Applications 79(1):1221–1240

  28. Radmehr A, Ghasemi A (2016) Error concealment via particle filter by Gaussian mixture modeling of motion vectors for H.264/AVC. Signal, Image, and Video Processing 10(2):311–318

  29. Ros F, Guillaume S (2019) A hierarchical clustering algorithm and an improvement of the single linkage criterion to deal with noise. Expert Syst Appl 128:96–108

    Article  Google Scholar 

  30. Sankisa A, Punjabi A, Katsaggelos AK (2018) Video Error Concealment Using Deep Neural Networks. In 2018 25th IEEE International Conference on Image Processing (ICIP), pp 380–384

  31. Sara U, Akter M, Uddin MS (2019) Image Quality Assessment through FSIM, SSIM, MSE and PSNR-A Comparative Study. J Comput Commun 7(3):8–18

    Article  Google Scholar 

  32. Tiwari V, Bhatnagar C (2021) A survey of recent work on video summarization: approaches and techniques. Multimed Tools Appl 80(18):27187–27221

    Article  Google Scholar 

  33. Usman M, He X, Xu M, Lam KM (2015) Survey of Error Concealment techniques: Research directions and open issues. In 2015 Picture Coding Symposium (PCS), pp 233–238

  34. Wani A, Khaliq R (2021) SDN-based intrusion detection system for IoT using deep learning classifier (IDSIoT-SDL). CAAI Trans. on Intelligence Technology 6(3):281–290

    Article  Google Scholar 

  35. Wu J, Cheng B, Wang M, Chen J (2017) Priority-Aware FEC Coding for High-Definition Mobile Video Delivery Using TCP. IEEE Trans Mob Comput 16(4):1090–1106

    Article  Google Scholar 

  36. Xiang C, Xu J, Yan C, Peng Q, Wu X (2019) Generative adversarial networks based error concealment for low resolution video. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp 1827–1831

  37. Xu J, Jiang W, Yan C, Peng Q, Wu X (2018) A Novel Weighted Boundary Matching Error Concealment Schema for HEVC. In 25th IEEE International Conference on Image Processing (ICIP), pp 3294–3298

  38. Zhang B, Cosman P, Milstein LB (2019) Energy Optimization For Wireless Video Transmission Employing Hybrid ARQ. IEEE Trans. Veh Technol 68(6):5606–5617

    Article  Google Scholar 

  39. Zhou S, Xu Z, Liu F (2017) Method for Determining the Optimal Number of Clusters Based on Agglomerative Hierarchical Clustering. IEEE Trans Neural Netw Learn Syst 28(12):3007–3017

    Article  MathSciNet  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the funding support of Babol Noshirvani University of Technology through grant program No. BNUT/389059/1401.

Author information

Authors and Affiliations

  1. Faculty of Electrical and Computer Engineering, Noshirvani University of Technology, Babol, Iran

    Ali Radmehr, Ali Aghagolzadeh & Seyed Mehdi Hosseini Andargoli

Authors
  1. Ali Radmehr
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ali Aghagolzadeh
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Seyed Mehdi Hosseini Andargoli
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ali Aghagolzadeh.

Ethics declarations

Funding and Conflicts of interests/Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Radmehr, A., Aghagolzadeh, A. & Hosseini Andargoli, S.M. PCA-based hierarchical clustering approach for motion vector estimation in H.265/HEVC video error concealment. Multimed Tools Appl 83, 20997–21017 (2024). https://doi.org/10.1007/s11042-023-16310-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16310-z

Keywords