Skip to main content

Advertisement

Log in

Detecting salient objects in non-stationary video image sequence for analyzing user perceptions of digital video contents

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The ubiquitous utilization of video applications in recent years has made research on video quality of experience paramount. Lack of sufficient bandwidth deters the effective transmission of raw video contents to users. This bandwidth challenge has given rise to encoders for compressing digital video contents for transmission over an internet protocol infrastructure. However, transmitting compressed video color images still has an intrinsic limitation of high bandwidth consumption. Simple linear iterative clustering algorithm was applied for binary segmentation of video color images to circumvent the challenge of efficiently transmitting video contents. Compressed binary segmented images are generally fast to transmit and require lower bandwidth consumption as opposed to compressed video color images. However, since color images contain more useful information than binary image counterparts, evaluation of binary segmentation results was performed using the mean opinion score metric to determine user quality of experience of the transmitted video contents. The practical application of our method will lead to the development of a novel encoder that can deliver binary video contents faster, hence solving the bandwidth hiccup.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2010) Slic superpixels

  2. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  3. Achanta R, Süsstrunk S (2010) Saliency detection using maximum symmetric surround. In: Proceedings of Image processing (ICIP), 2010 17th IEEE international conference on. IEEE, 2653–2656

  4. Ahn E, Bi L, Jung YH, Kim J, Li C, Fulham M, Feng DD (2015) Automated saliency-based lesion segmentation in dermoscopic images. In: Proceedings of Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE. IEEE, 3009–3012

  5. Ahn E, Kim J, Bi L, Kumar A, Li C, Fulham M, Feng DD (2017) Saliency-based lesion segmentation via background detection in dermoscopic images. IEEE Journal of Biomedical and Health Informatics 21(6):1685–1693

    Article  Google Scholar 

  6. Al-azawi RJ, Abdulhameed AA, Ahmed HM (2017) A Robustness Segmentation Approach for Skin Cancer Image Detection Based on an Adaptive Automatic Thresholding Technique. American Journal of Intelligent Systems 7(4):107–112

    Google Scholar 

  7. Bhandari AK, Kumar A, Singh GK (2015) Tsallis entropy based multilevel thresholding for colored satellite image segmentation using evolutionary algorithms. Expert Syst Appl 42(22):8707–8730

    Article  Google Scholar 

  8. Borji A, Cheng M-M, Jiang H, Li J (2015) Salient object detection: A benchmark. IEEE Trans Image Process 24(12):5706–5722

    Article  MathSciNet  MATH  Google Scholar 

  9. Brooks P, Hestnes B (2010) User measures of quality of experience: why being objective and quantitative is important. IEEE Netw 24(2)

  10. Chen C, Li S, Wang Y, Qin H, Hao A (2017) Video Saliency Detection via SpatialTemporal Fusion and Low-Rank Coherency Diffusion. IEEE Trans Image Process 26(7):3156–3170

    Article  MathSciNet  MATH  Google Scholar 

  11. Correa G, Assuncao P, Agostini L, da Silva Cruz LA (2012) Performance and computational complexity assessment of high-efficiency video encoders. IEEE Transactions on Circuits and Systems for Video Technology 22(12):1899–1909

    Article  Google Scholar 

  12. Dollár P, Appel R, Belongie S, Perona P (2014) Fast feature pyramids for object detection. IEEE Trans Pattern Anal Mach Intell 36(8):15321545

    Article  Google Scholar 

  13. Duanmu Z, Zeng K, Ma K, Rehman A, Wang Z (2016) A quality-of-experience index for streaming video. IEEE Journal of Selected Topics in Signal Processing 11(1):154–166

    Article  Google Scholar 

  14. El Abbadi NK, Miry AH (2014) Automatic segmentation of skin lesions using histogram thresholding. J Comput Sci 10(4):632–639

  15. Eng ET, Kopylov M, Negro CJ, Dallaykan S, Rice WJ, Jordan KD, Kelley K, Carragher B, Potter C (2018) The impact of data reduction and lossy image formats on electron microscope images. bioRxiv, p.451427

  16. Engelke U, Barkowsky M, Le Callet P, Zepernick H-J (2010) Modelling saliency awareness for objective video quality assessment. In: Proceedings of Quality of Multimedia Experience (QoMEX), 2010 Second International Workshop on. IEEE, 212–217

  17. Fang Y, Lin W, Chen Z, Tsai C-M, Lin C-W (2014) A video saliency detection model in compressed domain. IEEE Transactions on Circuits and Systems for Video Technology 24(1):27–38

    Article  Google Scholar 

  18. Fei L, Deng Y, Mahadevan S (2015) Which is the best belief entropy. Journal of Latex Class Files 13(9):1–4

    Google Scholar 

  19. Fu H, Cao X, Tu Z (2013) Cluster-based co-saliency detection. IEEE Trans Image Process 22(10):3766–3778

    Article  MathSciNet  MATH  Google Scholar 

  20. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of Proceedings of the IEEE conference on computer vision and pattern recognition. 580–587

  21. Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926

    Article  Google Scholar 

  22. Gong B, Chao W-L, Grauman K, Sha F (2014) Diverse sequential subset selection for supervised video summarization. In: Proceedings of Advances in Neural Information Processing Systems. 2069–2077

  23. Guo L, Cheng T, Huang Y, Zhao J, Zhang R (2017) Unsupervised video object segmentation by spatiotemporal graphical model. Multimed Tools Appl 76(1):1037–1053

    Article  Google Scholar 

  24. Guo C, Zhang L (2010) A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans Image Process 19(1):185–198

    Article  MathSciNet  MATH  Google Scholar 

  25. Hadizadeh H, Bajic IV (2014) Saliency-aware video compression. IEEE Trans Image Process 23(1):19–33

    Article  MathSciNet  MATH  Google Scholar 

  26. Han J, Zhou P, Zhang D, Cheng G, Guo L, Liu Z, Bu S, Wu J (2014) Efficient, simultaneous detection of multi-class geospatial targets based on visual saliency modeling and discriminative learning of sparse coding. ISPRS J Photogramm Remote Sens 89:37–48

    Article  Google Scholar 

  27. Helin H, Tolonen T, Ylinen O, Tolonen P, Näpänkangas J, Isola J (2018) Optimized JPEG 2000 compression for efficient storage of histopathological whole-Slide images. Journal of Pathology Informatics 9

  28. Huang L-K, Wang M-JJ (1995) Image thresholding by minimizing the measures of fuzziness. Pattern Recogn 28(1):41–51

    Article  Google Scholar 

  29. Itti L (2004) Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Trans Image Process 13(10):1304–1318

    Article  Google Scholar 

  30. Jiang B, Zhang L, Lu H, Yang C, Yang M-H (2013) Saliency detection via absorbing markov chain. In: Proceedings of Proceedings of the IEEE International Conference on Computer Vision. 1665–1672

  31. Kou G, Peng Y, Wang G (2014) Evaluation of clustering algorithms for financial risk analysis using MCDM methods. Inf Sci 275:1–12

    Article  Google Scholar 

  32. Kumar V, Barthwal S, Kishore R, Saklani R, Sharma A, Sharma S (2016) Lossy Data Compression Using Logarithm. arXiv preprint arXiv:1604.02035

  33. Lee S-H, Kang J-W, Kim C-S (2016) Compressed domain video saliency detection using global and local spatiotemporal features. J Vis Commun Image Represent 35:169–183

    Article  Google Scholar 

  34. Li Y, Fu K, Liu Z, Yang J (2015) Efficient saliency-model-guided visual co-saliency detection. IEEE Signal Processing Letters 22(5):588–592

    Article  Google Scholar 

  35. Li F, Kim T, Humayun A, Tsai D, Rehg JM (2013a) Video segmentation by tracking many figure-ground segments. In: Proceedings of Proceedings of the IEEE International Conference on Computer Vision. 2192–2199

  36. Li J, Levine MD, An X, Xu X, He H (2013b) Visual saliency based on scale-space analysis in the frequency domain. IEEE Trans Pattern Anal Mach Intell 35(4):996–1010

    Article  Google Scholar 

  37. Li B, Sullivan GJ, Xu J (2012) Comparison of compression performance of HEVC working draft 5 with AVC high profile. document JCTVC-H0360

  38. Lu H, Li X, Zhang L, Ruan X, Yang M-H (2016) Dense and sparse reconstruction error based saliency descriptor. IEEE Trans Image Process 25(4):1592–1603

    Article  MathSciNet  MATH  Google Scholar 

  39. Luo Q, Geng Y, Liu J, Li W (2014) Saliency and texture information based full reference quality metrics for video QoE assessment. In: Proceedings of Network Operations and Management Symposium (NOMS), 2014 IEEE. IEEE, 1–6

  40. Margolin R, Tal A, Zelnik-Manor L (2013) What makes a patch distinct? In: Proceedings of Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1139–1146

  41. Perazzi F, Pont-Tuset J, McWilliams B, Van Gool L, Gross M, Sorkine-Hornung A (2016) A benchmark dataset and evaluation methodology for video object segmentation. In: Proceedings of Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 724–732

  42. Podder PK, Paul M, Murshed M (2016) Fast mode decision in the HEVC video coding standard by exploiting region with dominated motion and saliency features. PLoS One 11(3):e0150673

    Article  Google Scholar 

  43. Pont-Tuset J, Marques F (2016) Supervised evaluation of image segmentation and object proposal techniques. IEEE Trans Pattern Anal Mach Intell 38(7):1465–1478

    Article  Google Scholar 

  44. Popovic A, De la Fuente M, Engelhardt M, Radermacher K (2007) Statistical validation metric for accuracy assessment in medical image segmentation. Int J Comput Assist Radiol Surg 2(3):169–181

    Article  Google Scholar 

  45. Potapov D, Douze M, Harchaoui Z, Schmid C (2014) Category-specific video summarization. In: Proceedings of European conference on computer vision. Springer, 540555

  46. Ren CY, Reid I (2011) gSLIC: a real-time implementation of SLIC superpixel segmentation. University of Oxford, Department of Engineering, Technical Report

  47. Samanthula BK, Jiang W (2016) Secure multiset intersection cardinality and its application to jaccard coefficient. IEEE Transactions on Dependable and Secure Computing 13(5):591–604

    Article  Google Scholar 

  48. Scharr H, Minervini M, French AP, Klukas C, Kramer DM, Liu X, Luengo I, Pape J-M, Polder G, Vukadinovic D (2016) Leaf segmentation in plant phenotyping: a collation study. Mach Vis Appl 27(4):585–606

    Article  Google Scholar 

  49. Shattuck DW, Prasad G, Mirza M, Narr KL, Toga AW (2009) Online resource for validation of brain segmentation methods. NeuroImage 45(2):431–439

    Article  Google Scholar 

  50. Shen J, Du Y, Wang W, Li X (2014) Lazy random walks for superpixel segmentation. IEEE Trans Image Process 23(4):1451–1462

    Article  MathSciNet  MATH  Google Scholar 

  51. Souly N, Shah M (2016) Visual saliency detection using group lasso regularization in videos of natural scenes. Int J Comput Vis 117(1):93–110

    Article  MathSciNet  Google Scholar 

  52. Taha AA, Hanbury A (2015) Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging 15(1):29

    Article  Google Scholar 

  53. Tasse FP, Kosinka J, Dodgson NA (2016) Quantitative analysis of saliency models. In: Proceedings of SIGGRAPH ASIA 2016 Technical Briefs. ACM, 19

  54. Tsai, D, Flagg M, Rehg J (2010) Motion coherent tracking with multi-label mrf optimization, algorithms

  55. Udupa JK, LeBlanc VR, Zhuge Y, Imielinska C, Schmidt H, Currie LM, Hirsch BE, Woodburn J (2006) A framework for evaluating image segmentation algorithms. Comput Med Imaging Graph 30(2):75–87

    Article  Google Scholar 

  56. Wang W, Shen J, Shao L (2015) Consistent video saliency using local gradient flow optimization and global refinement. IEEE Trans Image Process 24(11):41854196

    MathSciNet  MATH  Google Scholar 

  57. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  58. Zhang D, Meng D, Han J (2017a) Co-saliency detection via a self-paced multiple-instance learning framework. IEEE Trans Pattern Anal Mach Intell 39(5):865–878

    Article  Google Scholar 

  59. Zhang J, Sclaroff S (2016) Exploiting surroundedness for saliency detection: a Boolean map approach. IEEE Trans Pattern Anal Mach Intell 38(5):889902

    Google Scholar 

  60. Zhang L, Yang C, Lu H, Ruan X, Yang M-H (2017b) Ranking saliency. IEEE Trans Pattern Anal Mach Intell 39(9):1892–1904

    Article  Google Scholar 

  61. Zheng K, Zhang X, Zheng Q, Xiang W, Hanzo L (2015) Quality-of-experience assessment and its application to video services in LTE networks. IEEE Wirel Commun 22(1):70–78

    Article  Google Scholar 

  62. Qin C, Zhang G, Zhou Y, Tao W, Cao Z (2014) Integration of the saliency-based seed extraction and random walks for image segmentation. Neurocomputing 129:378–391

  63. Bylinskii Z, Recasens A, Borji A, Oliva A, Torralba A, Durand F (2016) October. Where should saliency models look next?. In European Conference on Computer Vision (pp. 809–824). Springer, Cham

  64. Gygli M, Grabner H, Riemenschneider H, Nater F, Van Gool L (2013) The interestingness of images. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1633–1640)

  65. Wei L, Wang F, Li X, Wu F, Xiao J (2017) Graph-theoretic spatiotemporal context modeling for video saliency detection. arXiv preprint arXiv:1707.07815, 1–5

  66. El Abbadi NK, Miry AH (2014) Automatic segmentation of skin lesions using histogram thresholding. Journal of Computer Science 10(4):632–639

  67. Rabbani, T., Van Den Heuvel, F. and Vosselmann, G. 2006. Segmentation of point clouds using smoothness constraint. International archives of photogrammetry, remote sensing and spatial information sciences 36(5):248–253

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Timothy Adeliyi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Adeliyi, T., Olugbara, O. Detecting salient objects in non-stationary video image sequence for analyzing user perceptions of digital video contents. Multimed Tools Appl 78, 31807–31821 (2019). https://doi.org/10.1007/s11042-019-08008-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08008-y

Keywords

Navigation