SkipStreaming: Pinpointing User-Perceived Redundancy in Correlated Web Video Streaming through the Lens of Scenes
Authors: 
Wei Liu, Xinlei Yang, 
Zhenhua Li, Feng QianAuthors Info & Claims
Wei Liu, Xinlei Yang, Zhenhua Li, Feng QianAuthors Info & ClaimsPages 3944 - 3953
Abstract
When streaming over the web, correlated videos (e.g., a series of TV episodes) appear to bear considerable redundant clips, mostly included in the intros, outros, recaps, and commercial breaks, leading to a waste of network traffic and playback time. Mainstream video content providers have taken various measures to identify these clips, but often result in unexpected and undesirable user experiences. In this paper, we conduct a large-scale, crowdsourced study to demystify the root causes of poor experiences. Driven by the findings, we propose to reconsider the problem from a novel perspective of scenes without going through the excessive video frames, which pays special attention to how the contents of correlated videos are organized during video production. To enable this idea, we design efficient approaches to the separation of video scenes and the identification of visual redundancy. We build an open-source system to embody our design, which achieves fast (e.g., taking ~38 seconds to process a 45-minute video using a common commodity server) and accurate (incurring only 770-ms deviation on average) redundancy recognition on representative workloads.
References
[1]
2018. UX in Daily Life - Netflix's "Skip Intro". https://www.linkedin.com/pulse/ux-daily-life-netflixs-skip-intro-imran-razaq/.
[2]
2019. The Intros of Some Episodes in Disney Cannot Be Skipped. https://www.reddit.com/r/DisneyPlus/comments/dv6369/disney_has_a_skip_intro_function/f7au5dh/.
[3]
2020. Cisco Annual Internet Report - Cisco Annual Internet Report (2018-2023) White Paper. https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11--741490.html.
[4]
2020. Go Ahead and Skip That Intro. https://www.plex.tv/blog/go-ahead-and-skip-that-intro/.
[5]
2021. Accessibility Features on Disney. https://help.disneyplus.com/csp?id=csp _article_content&sys_kb_id=bc24b6abdb054090fbf26ac2ca961950.
[6]
2021. Disney. https://www.disneyplus.com.
[7]
2021. Google Video AI. https://cloud.google.com/video-intelligence.
[8]
2021. Here's How Much Time You Can Save by Skipping TV Intros. https: //www.lifehacker.com.au/2021/06/save-time-skipping-tv-intros/.
[9]
2021. iQIYI and Tencent Video Responded to the Problem That Subscribers Still Have to Watch Ads: You Can Drag the Progress Bar to Skip. https://lujuba.cc/en /614223.html.
[10]
2021. Netflix. https://www.netflix.com/.
[11]
2021. Puppeteer: Headless Chrome Node.js API. https://github.com/puppeteer/p uppeteer.
[12]
2021. T.O.T.S. Season 2 Is Steaming Incorrectly. https://www.reddit.com/r/Disne yPlus/comments/rkkzqw/tots_season_2_is_steaming_incorrectly_we_are/.
[13]
2022. Auto "Skip Intro" Option? https://www.reddit.com/r/PleX/comments/rvu nb5/auto_skip_intro_option/.
[14]
2022. China 2022 Survey Report: iQIYI, Tencent Video Tops in Online Video. https://www.spglobal.com/marketintelligence/en/news-insights/research/china-2022-survey-report-iqiyi-tencent-video-tops-in-online-video.
[15]
2023. Bilibili. https://www.bilibili.com/.
[16]
2023. The Free Software Media System | Jellyfin. https://jellyfin.org/.
[17]
2023. HTMLMediaElement: currentTime Property - Web APIs | MDN. https://de veloper.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/currentTime.
[18]
2023. Kodi Open Source Home Theater Software. https://kodi.tv/.
[19]
2023. Watch Naruto | Netflix. https://www.netflix.com/title/70205012.
[20]
2023. Watch The Coyotes | Netflix. https://www.netflix.com/title/81493687.
[21]
2023. YouTube. https://www.youtube.com/.
[22]
Margareta Ackerman and Shai Ben-David. 2016. A Characterization of Linkage-Based Hierarchical Clustering. The Journal of Machine Learning Research 17, 1 (2016), 8182--8198.
[23]
Adam Hayes. 2023. YouTube Stats: Everything You Need to Know in 2023. https://www.wyzowl.com/youtube-stats/.
[24]
Lorenzo Baraldi, Costantino Grana, and Rita Cucchiara. 2015. A Deep Siamese Network for Scene Detection in Broadcast Videos. In Proc. of ACM MM. 1199--1202.
[25]
Karl Bringmann, Paweŀ Gawrychowski, Shay Mozes, and Oren Weimann. 2020. Tree Edit Distance Cannot Be Computed in Strongly Subcubic Time (Unless APSP Can). ACM Transactions on Algorithms 16, 4 (2020), 1--22.
[26]
Cameron Johnson. 2022. Looking Back on the Origin of Skip Intro Five Years Later. https://about.netflix.com/en/news/looking-back-on-the-origin-of-skip-intro-five-years-later.
[27]
Chao Chen, Yao-Chung Lin, Steve Benting, and Anil Kokaram. 2018. Optimized Transcoding for Large Scale Adaptive Streaming Using Playback Statistics. In Proc. of IEEE ICIP. 3269--3273.
[28]
Liang-Hua Chen, Yu-Chun Lai, and Hong-Yuan Mark Liao. 2008. Movie Scene Segmentation Using Background Information. Pattern Recognition 41, 3 (2008), 1056--1065.
[29]
Shixing Chen, Xiaohan Nie, David Fan, Dongqing Zhang, Vimal Bhat, and Raffay Hamid. 2021. Shot Contrastive Self-Supervised Learning for Scene Boundary Detection. In Proc. of IEEE/CVF CVPR. 9796--9805.
[30]
Zhi-Qi Cheng, Yang Liu, Xiao Wu, and Xian-Sheng Hua. 2016. Video Ecommerce: Towards Online Video Advertising. In Proc. of ACM MM. 1365--1374.
[31]
Sam Cook. 2022. 50 Netflix Statistics, Facts and Figures in 2023. https://www. comparitech.com/blog/vpn-privacy/netflix-statistics-facts-figures/.
[32]
William H. E. Day. 1985. Optimal Algorithms for Comparing Trees with Labeled Leaves. Journal of Classification 2, 1 (1985), 7--28.
[33]
Jan De Cock, Zhi Li, Megha Manohara, and Anne Aaron. 2016. Complexity-Based Consistent-Quality Encoding in the Cloud. In Proc. of IEEE ICIP. 1484--1488.
[34]
Shaojin Ding, Tianlong Chen, and Zhangyang Wang. 2022. Audio Lottery: Speech Recognition Made Ultra-Lightweight, Noise-Robust, and Transferable. In Proc. of ICLR.
[35]
Boyuan Feng, Yuke Wang, Gushu Li, Yuan Xie, and Yufei Ding. 2021. Palleon: A Runtime System for Efficient Video Processing toward Dynamic Class Skew. In Proc. of USENIX ATC. 427--441.
[36]
A. Hampapur, T. Weymouth, and R. Jain. 1994. Digital Video Segmentation. In Proc. of ACM MM. 357--364.
[37]
Xiang Hao, Kripa Chettiar, Ben Cheung, Vernon Germano, and Raffay Hamid. 2021. Intro and Recap Detection for Movies and TV Series. In Proc. of IEEE WACV. 167--176.
[38]
Zi Huang, Heng Tao Shen, Jie Shao, Bin Cui, and Xiaofang Zhou. 2010. Practical Online Near-Duplicate Subsequence Detection for Continuous Video Streams. IEEE Transactions on Multimedia 12, 5 (2010), 386--398.
[39]
Atul Katiyar and Jon Weissman. 2011. ViDeDup: An Application-Aware Framework for Video De-Duplication. In Proc. of USENIX HotStorage.
[40]
Ephraim Katz and Ronald Dean Nolen. 2012. The Film Encyclopedia 7th Edition: The Complete Guide to Film and the Film Industry. Collins Reference.
[41]
J.R. Kender and Boon-Lock Yeo. 1998. Video Scene Segmentation via Continuous Video Coherence. In Proc. of IEEE CVPR. 367--373.
[42]
Giorgos Kordopatis-Zilos, Symeon Papadopoulos, Ioannis Patras, and Ioannis Kompatsiaris. 2019. ViSiL: Fine-Grained Spatio-Temporal Video Similarity Learning. In Proc. of IEEE/CVF ICCV. 6351--6360.
[43]
Zhenhua Li, Yafei Dai, Guihai Chen, and Yunhao Liu. 2023. Content Distribution for Mobile Internet: A Cloud-Based Approach, Second Edition. Springer Nature Press.
[44]
Jiajun Liu, Zi Huang, Hongyun Cai, Heng Tao Shen, Chong Wah Ngo, and Wei Wang. 2013. Near-Duplicate Video Retrieval: Current Research and Future Trends. Comput. Surveys 45, 4 (2013), 44:1--44:23.
[45]
Wei Liu, Xinlei Yang, Hao Lin, Zhenhua Li, and Feng Qian. 2022. Fusing Speed Index during Web Page Loading. In Proc. of ACM SIGMETRICS. 1--23.
[46]
Yao Liu, Sam Blasiak, Weijun Xiao, Zhenhua Li, and Songqing Chen. 2015. A Quantitative Study of Video Duplicate Levels in YouTube. In Proc. of Springer PAM. 235--248.
[47]
Amanda D. Lotz. 2022. Netflix and Streaming Video: The Business of Subscriber-Funded Video on Demand. John Wiley & Sons.
[48]
Amanda D Lotz, Oliver Eklund, and Stuart Soroka. 2022. Netflix, Library Analysis, and Globalization: Rethinking Mass Media Flows. Journal of Communication 72, 4 (2022), 511--521.
[49]
Ben Munson. 2019. Nearly 75% of U.S. Households Have Either Netflix, Hulu or Amazon Prime Video. https://www.fiercevideo.com/video/nearly-75-u-s-households-have-either-netflix-hulu-or-amazon-prime-video.
[50]
Fionn Murtagh and Pedro Contreras. 2012. Algorithms for Hierarchical Clustering: An Overview. WIREs Data Mining and Knowledge Discovery 2, 1 (2012), 86--97.
[51]
Maryam Nematollahi and Xiao-Ping Zhang. 2014. Automatic Video Intro and Outro Detection on Internet Television. In Proc. of ACM WISMM. 43--46.
[52]
Netflix. 2021. How to Skip Intros on TV Shows. https://help.netflix.com/en/nod e/63402.
[53]
Mohammad Norouzi, Ali Punjani, and David J. Fleet. 2014. Fast Exact Search in Hamming Space with Multi-Index Hashing. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 6 (2014), 1107--1119.
[54]
Mateusz Pawlik and Nikolaus Augsten. 2011. RTED: A Robust Algorithm for the Tree Edit Distance. Proceedings of the VLDB Endowment 5, 4 (2011), 334--345.
[55]
Mateusz Pawlik and Nikolaus Augsten. 2015. Efficient Computation of the Tree Edit Distance. ACM Transactions on Database Systems 40, 1 (2015), 1--40.
[56]
Anyi Rao, Linning Xu, Yu Xiong, Guodong Xu, Qingqiu Huang, Bolei Zhou, and Dahua Lin. 2020. A Local-to-Global Approach to Multi-Modal Movie Scene Segmentation. In Proc. of IEEE/CVF CVPR. 10143--10152.
[57]
Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional Recurrent Neural Networks. IEEE Transactions on Signal Processing 45, 11 (1997), 2673--2681.
[58]
Susanna Schwarzmann, Nick Hainke, Thomas Zinner, Christian Sieber, Werner Robitza, and Alexander Raake. 2020. Comparing Fixed and Variable Segment Durations for Adaptive Video Streaming: A Holistic Analysis. In Proc. of ACM MMSys. 38--53.
[59]
Roger Silverstone. 1984. Narrative Strategies in Television Science-A Case Study. Media, Culture & Society 6, 4 (1984), 377--410.
[60]
Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proc. of ICLR.
[61]
Kevin Spiteri, Rahul Urgaonkar, and Ramesh K. Sitaraman. 2020. BOLA: Near-Optimal Bitrate Adaptation for Online Videos. IEEE/ACM Transactions on Networking 28, 4 (2020), 1698--1711.
[62]
Julia Stoll. 2019. Number of Paid SVoD Services Subscriptions Among Users in the U.S. https://www.statista.com/statistics/786665/number-paid-svod-service-subscriptions-us/.
[63]
Julia Stoll. 2019. TV Binge-Watching Preferences in the U.S. by Age 2019. https: //www.statista.com/statistics/687388/binge-watching-preference-usa/.
[64]
Makarand Tapaswi, Martin Bauml, and Rainer Stiefelhagen. 2014. StoryGraphs: Visualizing Character Interactions as a Timeline. In Proc. of IEEE/CVF CVPR. 827--834.
[65]
trillmercy. 2020. Skip Intro Button Has Been Added. https://www.reddit.com/r /Hulu/comments/iqqhi4/skip_intro_button_has_been_added/.
[66]
Haoqian Wu, Keyu Chen, Yanan Luo, Ruizhi Qiao, Bo Ren, Haozhe Liu, Weicheng Xie, and Linlin Shen. 2022. Scene Consistency Representation Learning for Video Scene Segmentation. In Proc. of IEEE/CVF CVPR. 14001--14010.
[67]
Xiao Wu, Alexander G. Hauptmann, and Chong-Wah Ngo. 2007. Practical Elimination of Near-Duplicates from Web Video Search. In Proc. of ACM MM. 218--227.
[68]
Wen Xia, Hong Jiang, Dan Feng, Fred Douglis, Philip Shilane, Yu Hua, Min Fu, Yucheng Zhang, and Yukun Zhou. 2016. A Comprehensive Study of the Past, Present, and Future of Data Deduplication. In Proc. of IEEE, Vol. 104. 1681--1710.
[69]
Francis Y Yan, James Hong, Hudson Ayers, Keyi Zhang, Chenzhi Zhu, Philip Levis, Sadjad Fouladi, and Keith Winstein. 2020. Learning in Situ: A Randomized Experiment in Video Streaming. In Proc. of USENIX NSDI. 495--511.
[70]
Xinlei Yang, Wei Liu, Hao Lin, Zhenhua Li, Feng Qian, Xianlong Wang, Yunhao Liu, and Tianyin Xu. 2023. Visual-Aware Testing and Debugging for Web Performance Optimization. In Proc. of ACM WWW. 2948--2959.
[71]
Christoph Zauner. 2010. Implementation and Benchmarking of Perceptual Image Hash Functions. (2010).
[72]
Haoyu Zhang, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, and Michael J Freedman. 2017. Live Video Analytics at Scale with Approximation and Delay-Tolerance. In Proc. of USENIX NSDI. 377--392.
[73]
Bilei Zhu, Wei Li, Zhurong Wang, and Xiangyang Xue. 2010. A Novel Audio Fingerprinting Method Robust to Time Scale Modification and Pitch Shifting. In Proc. of ACM MM. 987--990.
Index Terms
- SkipStreaming: Pinpointing User-Perceived Redundancy in Correlated Web Video Streaming through the Lens of Scenes
Recommendations
Spatial-Channel Specific Snake-YOLOv8 for Video Logo Detection in Live Streaming Scenes
ICIGP '24: Proceedings of the 2024 7th International Conference on Image and Graphics ProcessingLive video platforms have attracted many active streamers and daily users, and quickly understanding live video streaming scenes is crucial for ensuring the clean and healthy cyberspace. Video logo often appear in live video and can serve as key clues ...
Link-Time Path-Sensitive Memory Redundancy Elimination
HPCA '04: Proceedings of the 10th International Symposium on High Performance Computer ArchitectureOptimizations performed at link-time or directly applied to final program executables have received increased attention in recent years. This paper discusses the discovery and elimination of redundant memory operations in the context of a link-time ...
Single Lens Stereo with a Plenoptic Camera
Special issue on interpretation of 3-D scenes—part IIOrdinary cameras gather light across the area of their lens aperture, and the light striking a given subregion of the aperture is structured somewhat differently than the light striking an adjacent subregion. By analyzing this optical structure, one can ...
Comments
Information & Contributors
Information
Published In
October 2023
9913 pages
ISBN:9798400701085
DOI:10.1145/3581783
- General Chairs:
- Abdulmotaleb El Saddik,
- Tao Mei,
- Rita Cucchiara,
- Program Chairs:
- Marco Bertini,
- Diana Patricia Tobon Vallejo,
- Pradeep K. Atrey,
- M. Shamim Hossain
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
MM '23
Sponsor:
MM '23: The 31st ACM International Conference on Multimedia
October 29 - November 3, 2023
Ottawa ON, Canada
Acceptance Rates
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 211Total Downloads
- Downloads (Last 12 months)141
- Downloads (Last 6 weeks)21
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in