Qingzhu
Abstract
Clearly, no one likes webpages with poor quality of experience (QoE). Being perceived as slow or fast is a key element in the overall perceived QoE of web applications. While extensive effort has been put into optimizing web applications (both in industry and academia), not a lot of work exists in characterizing what aspects of webpage loading process truly influence human end-user's perception of the \emph{Speed} of a page. In this paper we present \emph{SpeedPerception}, a large-scale web performance crowdsourcing framework focused on understanding the perceived loading performance of above-the-fold (ATF) webpage content. Our end goal is to create free open-source benchmarking datasets to advance the systematic analysis of how humans perceive webpage loading process.
In Phase-1 of our \emph{SpeedPerception} study using Internet Retailer Top 500 (IR 500) websites, we found that commonly used navigation metrics such as \emph{onLoad} and \emph{Time To First Byte (TTFB)} fail (less than 60\% match) to represent majority human perception when comparing the speed of two webpages. We present a simple 3-variable-based machine learning model that explains the majority end-user choices better (with $87 \pm 2\%$ accuracy). In addition, our results suggest that the time needed by end-users to evaluate relative perceived speed of webpage is far less than the time of its \emph{visualComplete} event.
- 1 Google Chrome Lighthouse Project https://github.com/googlechrome/lighthouse.Google Scholar
- 2 SpeedIndex https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index.Google Scholar
- 3 SpeedPerception Benchmark and Results https://github.com/pahammad/speedperception.Google Scholar
- 4 SpeedPerception Experimental UI http://speedperception.com.Google Scholar
- 5 The Very Real Performance Impact on Revenue http://blog.catchpoint.com/2017/01/06/performance-impact-revenue-real/.Google Scholar
- 6 P. Ahammad, R. Gaunker, B. Kennedy, M. Reshadi, K. Kumar, A. Pathan, and H. Kolam. A flexible platform for QoE-driven delivery of image-rich web applications. In IEEE ICME 2015, pages 1–6, 2015. Google ScholarCross Ref
- 7 A. Balachandran, V. Aggarwal, E. Halepovic, J. Pang, S. Seshan, S. Venkataraman, and H. Yan. Modeling web quality-of-experience on cellular networks. In ACM MobiCom 2014, pages 213–224, 2014. Google ScholarDigital Library
- 8 E. Bocchi, L. De Cicco, and D. Rossi. Measuring the quality of experience of web users. ACM SIGCOMM Computer Communication Review, 46(4):8–13, 2016. Google ScholarDigital Library
- 9 L. Breiman. Random forests. Machine learning, 45(1):5–32, 2001. Google ScholarDigital Library
- 10 M. Butkiewicz, H. V. Madhyastha, and V. Sekar. Understanding website complexity: measurements, metrics, and implications. In ACM SIGCOMM IMC, pages 313–328, 2011. Google ScholarDigital Library
- 11 E. Carbery. Website Performance: The Need for Speed http://www.6smarketing.com/blog/website-performance-the-need-for-speed/.Google Scholar
- 12 E. Cohen and H. Kaplan. Prefetching the means for document transfer: A new approach for reducing web latency. Computer Networks, 39(4):437–455, 2002. Google ScholarCross Ref
- 13 H. Cramér. Mathematical Methods of Statistics, volume 9. Princeton university press, 2016.Google Scholar
- 14 S. Egger, P. Reichl, T. Hoßfeld, and R. Schatz. “Time is bandwidth”? Narrowing the gap between subjective time perception and quality of experience. In IEEE ICC 2012, pages 1325–1330, 2012.Google ScholarCross Ref
- 15 J. H. Friedman. Greedy function approximation: a gradient boosting machine. Annals of statistics, pages 1189–1232, 2001. Google ScholarCross Ref
- 16 T. Hossfeld, C. Keimel, M. Hirth, B. Gardlo, J. Habigt, K. Diepold, and P. Tran-Gia. Best practices for QoE crowdtesting: QoE assessment with crowdsourcing. IEEE Transactions on Multimedia, 16(2):541–558, 2014. Google ScholarDigital Library
- 17 A. Iyengar, E. Nahum, A. Shaikh, and R. Tewari. Web caching, consistency, and content distribution. The Practical Handbook of Internet Computing, 2005.Google Scholar
- 18 C. Kelton, J. Ryoo, A. Balasubramanian, and S. R. Das. Improving user perceived page load times using Gaze. In USENIX NSDI 17, pages 545–559.Google Scholar
- 19 R. Netravali, A. Goyal, J. Mickens, and H. Balakrishnan. Polaris: Faster page loads using fine-grained dependency tracking. In USENIX NSDI 2016, 2016.Google Scholar
- 20 M. Varvello, J. Blackburn, D. Naylor, and K. Papagiannaki. EYEORG: A platform for crowdsourcing web quality of experience measurements. In ACM CoNEXT, pages 399–412, 2016. Google ScholarDigital Library
- 21 X. S. Wang, A. Krishnamurthy, and D. Wetherall. Speeding up web page loads with Shandian. In USENIX NSDI, 2016.Google ScholarDigital Library
- 22 Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. Google ScholarDigital Library
Index Terms
- Perceived Performance of Top Retail Webpages In the Wild
Recommendations
Perceived Performance of Top Retail Webpages In the Wild: Insights from Large-scale Crowdsourcing of Above-the-Fold QoE
Internet QoE '17: Proceedings of the Workshop on QoE-based Analysis and Management of Data Communication NetworksClearly, no one likes webpages with poor quality of experience (QoE). Being perceived as slow or fast is a key element in the overall perceived QoE of web applications. While extensive effort has been put into optimizing web applications (both in ...
Measuring the Quality of Experience of Web users
Measuring quality of Web users experience (WebQoE) faces the following trade-off. On the one hand, current practice is to resort to metrics, such as the document completion time (onLoad), that are simple to measure though knowingly inaccurate. On the ...
Measuring the Quality of Experience of Web users
Internet-QoE '16: Proceedings of the 2016 workshop on QoE-based Analysis and Management of Data Communication NetworksMeasuring quality of Web users experience (WebQoE) faces the following trade-off. On the one hand, current practice is to resort to metrics, such as the document completion time (onLoad), that are simple to measure though knowingly inaccurate. On the ...
Comments