skip to main content
10.1145/3611643.3616336acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
research-article

Metamong: Detecting Render-Update Bugs in Web Browsers through Fuzzing

Published: 30 November 2023 Publication History

Abstract

A render-update bug arises when a web browser produces an erroneous rendering output due to incorrect rendering updates. Such render-update bugs seriously harm the usability and reliability of web browsers. However, we find that detecting render-update bugs is challenging because the render-update bug is a semantic bug - given a rendering result, it is difficult to determine if it is correct due to the complex rendering specification of DOM and CSS. Thus, unlike memory corruption bugs, the incorrect rendering output does not raise the violation or crash. In practice, render-update bug detection relies on the time-prohibitive manual analysis of domain experts to determine the bug.
This paper proposes Metamong, an automated framework to detect render-update bugs without false positive issues via differential fuzz testing. Metamong features two key components: (i) page mutator, and (ii) render-update oracle. The page mutator generates render-update operations, which change the content of the web page, to trigger a render-update bug. The render-update oracle exploits an HTML standard rule, so-called yielding, to produce the correct rendering result of a given web page. Combining these components, Metamong creates two HTML files where each constructs the same web page, but only one of them induces the render-update. It then uses differential testing to compare their rendering outputs to determine a bug. We implemented a prototype of Metamong, which performs differential fuzz testing on popular browsers, Chrome and Firefox. By far, Metamong identified 19 new render-update bugs, 17 in Chrome and two in Firefox. All of those have been confirmed by each browser vendor and five are already fixed, demonstrating the practical effectiveness of Metamong in identifying render-update bugs.

Supplementary Material

Video (fse23main-p966-p-video.mp4)
"Resource leaks, which are caused by acquired resources not being released, often result in performance degradation and system crashes. Resource leak detection mainly relies on two essential components. First, identify the potential pairs of the Resource Acquisition API method and the corresponding Resource Release API method (textit{RAR pairs} for short); then based on the RAR pairs, analyze the code to check whether the release API is not subsequently called after the acquisition API. While the majority of existing resource leak detection techniques are concentrated on proposing more precise and more scalable code analysis, a few of them focus on building a more textit{complete} RAR pair pool. In particular, existing techniques only consider RAR pairs that are manually predefined or mined from project-specific code corpus. Such RAR pairs have limited coverage in libraries/APIs and might miss potential RAR pairs in the code, which further limits the effectiveness of the subsequent analysis. To build a more complete RAR pair pool for resource leak detection, in this work, we propose to represent resource-operation knowledge as textbf{abstract resource acquisition/release operation pairs} (textit{Abs-RAR pairs} for short), and mine such Abs-RAR pairs from a large code corpus. Based on this idea, we propose MiROK, a novel approach for textbf{Mi}ning textbf{R}esource textbf{O}peration textbf{K}nowledge, which aims at constructing a better RAR pair pool to support resource leak detection. Given a large code corpus, MiROK first mines Abs-RAR pairs with novel rule-based pair expansion and learning-based pair identification strategies, and then instantiates these Abs-RAR pairs into concrete RAR pairs. We implement MiROK and apply it to mine RAR pairs from a large code corpus of 1,454,224 Java methods and 20,000 Maven libraries. We then perform an extensive evaluation to investigate the mining effectiveness of MiROK and the practical usage of its mined RAR pairs for supporting resource leak detection. Our results show that MiROK mines 1,313 new Abs-RAR pairs and instantiates them into 10,766 RAR pairs with a high precision (i.e., 93.3%). In addition, we feed our mined RAR pairs to existing resource leak analysis approaches, and help them detect more resource leak defects in both online code examples and open-source projects. Our results indicate both the high quality and practical usage of our mined RAR pairs."

References

[1]
has() - css: Cascading style sheets - mdn web docs. https://developer.mozilla.org/en-US/docs/Web/CSS/:has.
[2]
american fuzzy lop. https://lcamtuf.coredump.cx/afl/.
[3]
Canonical randomized crawl version optimal for most browsers. https://lcamtuf.coredump.cx/cross_fuzz/.
[4]
Css standard. https://www.w3.org/TR/?tag=css.
[5]
Dharma. https://github.com/MozillaSecurity/dharma.
[6]
Dom standard, . https://dom.spec.whatwg.org/.
[7]
Domato, . https://github.com/googleprojectzero/domato.
[8]
Domfuzz, . https://github.com/MozillaSecurity/domfuzz/tree/master/dom.
[9]
Issue 1364376: Empty <span> still has height after removing border. https://bugs.chromium.org/p/chromium/issues/detail?id=1364376#c4.
[10]
Html specification. https://html.spec.whatwg.org/.
[11]
Life of a pixel. https://docs.google.com/presentation/d/1boPxbgNrTU0ddsc144rcXayGA_WF53k96imRH8Mp34Y/edit.
[12]
Browser layout testing - quickcheck. https://github.com/nathand8/layout-quickcheck.
[13]
Imagehash python libaray. https://github.com/JohannesBuchner/imagehash.
[14]
The list of render-update bugs in chrome bug tracker, . https://bugs.chromium.org/p/chromium/issues/list?q=opened
[15]
The list of render-update bugs in firefox bug tracker, . https://bugzilla.mozilla.org/buglist.cgi?short_desc_type=allwordssubstr&short_desc=
[16]
Inside look at modern web browser (part 3), . https://developer.chrome.com/blog/inside-browser-part3/.
[17]
Rendering overview, . https://firefox-source-docs.mozilla.org/gfx/RenderingOverview.html/.
[18]
Issue 1214206: Wide-cell percentage widths are distributed incorrectly. https://crbug.com/1214206.
[19]
Issue 1132218: tesla.com: Color options not rendered until window resize when compositesvg is enabled. https://bugs.chromium.org/p/chromium/issues/detail?id=1132218.
[20]
Undefinedbehaviorsanitizer. https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html.
[21]
Wadi. https://github.com/sensepost/wadi.
[22]
C. Aschermann, T. Frassetto, T. Holz, P. Jauernig, A.-R. Sadeghi, and D. Teuchert. Nautilus: Fishing for deep bugs with grammars. In Proceedings of the 2019 Annual Network and Distributed System Security Symposium (NDSS), San Diego, CA, Feb. 2019.
[23]
M. Böhme, L. Szekeres, and J. Metzman. On the reliability of coverage-based fuzzer benchmarking. In Proceedings of the 44th International Conference on Software Engineering (ICSE), Pittsburgh, PA, May–May 2022.
[24]
T. Brennan, S. Saha, and T. Bultan. Jvm fuzzing for jit-induced side-channel detection. In Proceedings of the 42nd International Conference on Software Engineering (ICSE), Seoul, South Korea, June–July 2020.
[25]
P. Chen and H. Chen. Angora: Efficient fuzzing by principled search. In Proceedings of the 39th IEEE Symposium on Security and Privacy (Oakland), San Francisco, CA, May 2018.
[26]
P. Chen, J. Liu, and H. Chen. Matryoshka: fuzzing deeply nested branches. In Proceedings of the 26th ACM Conference on Computer and Communications Security (CCS), London, UK, Nov. 2019.
[27]
Y. Chen, T. Su, C. Sun, Z. Su, and J. Zhao. Coverage-directed differential testing of jvm implementations. In Proceedings of the 2016 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Santa Barbara, CA, June 2016.
[28]
Y. Chen, T. Su, and Z. Su. Deep differential testing of jvm implementations. In Proceedings of the 41st International Conference on Software Engineering (ICSE), Montreal, QC, Canada, May 2019.
[29]
S. R. Choudhary, H. Versee, and A. Orso. Webdiff: Automated identification of cross-browser issues in web applications. In 2010 IEEE International Conference on Software Maintenance, pages 1–10. IEEE, 2010.
[30]
S. R. Choudhary, M. R. Prasad, and A. Orso. Crosscheck: Combining crawling and differencing to better detect cross-browser incompatibilities in web applications. In 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, pages 171–180. IEEE, 2012.
[31]
S. R. Choudhary, M. R. Prasad, and A. Orso. X-pert: Accurate identification of cross-browser issues in web applications. In Proceedings of the 35th International Conference on Software Engineering (ICSE), San Francisco, CA, May 2013.
[32]
Y. Deng, C. Yang, A. Wei, and L. Zhang. Fuzzing deep-learning libraries via automated relational api inference. In Proceedings of the 30th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), Singapore, Nov. 2022.
[33]
G. A. Di Luna, D. Italiano, L. Massarelli, S. Österlund, C. Giuffrida, and L. Querzoni. Who’s debugging the debuggers? exposing debug information bugs in optimized binaries. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), Virtual, Apr. 2021.
[34]
A. F. Donaldson, H. Evrard, A. Lascu, and P. Thomson. Automated testing of graphics shader compilers. Proceedings of the ACM on Programming Languages, 1 (OOPSLA): 1–29, 2017.
[35]
J. Hur, S. Song, D. Kwon, E. Baek, J. Kim, and B. Lee. Difuzzrtl: Differential fuzz testing to find cpu bugs. In Proceedings of the 42nd IEEE Symposium on Security and Privacy (Oakland), San Francisco, CA, May 2021.
[36]
D. R. Jeong, K. Kim, B. Shivakumar, B. Lee, and I. Shin. Razzer: Finding kernel race bugs through fuzzing. In Proceedings of the 40th IEEE Symposium on Security and Privacy (Oakland), San Francisco, CA, May 2019.
[37]
H. Kim, M. O. Ozmen, A. Bianchi, Z. B. Celik, and D. Xu. Pgfuzz: Policy-guided fuzzing for robotic vehicles. In Proceedings of the 2021 Annual Network and Distributed System Security Symposium (NDSS), Virtual, Feb. 2021.
[38]
S. Kim, Y. M. Kim, J. Hur, S. Song, G. Lee, and B. Lee. FuzzOrigin: Detecting UXSS vulnerabilities in browsers through origin fuzzing. In Proceedings of the 31st USENIX Security Symposium (Security), Boston, MA, Aug. 2022.
[39]
D. Lehmann and M. Pradel. Feedback-directed differential testing of interactive debuggers. In Proceedings of the 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), Lake Buena Vista, FL, Nov. 2018.
[40]
S. Mahajan and W. G. Halfond. Finding html presentation failures using image comparison techniques. In Proceedings of the 29th IEEE/ACM International Conference on Automated Software Engineering (ASE), Västerås, Sweden, Sept. 2014.
[41]
S. Mahajan and W. G. Halfond. Detection and localization of html presentation failures using computer vision-based techniques. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pages 1–10. IEEE, 2015.
[42]
R. Meng, Z. Dong, J. Li, I. Beschastnikh, and A. Roychoudhury. Linear-time temporal logic guided greybox fuzzing. In Proceedings of the 44th International Conference on Software Engineering (ICSE), Pittsburgh, PA, May–May 2022.
[43]
A. Mesbah and M. R. Prasad. Automated cross-browser compatibility testing. In Proceedings of the 33th International Conference on Software Engineering (ICSE), Honolulu, HI, May 2007.
[44]
L. A. Meyerovich and R. Bodik. Fast and parallel webpage layout. In Proceedings of the 19th International World Wide Web Conference (WWW), Raleigh, NC, Apr. 2010.
[45]
P. Panchekha and E. Torlak. Automated reasoning for web page layout. In Proceedings of the ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), Amsterdam, Netherlands, Nov. 2016.
[46]
P. Panchekha, A. T. Geller, M. D. Ernst, Z. Tatlock, and S. Kamil. Verifying that web pages have accessible layout. In Proceedings of the 2018 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Philadelphia, PA, June 2018.
[47]
T. Petsios, A. Tang, S. Stolfo, A. D. Keromytis, and S. Jana. Nezha: Efficient domain-independent differential testing. In Proceedings of the 38th IEEE Symposium on Security and Privacy (Oakland), San Jose, CA, May 2017.
[48]
K. Serebryany, D. Bruening, A. Potapenko, and D. Vyukov. Addresssanitizer: A fast address sanity checker. In Proceedings of the 2012 USENIX Annual Technical Conference (ATC), Boston, MA, June 2012.
[49]
S. Song, J. Hur, S. Kim, P. Rogers, and B. Lee. R2z2: Detecting rendering regressions in web browsers through differential fuzz testing. In Proceedings of the 44th International Conference on Software Engineering (ICSE), Pittsburgh, PA, May–May 2022.
[50]
J. Wang, B. Chen, L. Wei, and Y. Liu. Skyfire: Data-driven seed generation for fuzzing. In Proceedings of the 38th IEEE Symposium on Security and Privacy (Oakland), San Jose, CA, May 2017.
[51]
A. Wei, Y. Deng, C. Yang, and L. Zhang. Free lunch for testing: Fuzzing deep-learning libraries from open source. In Proceedings of the 44th International Conference on Software Engineering (ICSE), Pittsburgh, PA, May–May 2022.
[52]
M. Wu, L. Jiang, J. Xiang, Y. Huang, H. Cui, L. Zhang, and Y. Zhang. One fuzzing strategy to rule them all. In Proceedings of the 44th International Conference on Software Engineering (ICSE), Pittsburgh, PA, May–May 2022.
[53]
W. Xu, S. Park, and T. Kim. Freedom: Engineering a state-of-the-art dom fuzzer. In Proceedings of the 27th ACM Conference on Computer and Communications Security (CCS), Virtual, USA, Nov. 2020.
[54]
Y. Yang, Y. Zhou, H. Sun, Z. Su, Z. Zuo, L. Xu, and B. Xu. Hunting for bugs in code coverage tools via randomized differential testing. In Proceedings of the 41st International Conference on Software Engineering (ICSE), Montreal, QC, Canada, May 2019.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
November 2023
2215 pages
ISBN:9798400703270
DOI:10.1145/3611643
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 November 2023

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. fuzz testing
  2. rendering update
  3. web-browser

Qualifiers

  • Research-article

Conference

ESEC/FSE '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 208
    Total Downloads
  • Downloads (Last 12 months)119
  • Downloads (Last 6 weeks)11
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media