Abstract
Web application scanners are popular tools to perform black box testing and are widely used to discover bugs in websites. For them to work effectively, they either rely on a set of URLs that they can test, or use their own implementation of a crawler that discovers new parts of a web application. Traditional crawlers would extract new URLs by parsing HTML documents and applying static regular expressions. While this approach can extract URLs in classic web applications, it fails to explore large parts of modern JavaScript-based applications.
In this paper, we present a novel technique to explore web applications based on the dynamic analysis of the client-side JavaScript program. We use dynamic analysis to hook JavaScript APIs, which enables us to detect the registration of events, the use of network communication APIs, and dynamically-generated URLs or user forms. We then propose to use a navigation graph to perform further crawling. Based on this new crawling technique, we present jÄk, a web application scanner. We compare jÄk against four existing web-application scanners on 13 web applications. The experiments show that our approach can explore a surface of the web applications that is 86 % larger than with existing approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A self-submitting form is an HTML form that is submitted by firing submit or mouse click events within the JavaScript program.
- 2.
W3af implements a mechanism to terminate which is based on the following two conditions. First, W3af does not crawl twice the same URL and then it does not crawl “similar” URLs more than five times. Two URLs are similar if they differ only from the content of URL parameters.
- 3.
This model is the event bubbling and is the default model. Another model is the event capturing in which the event are propagated from the outermost to the innermost.
References
Zhou, J., Ding, Y.: An analysis of URLs generated from javascript code. In: 2012 IEEE/ACIS 11th International Conference on Computer and Information Science (ICIS), vol. 5, pp. 688–693 (2012)
Mesbah, A., van Deursen, A., Lenselink, S.: Crawling ajax-based web applications through dynamic analysis of user interface state changes. ACM Trans. Web 6(1), 3:1–3:30 (2012)
Urgun, B.: Web Input Vector Extractor Teaser (2015). https://github.com/bedirhan/wivet
Hickson, I.: A vocabulary and associated APIs for HTML and XHTML (2014). http://dev.w3.org/html5/workers/
van Kesteren, A., Gregor, A., Ms2ger, Russell, A., Berjon, R.: W3C DOM4 (2015). http://www.w3.org/TR/dom/
The Python Software Foundation: Python (2015). https://www.python.org/
Apple Inc.: The WebKit Open Source Project (2015). https://www.webkit.org/
Riverbank Computing Limited: PyQt - The GPL Licensed Python Bindings for the Qt Application Framework (2015). http://pyqt.sourceforge.net/
Google Inc.: V8 JavaScript Engine (2015). https://code.google.com/p/v8/
Mozilla Foundation: SpiderMonkey (2015). https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey
Zalewski, M.: Skipfish (2015). https://code.google.com/p/skipfish/
Riancho, A.: w3af: Web Application Attack and Audit Framework (2015). http://w3af.org/
Nikšić, H., Scrivano, G.: GNU Wget (2015). http://www.gnu.org/software/wget/
Doupé, A., Cavedon, L., Kruegel, C., Vigna, G.: Enemy of the state: a state-aware black-box vulnerability scanner. In: Proceedings of the 2012 USENIX Security Symposium (USENIX 2012), Bellevue, WA (2012)
Bau, J., Bursztein, E., Gupta, D., Mitchell, J.: State of the art: automated black-box web application vulnerability testing. In: 2010 IEEE Symposium on Security and Privacy (SP) (2010)
Doupé, A., Cova, M., Vigna, G.: Why Johnny can’t pentest: an analysis of black-box web vulnerability scanners. In: Kreibich, C., Jahnke, M. (eds.) DIMVA 2010. LNCS, vol. 6201, pp. 111–131. Springer, Heidelberg (2010)
Guangdong, B., Guozhu, M., Jike, L., Sai, S.V., Prateek, S., Jun, S., Yang, L., Jinsong, D.: Authscan: Automatic extraction of web authentication protocols from implementations. In: 2013 Annual Network and Distributed System Security Symposium (NDSS). The Internet Society (2013)
Zhou, Y., Evans, D.: Ssoscan: automated testing of web applications for single sign-on vulnerabilities. In: 23rd USENIX Security Symposium (USENIX Security 2014), pp. 495–510. USENIX Association, San Diego, CA (2014)
Artzi, S., Dolby, J., Jensen, S.H., Møller, A., Tip, F.: A framework for automated testing of javascript web applications. In: Proceedings of the 33rd International Conference on Software Engineering, ICSE 2011, pp. 571–580. ACM, New York, NY, USA (2011). http://doi.acm.org/10.1145/1985793.1985871
Acknowledgements
This work was supported by the German Ministry for Education and Research (BMBF) through funding for the project 13N13250, EC SPRIDE and ZertApps, by the Hessian LOEWE excellence initiative within CASED, and by the DFG within the projects RUNSECURE, TESTIFY and INTERFLOW, a project within the DFG Priority Programme 1496 Reliably Secure Software Systems \(-{ RS}^3\).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Pellegrino, G., Tschürtz, C., Bodden, E., Rossow, C. (2015). jÄk: Using Dynamic Analysis to Crawl and Test Modern Web Applications. In: Bos, H., Monrose, F., Blanc, G. (eds) Research in Attacks, Intrusions, and Defenses. RAID 2015. Lecture Notes in Computer Science(), vol 9404. Springer, Cham. https://doi.org/10.1007/978-3-319-26362-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-26362-5_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26361-8
Online ISBN: 978-3-319-26362-5
eBook Packages: Computer ScienceComputer Science (R0)