Semantic web Racer: Dynamic security testing of the web application against race condition in the business layer

https://doi.org/10.1016/j.eswa.2022.116569Get rights and content

Highlights

  • Classifying various race conditions in the web applications.

  • Defining various race conditions existing in e-commerce web applications.

  • Defining race window for defined race conditions in the web applications.

  • Presenting the black box approach for detecting race-prone processes.

  • Presenting the black box approach for detecting race conditions.

Abstract

The parallel execution of multiple threads of a web application will cause races if the web application is not synchronized correctly. Detecting the race condition in web applications depends on the application’s business logic. No logic-aware approach has been presented so far for detecting various race conditions in web applications. The existing approaches only detect part of server-side races or a few client-side race conditions. Most existing approaches result in DoS or they have a high vulnerability detection time.

In this study, various race conditions existing in a web application, both on the server-side and on the client-side, are classified and described. In addition, we present Semantic Web Racer, a black-box approach for dynamic application security testing, to detect the business-layer vulnerability of the web application against race conditions both on the server-side and on the client-side. Semantic Web Racer detects race conditions by identifying the business logic of the web application. First, it identifies the business processes in the web application and, by defining a trace pattern for each type of race condition, identifies critical business processes. The detected critical processes are performed in the defined race window in both normal mode and race-prone mode, and the results are checked to identify vulnerabilities.

The evaluation of well-known and widely used web applications and web pages shows that Semantic Web Racer can detect the business layer vulnerabilities of these applications against race conditions. Experiments showed that out of 38 detected race conditions by Semantic Web Racer in selected applications, 24 are new vulnerabilities that were not identified by related works. The amount of traffic generated to identify vulnerabilities has been improved by about 98.29% by identifying the business layer of the application. Thus, Semantic Web Racer does not result in DoS. Semantic Web Racer has improved race detection time by about 96.78%.

Introduction

Applications provide the possibility of simultaneous and parallel access of users to the existing content. Concurrency attacks on the Flexcoin Bitcoin exchange in 2014 led to the theft of Flexcoin users’ account balances. By sending thousands of simultaneous requests and abusing the error in the code, an attacker can transfer a balance from one person’s account to another until the account runs out. The attacker repeated the attack to other users’ accounts, leading to the theft of the entire Flexcoin exchange balance.

Applications should be aware of the execution of parallel threads and if several threads use a common object in parallel, the web application should schedule the threads. Time-dependent bugs are known as race conditions.

Detecting race conditions in an application depends on detecting the logic of the application (Emous, 2019). No logic-awareness approach has been yet proposed to detect concurrency problems in web applications. Detecting logic vulnerabilities is difficult because they are specific to the application. Logic vulnerabilities are strong vulnerabilities that affect the security of applications (Alidoosti et al., 2020, Braun et al., 2013, Liang et al., 2020, Liu et al., 2018, Meng et al., 2019, Mesli-Kesraoui et al., 2020, Mitra ALidoosti and Alireza Nowroozi, 2021, Zhao et al., 2018). The damage caused by race conditions includes bypassing constraints of the application like reusing vouchers, purchasing out-of-inventory products, privilege escalation, and DoS attacks. The race occurs when the operations are not atomic.

Detecting race condition in a web application is very difficult (Lee, 2020) because a race condition is a probabilistic event that is state and time-related and depends on the logic of the web application (Adamsen, 2018, Emous, 2019, Endo and Moller, 2020). For the race condition to occur, the application should be in a specific state and even the operations should be done in a minimal timeframe. Thus, detecting the race condition is necessary.

In general, two types of race conditions might occur in the application, server-side race and client-side race. The server-side race is one of the most common race conditions in web applications occurring because of flaws in the interaction of the web application with the database (Braun et al., 2013, Liang et al., 2020, Liu et al., 2018, Meng et al., 2019, Mesli-Kesraoui et al., 2020, Zhao et al., 2018). The client-side race occurs because of flaws in the server-client communication (Endo, 2020).

Existing solutions for identifying server-side race conditions (Braun et al., 2013, Halfond et al., 2006, Liang et al., 2020, Liu et al., 2018, Meng et al., 2019, Mesli-Kesraoui et al., 2020, Zhao et al., 2018) and existing solutions for identifying client-side race conditions (Adamsen, 2018, Adamsen et al., 2017, Adamsen et al., 2017, Adamsen et al., 2018, Endo and Moller, 2020, Hong et al., 2014, Mutlu and Livshits, 2016, Petrov et al., 2012, Raychev et al., 2013, Wang et al., 2016, Lee, 2020, Zhang and Wang, 2017) do not understand the logic of the web applications. These solutions identify the vulnerability without identifying the business logic of the web application, thus reducing the accuracy of the vulnerability detection. These solutions can only identify a few of race conditions. INITRACER (Adamsen et al., 2017) and AjaxRacer (Adamsen et al., 2018)are not able to navigate the application and are only able to detect the race conditions on the home page of the application and do not detect race conditions deep inside the web pages.

Paleari et al. (Paleari, Marrone, Bruschi, & Monga, 2008), CompuRacer (Emous, 2019), Zheng et al. (Zheng, 2012), ACIDRain (Warszawski & Bailis, 2017) and Raccoon (Koch, Sauer, Johns, & Pellegrino, 2020) identify the server-side race conditions by analyzing SQL logs. A regular user cannot access SQL logs, so these solutions are not black-box and need to have higher permission, even admin-level for vulnerability assessment. Paleari et al. (Paleari et al., 2008), ACIDRain (Warszawski & Bailis, 2017), Raccoon (Koch et al., 2020) to identify race conditions, run a critical transaction in parallel with many threads, which increases the likelihood of a denial-of-service attack in the web application and the application may become out of reach.

Existing solutions apply for applications written in a particular language and cannot to examine advanced cases in the language. For example, Paleari et al. (Paleari et al., 2008), Zheng et al. (Zheng, 2012), and ACIDRain (Warszawski & Bailis, 2017), the only supported SQL, and Raccon (Koch et al., 2020) only can detect vulnerabilities in PHP-based applications.

The average time to identify a vulnerability is very high in some existing solutions. Raccoon (Koch et al., 2020) takes an average of about 7 h to detect race conditions, which is very high and makes it impractical. AjaxRacer (Adamsen et al., 2018) takes an average of 35 min to detect vulnerabilities.

Research Gap: to identify the race conditions in web applications, it is necessary to analyze the different races and their creation conditions. So far, no research examines both server-side race conditions and client-side race conditions and no one provides a comprehensive list of various types of race conditions. Related articles identified only one or a few limited cases of server-side race conditions or only one or a few limited cases of client-side race conditions without analyzing the type and reason for its occurrence, and sometimes a very limited solution to prevent it. Race conditions highly depend on the application’s business logic, so understanding the logic of the application improves the recognition of race conditions, but so far, no business logic-aware solution for identifying race conditions in the application has been provided. Existing solutions have a high detection time, even several hours, and sending multiple parallel requests to the application to create race conditions increases the likelihood of a denial of service attack in the web application. In addition, the proposed models, because of the lack of knowledge of the business logic, can expand to infinity in large web applications and are only for programs written in a specific language. Related works to identify race conditions are not black box and require a high permission access level and even the administrator access level.

Research questions:

  • What are the comprehensive categories of race conditions in the web application?

  • What are the conditions for the occurrence of various race conditions in the web application?

  • How does identifying the business logic of the web application helps detect race conditions?

  • How to identify different race conditions in the web application without applying a denial of service attack?

In this paper, for the first time, a comprehensive list of types of race conditions in the web application, both on the server-side and on the client-side, is presented. Existing race conditions are categorized and named and their creation conditions are analyzed. In addition, a black-box approach called Semantic Web Racer is presented for dynamic application security testing in the business layer for detecting race conditions in web applications. The proposed solution is aware of application’s business logic and identifies the business processes. Semantic Web Racer defines processes that have a specific trace pattern as a critical business process, and for critical processes, it defines a specific period called the race window. Only in this time window can race conditions occur. Semantic Web Racer executes critical processes and identifies vulnerabilities in the web application by considering a clever chronology based on the race window. It improves vulnerability detection traffic by understanding the business logic and setting the race window for critical processes so that it detects race conditions by sending limited traffic. As a result, unlike previous solutions, it prevents DoS attacks on applications. The proposed model, because of the knowledge of the business logic and the identification of duplicate processes, prevents the infinite spread of the model in large applications and identifies vulnerabilities only by analyzing the user's HTTP traffic with no administrator access. The proposed method is independent of the written language of web applications and detects vulnerabilities automatically.

The contributions of this study are:

  • Provide a comprehensive list of race conditions types available in the web application, categorize and name race conditions based on race conditions and the conditions for creating them

  • Define the race conditions and analyze the conditions for their creation

  • Define the race window for the various race conditions available on the web application

  • Presenting the black-box approach for detecting race-prone processes in the web applications, Semantic Web Racer by understanding the application’s business logic and defining a trace pattern for different race conditions, can identify race-prone business processes

  • Presenting the black-box approach for the dynamic application security testing of the web application to detect business-layer vulnerabilities against server-side race conditions, Semantic Web Racer detects application vulnerabilities by running critical business processes in the race window. Critical processes are performed in a specific chronological order in both normal mode and race prone mode.

  • Presenting the black-box approach for the dynamic application security testing of the web application to detect business-layer vulnerabilities against client-side race conditions

  • Improved traffic of vulnerability detection by 92.8%. Semantic Web Racer, because of its knowledge of business logic, determining the race window for critical processes, and executing critical processes in a specific chronological order, has significantly improved traffic of vulnerability detection by about 92.8% and Prevents DoS attacks during the test scenario.

In this study, section 2 reviews the previous studies. Section 3 defines race condition types in details. Section 4 presents the proposed method. Section 5 and 6 shows implementation, experimental results and evaluation. Finally, the paper is concluded in section 7.

Section snippets

Related work

In this section, the existing solutions for identifying the race conditions are studied and the weaknesses and problems of the existing approach are expressed.

Existing approaches to detect race conditions only identify a few server-side race conditions or a few client-side race conditions and cannot identify both server-side race conditions and client-side race conditions. They focused on one type of race conditions, and none provided a general list of race conditions.

Methods of ACIDRain

Defining race condition and types of race condition

In web applications, the race condition is divided into two categories, server-side race and client-side race. Server-side race conditions occur when different processes access common data without proper synchronization and at least one of them intends to write on the common data. The race condition originates from the bugs of programmers because of improper synchronization. A server-side race condition occurs when the operations are not atomic (Mutlu, 2016). A client-side race condition occurs

Semantic web Racer: Dynamic application security testing in the business layer against race condition

To evaluate the security of the application against race conditions, Semantic Web Racer is proposed. Semantic Web Racer employs dynamic web application security testing in the business layer to detect the vulnerability of the application against race condition attacks. Fig. 19 depicts the Semantic Web Racer overview. Semantic Web Racer is situated as a proxy between the web application and the web server

To detect a race condition in the business layer of the application, Semantic Web Racer

Implementation

First, a normal user crawls the web application sequentially and in-depth but its traffic is saved. According to the user roles, a user can have different levels of access; therefore, different user navigation graphs can be got based on the levels of access. In this research, the normal user has the user access level and navigates the authorized pages. The normal user crawls all different parts of the application that are allowed to navigate. If the user reaches the page without a hyperlink,

Experiment results and evaluation

To evaluate the proposed method, Semantic Web Racer applies to different benchmark applications. The experiments are applied to 6 open-source and well-known applications that may be loaded offline. Table 2 shows the applications used for evaluations; Opencart v3.0.3.3,1 Ror_ecommerce,2 Lightning Fast Shop,3 Broadleaf,4 MyBB v1.8.15,5

Conclusion

A race condition occurs when the synchronization of processes affects each other, resulting in unpredictable behavior of the application. The approaches that have been proposed so far for detecting race conditions detect race without considering the application’s business logic. Therefore, they can only detect a few race conditions in the applications. The existing approach to identify the race condition, apply many transactions to the application to create a race condition in the application,

Future work

In this article, we focus on the business logic vulnerabilities of web applications against race conditions and present a black box solution to identify them. The proposed solution can be developed and, when identifying critical processes, processes that are prone to other types of business layer attacks can be identified. Other attack vectors in the business layer include Abuse of Functionality, Insufficient Process Validation, Insufficient Authentication, and Insufficient Authorization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (28)

  • C.Q. Adamsen

    Automated Testing Techniques for Event-Driven and Dynamically Typed Software Applications

    (2018)
  • Adamsen, C. Q., Moller, A., Karim, R., Sridharan, M., Tip, F., & Sen, K. (2017). Repairing event race errors by...
  • C.Q. Adamsen et al.

    Practical initialization race detection for JavaScript web applications

    Proceedings of the ACM on Programming Languages

    (2017)
  • C.Q. Adamsen et al.

    Practical AJAX race detection for JavaScriptWeb applications

  • Alidoosti, M., & Nowroozi, A. (2018). BLProM: Business-Layer Process Miner of the Web Application. In 2018 15th...
  • M. Alidoosti et al.

    BLProM: A black-box approach for detecting business-layer processes in the web applications

    Journal of Computing and Security

    (2019)
  • M. Alidoosti et al.

    Evaluating the web-application resiliency to business-layer DoS attacks

    ETRI Journal

    (2020)
  • Braun, B., Pollak, C. V., & Posegga, J. (2013). A Survey on Control-Flow Integrity Means in Web Application Frameworks....
  • R.J. Van Emous

    Towards Systematic Black-Box Testing for Exploitable Race Conditions in Web Apps

    (2019)
  • Endo, A. T., & Moller, A. (2020). NodeRacer: Event Race Detection for Node.js Applications. In Proceedings - 2020 IEEE...
  • Halfond, W. G. J., Viegas, J., & Orso, A. (2006). A Classification of SQL Injection Attacks and Countermeasures. In In:...
  • Hong, S., Park, Y., & Kim, M. (2014). Detecting concurrency errors in client-side java script web applications. In...
  • S. Koch et al.

    Raccoon: Automated verification of guarded race conditions in web applications

  • H. Liang et al.

    Automated data race bugs addition

  • Cited by (3)

    View full text