Practical Black-Box Analysis for Network Functions and Services

Modern networks are exploding with an increasing array of diverse network functions (e.g., network firewalls) and services (e.g., public servers). Despite their critical role in our modern infrastructure, they remain largely black-box in nature, given that they are proprietary or configured and deployed by third parties. This black-box nature makes it fundamentally difficult for operators and Internet security

experts to reason about security implications and correctness of these functions and services. Unfortunately, this lack of understanding and analysis leaves gaps for

high-impact network attacks exploiting their insecurities and network outages. This dissertation aims to bridge this operational gap by building techniques to automatically analyze the behavior and vulnerabilities of these network devices and services. Specifically, we design techniques to (1) automatically infer high-fidelity models to enable accurate testing and verification, and (2) identify new avenues for potential abuse against network functions and services. Given that we only have black-box access, our techniques do not require access to the code or binary for

instrumentation. However, designing these techniques is challenging. First, we need to reason about their behavior under a large traffic space and possible configurations.

Second, they may exhibit complicated (hidden) behaviors. Our high-level approach in building these tools is to leverage structural properties inherent to black-boxes and their input and configuration space. This insight allows us to reduce the relevant search space and efficiently search over the relevant part of the search space. The key contributions of this thesis are three concrete tools. First, is Alembic, a

tool that can automatically synthesize high-fidelity models of stateful network functions, for accurate testing and verification workflow. Second, is Pryde, a tool which

provides operators with capabilities for identifying subtle evasion vulnerabilities in stateful firewalls. Lastly, is AmpMap, a low-footprint measurement framework that

can systematically quantify the amplification risk against black-box protocol servers at scale. In presenting each of these tools, we highlight how each tool (1) uncovered

unexpected behavior and new security vulnerabilities, and (2) highlighted significant variability in the behavior and security implications of these black-boxes across

vendors and implementations. Our findings and results affirm the need for automatic tools to analyze the behaviors for black-box functions and services to properly understand their security implications.