Toward a general defense against kernel queue hooking attacks
Introduction
Rootkits have become one of the most dangerous threats to systems security. Because they run at the same privilege level as the operating systems kernel, they can modify the OS behavior in arbitrary ways. For example, they can tamper with existing code and/or data of the OS to conceal the runtime state of the system in terms of running processes, network connections, and files. Besides, they can add new functionalities to the OS to carry out malicious activities such as key logging and sensitive information collection. In recent years, significant work has been proposed to detect (Baliga et al., 2008; Carbone et al., 2009; Yin et al., 2010; Garfinkel and Rosenblum, 2003; Jiang et al., 2007; Petroni et al., 2004; Petroni and Hicks, 2007; Rutkowska, 2005; Wang et al., 2005) analyze (Lanzi et al., 2009; Riley et al., 2009; Wang et al., 2008; Yin et al., 2007, 2008), or defend against (Litty et al., 2008; Riley et al., 2008; Seshadri et al., 2007; Wang et al., 2009) rootkits.
However, existing work on rootkits mainly focuses on attacks that change legitimate kernel code (Petroni et al., 2004; Litty et al., 2008; Riley et al., 2008; Seshadri et al., 2007) or change legitimate kernel hooks (locations in kernel space that hold function pointers) (Petroni and Hicks, 2007; Wang et al., 2009), but falls short of attacks that create malicious hooks in dynamically allocated kernel objects, as demonstrated by kernel queue hooking (KQH) attacks (Section 2). Briefly speaking, KQH rootkits leverage various callback mechanisms of the kernel, which enable the rootkits to direct kernel control flow as effectively as exploiting buffer overflows, and by reusing legitimate kernel code, these rootkits can successfully hijack control flow of the victim kernel, yet remain invisible to state-of-the-art defense techniques. Specifically, KQH attacks are unique in three important ways:
- •
They do not hijack existing, legitimate kernel hooks; instead, they create their own malicious kernel hooks.
- •
They leverage kernel data structures that can have multiple instances at the same time. For example, although the Linux kernel allows each type of interrupt to have only one handler registered in the interrupt descriptor table (IDT), it allows multiple IRQ action handlers for the same interrupt to coexist.
- •
They leverage dynamic kernel data structures. Again take IRQ action handlers as example, the kernel uses a queue to keep track of currently registered IRQ action handlers and this queue can grow or shrink at runtime depending on which entities are interested in a particular IRQ, including rootkits.
Therefore, it is much harder to detect malicious manipulations of the IRQ action queue than those of the IDT because it is non-trivial to find the known-good values for the IRQ action queue; and it is also much harder to defend against KQH attacks on the IRQ action queue. For example, we cannot simply make the embedded hooks immutable (as proposed by HookSafe (Wang et al., 2009)), because we do not know whether a hook is benign or malicious to start with.
For ease of presentation, we call data structures such as the IRQ action queue kernel schedulable queues (or K-Queues for short), and attacks that insert malicious requests to such queue-like data structures K-Queue hooking (KQH) attacks. We elaborate on the difference between KQH attacks and other attacks in Section 3.2.
Moreover, as we will discuss in Section 2.2, advanced and real malware is already misusing K-Queues to their advantage; examples of such malware include major spam bots such as Rustock, Pushdo/Cutwail, and Storm/Peacomm.
Therefore, the technical novelty and the realistic threat of KQH attacks call for new defense approaches. The challenge is that malicious data objects share the same K-Queues with legitimate data objects, and both kinds can be created and destroyed at runtime. So a reasonable defense has to check the legitimacy of each object each time the kernel uses it for control transfer decisions (because such objects may not be persistent). To this end, we design and implement a hypervisor-supported reference monitor that intercepts and rejects malicious callback requests while allowing legitimate kernel callback requests to proceed. Secondly, it is very subtle and tricky to develop a specification for legitimate kernel data objects. Manually doing this for a code base as large as the Linux kernel is hopeless and error-prone at best. To address this issue, we build automated tools to derive such specifications more efficiently. Moreover, we employ code generation techniques to automatically translate the inferred specifications into runtime check code.
Specifically, we employ static program analysis (e.g., points-to analysis and transitive closure analysis) of the kernel source code to derive legitimacy specifications. To facilitate the generation of specifications for the entire class of K-Queues, we build a unified static analysis framework and a set of tools that can not only derive specifications from kernel source code, but also generate the corresponding checking code for different K-Queue instances. Furthermore, we design and implement a proof-of-concept runtime reference monitor that checks pending requests for four types of K-Queues (IRQ action queue, tasklet queue, soft timer queue, and task queue) in the Linux kernel and guards the check result against tampering. Finally, we perform a comprehensive experimental evaluation of our tools as well as a detailed performance overhead analysis of the reference monitor, which shows that our approach is able to detect all sample KQH rootkits that we have.
The rest of this paper is organized as follows. Section 2 introduces KQH attacks and the threats they pose to systems security. Section 3 presents our modeling of KQH attacks and a high level description of our defense. Section 4 discusses our design of a unified static analysis framework and tools that can generate specifications of legitimate K-Queue requests and turn them into checking code. Section 5 presents our implementation of the defense. Section 6 discusses evaluation of our defense, Section 7 talks about related work, and Section 8 concludes the paper.
Section snippets
Kernel queue hooking
Kernel Queue Hooking (KQH) is a rootkit technique that leverages extensible kernel data structures, such as queues, to inject malicious control flows into the victim kernel. The queues contain instances of dynamic kernel data structures and they can be hooked because they are organized in linked lists. Finally, running malicious logic is possible because such data structures contain function pointers.
Threat modeling
We adopt the standard queuing model for K-Queues. That is, each K-Queue contains a request queue and a server. A request is inserted into the queue by an enqueue operation and it is removed from the queue by a dequeue operation by the server. This model works well for the K-Queues. For example, all instances of K-Queues provide APIs (the enqueue operations) for submitting an execution request for some callback function, and they all have a dispatch engine (the server) that takes the request
Overview
We build a unified static analysis framework (Fig. 8) and a set of tools that can be used to derive security specifications for legitimate K-Queue requests. We assume that source code of the kernel is available for a static analysis, and such code is the only one that can be trusted. We note that although details of different K-Queues may vary, their specifications can be derived by a common set of analysis tasks. For example, the top-level legitimate K-Queue callback functions can be derived
The K-queue analyzers
We implement the static analyzers for the IRQ action queue, the tasklet queue, the task queue, and the soft timer queue based on our static analysis framework, using CIL (C Intermediate Language) (Necula et al., 2002). We implement the analysis engine in Shell scripts, which invokes our CIL modules that implement the basic analysis tools (Section 4.5). CIL provides an implementation of OLF (Das, 2000), but it is not field-sensitive, so we improve it to satisfy our needs. All of our CIL modules
Effectiveness against KQH attacks
We test the effectiveness of our K-Queue defense by running synthetic proof-of-concept rootkits in the guest kernel, including the keylogger and the CPU cycle stealer described in (Wei et al., 2008), and the keylogger described in Section 2. These rootkits leverage the Linux K-Queues in Table 1 and employ type 1 or type 2 attacks (Section 3.2.3). We observe that our K-Queue guards can immediately detect such rootkits. This shows that our defense is effective again KQH attacks.
Our K-Queue
Related work
Rootkits have attracted a lot of attention in the research community. Existing work can be classified into three areas: detection, defense, and analysis.
Related work in detecting rootkit execution includes integrity-based approaches such as Gibraltar (Baliga et al., 2008), HookScout (Yin et al., 2010), Livewire (Garfinkel and Rosenblum, 2003), Copilot (Petroni et al., 2004), SBCFI (Petroni and Hicks, 2007), Patagonix (Litty et al., 2008), and System Virginity Verifier (Rutkowska, 2005), and
Conclusion
We present a solution to kernel queue hooking (KQH) attacks that manipulate K-Queues to achieve stealthy and continual malicious function execution. Such attacks are actively used by advanced malware such as the Rustock spam bot, but they remain invisible to state-of-the-art kernel integrity monitors. We propose the PLCP (Precise Lookahead Checking of function Pointers) approach that checks the legitimacy of pending K-Queue requests by proactively checking function pointers that may be invoked
Jinpeng Wei received a PhD in Computer Science from Georgia Institute of Technology, Atlanta, GA in 2009. He is currently an assistant professor at the School of Computing and Information Sciences, Florida International University, Miami, FL. His research interests include malware detection, information flow security in distributed systems, could computing security, and file-based race condition vulnerabilities. He is a member of the IEEE and the ACM.
References (39)
- Abadi M, Budiu M, Erlingsson U, Ligatti J. Control-flow integrity. In: Proceedings of the 12th ACM conference on...
- Anderson LO. Program analysis and specialization for the C programming language. PhD thesis: University of Copenhagen;...
- Arbaugh WA, Farber DJ, Smith JM. A secure and reliable bootstrap architecture. In: Proceedings of IEEE symposium on...
- Baliga A, Ganapathy V, Iftode L. Automatic inference and enforcement of kernel data structure invariants. In:...
- Barham P, Dragovic B, Fraser K, Hand S, Harris T, Ho A, et al. Xen and the art of virtualization. In: Proceedings of...
- Becher M, Dornseif M, Klein CN. FireWire: all your memory are belong to us. In: Proceedings of CanSecWest;...
- et al.
Checking for race conditions in file accesses
Computing Systems
(Spring 1996) Peacomm.C – cracking the nutshell
(September 2007)- Carbone M, Cui W, Lu L, Lee W, Peinado M, Jiang X. Mapping kernel objects to enable systematic integrity checking. In:...
- Das M. Unification-based pointer analysis with directional assignments. In: Proceedings of the 2000 ACM SIGPLAN...
Pushdo/Cutwail: a study of the Pushdo/Cutwail botnet
Yet another Rustock analysis
Virus Bulletin
Cited by (5)
Binary-centric defense of production operating systems against kernel queue injection attacks
2019, Journal of Computer Virology and Hacking TechniquesRandomization of soft keyboard layout based on cellular automata
2013, Beijing Jiaotong Daxue Xuebao/Journal of Beijing Jiaotong UniversityKQguard: Binary-centric defense against kernel queue injection attacks
2013, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Jinpeng Wei received a PhD in Computer Science from Georgia Institute of Technology, Atlanta, GA in 2009. He is currently an assistant professor at the School of Computing and Information Sciences, Florida International University, Miami, FL. His research interests include malware detection, information flow security in distributed systems, could computing security, and file-based race condition vulnerabilities. He is a member of the IEEE and the ACM.
Calton Pu received the PhD degree from the University of Washington in 1986. He is a professor and the John P. Imlay Jr. chair in software at Georgia Institute of Technology, Atlanta, GA. He has published more than 60 journal papers and book chapters, 170 refereed workshop and conference papers in operating systems, transaction processing, systems reliability and security, and Internet data management. He has served on more than 100 program committees for more than 50 international conferences and workshops. He is a member of the ACM, a senior member of the IEEE, and a fellow of the AAAS.