An online cross view difference and behavior based kernel rootkit detector

Kernel level rootkits pose a serious threat today as they not only mask the presence of themselves but also mask the malware that comes attached with them. Rootkits achieve such stealthy behavior by manipulating the control flow of system calls by hooks and kernel objects, viz., driver and process list directly. Existing Antiviruses that rely on signature based techniques for detection of malwares are effective only against known rootkits. However, as hackers change coding style of rootkits, Antiviruses fail to detect them and rootkits and their malicious activities are hidden from the view of the administrator. Thus, all data on the compromised system becomes vulnerable to theft and all services running on it can be misused by the remote attacker without even the slightest chance of being discovered. Other rootkit detection techniques such as integrity checking, alternate trusted medium, and memory dumping require frequent offline analysis and fail to unload or block the rootkit.

This paper addresses, these challenges and proposes an online cross view difference and behavior based kernel rootkit detector to overcome them. Our proposed solution Kernel Rootkit Trojan Detector (KeRTD) is a host-based and cross view difference-based solution that enables online analysis and aids detection of rootkit immediately. A simple view difference of snapshot of Task manager in user mode and KeRTD Process and Driver List helps the detection of hidden rootkits and other hidden malwares. All rootkits follow a generic pattern of infection such as installing kernel hooks and modification of kernel objects, etc. This very generic behavior of rootkit is exploited in KeRTD to detect and restore the kernel hooks, thus blocking them from further infection. Every file and memory accesses are verified against Access Control List to avoid subversion of KeRTD and operating system kernel. This proposal has been implemented on windows operating system and tested for various methods of attack by kernel rootkits. The results confirm the detection of the kernel rootkits.