From student research to intrusion detection

We describe a multi-year project that began as mostly undergraduate student research in data mining applied to computer forensics and has now grown into a prototype for an intrusion detection system. The IDS assumes we have delimited data that can be separated into records such as IP packets, system calls, etc. The data mining approach uses the Bag of Words methodology where we form a matrix model of the data, and then cluster the records using k-means clustering and sparse nonnegative matrix factorization. With no training, these clusters are evaluated to determine if they represent normal system actions or attack vectors. This prototype system has accuracy levels similar to systems that use supervised learning on a specific set of data. We discuss future plans to make improvements with continued student investigation. Overall, we found this to be a great partnership between faculty and student research.