BotMosaic: Collaborative network watermark for the detection of IRC-based botnets

https://doi.org/10.1016/j.jss.2012.11.005Get rights and content

Abstract

Recent research has made great strides in the field of detecting botnets. However, botnets of all kinds continue to plague the Internet, as many ISPs and organizations do not deploy these techniques. We aim to mitigate this state by creating a very low-cost method of detecting infected bot host. Our approach is to leverage the botnet detection work carried out by some organizations to easily locate collaborating bots elsewhere.

We created BotMosaic as a countermeasure to IRC-based botnets. BotMosaic relies on captured bot instances controlled by a watermarker, who inserts a particular pattern into their network traffic. This pattern can then be detected at a very low cost by client organizations and the watermark can be tuned to provide acceptable false-positive rates. A novel feature of the watermark is that it is inserted collaboratively into the flows of multiple captured bots at once, in order to ensure the signal is strong enough to be detected. BotMosaic can also be used to detect stepping stones and to help trace back to the botmaster. It is content agnostic and can operate on encrypted traffic. We evaluate BotMosaic using simulations and a testbed deployment.

Highlights

► We propose a novel application for information hiding. ► A light-weight system for detection of botnets. ► The very first collaborative design for flow watermarking. ► Our system is able to detect not only the bots, but also the botmasters, and the compromised machines in a deploying network. ► BotMosaic provides very tiny false errors in its detection.

Introduction

A botnet is a network of compromised machines, bots, that is controlled by one or more botmasters to perform coordinated malicious activity. Botnets are among the most serious threats in cyberspace due to their large size (Ramachandran and Feamster, 2006). This enables the bots to carry out various attacks, such as distributed denial of service, spam, and identity theft, on a massive scale.

Botnets are controlled by means of a command-and-control (C&C) channel. A common approach is to use an Internet Relay Chat (IRC) channel for C&C: all the bots and a botmaster join a channel and the botmaster uses the channel to broadcast commands, with responses being sent back via broadcast or private messages to the botmaster. The IRC protocol is designed to support large groups of users and a network of servers to provide scalability and resilience to failures, thus it forms a good fit for providing a C&C infrastructure. Because of their simple design and deployment, IRC botnets have been widely used by cybercriminals since 2001 (Kharouni, 2009). Some botnets use a more advanced structure, with bots communicating directly with each other in a peer-to-peer fashion, but recent studies show that many existing botnets use the IRC model because of its simple-yet-effective structure (Kharouni, 2009, Zhuge et al., 2007). In this research we focus on the IRC botnets.

Much research has been devoted to the detection of IRC botnets (Binkley and Singh, 2006, Ramachandran et al., 2006, Karasaridis et al., 2007, Collins et al., 2007, Villamarín-Salomón and Brustoloni, 2009, Zilong et al., 2010). However, most effective detection techniques are complex and have potential to generate false positives. This means that organizations with a large security budget are able to find potential bot infections and disable, investigate, and disinfect affected machines. Organizations with less developed IT practices, as well as home users, however, remain vulnerable to bot infections and provide a fertile ground for botnets, allowing them to remain strong.

We propose a technique that follows a service model. It leverages the efforts of one organization to capture and instantiate bot instances to provide low-cost detection of bots in other networks. We develop BotMosaic – a watermark that, when inserted into the communication between the captured bots and an IRC server, creates a pattern that is observable at other sites hosting botnets. The pattern can be recognized simply by observing the timings of the packets in a given flow, thus the detection can be carried out at a large scale by border routers. By inserting an artificial pattern, we can ensure that false-positive rates are very low, enabling automated actions to disconnect infected bots. Since only packet timings are used, BotMosaic works even when the botnet uses encrypted connections to the IRC server.

The watermark will be visible on all connections between the bots and the IRC server. It will likewise appear in the connection from the botmaster to the IRC server. Botmasters typically use stepping stones (Zhang and Paxson, 2000) to hide their true location. The watermark can be used to detect such stepping stones and aid in botmaster traceback.

A novel and unique feature of our watermark is that it is collaborative: the watermark is inserted simultaneously into the flows of all captured bots. This is in contrast to past watermarks that affect a single flow at a time (Wang and Reeves, 2003, Wang et al., 2005, Wang et al., 2007, Pyun et al., 2007, Yu et al., 2007, Houmansadr et al., 2009b, Ramsbrock et al., 2008). The collaborative feature amplifies the effect of the watermark and is necessary to create a timing pattern that is recognizable among the noise generated by traffic from other bots. In other words, this collaborative behavior allows a BotMosaic watermark to persist on the watermarked flows even after they are mixed with other flows in the botnet's C&C channel.

In summary, BotMosaic has the following unique features as compared to previous approaches: (1) BotMosaic is implemented by one organization, and can be used as a low-cost service by other organizations, i.e., clients. A client organization only needs to deploy the low-cost watermark detectors of BotMosaic on their border routers. This is in contrast to other approaches that suggest each organization to deploy its own, resource-intensive botnet detection mechanism. (2) A client organization can use BotMosaic to detect various instances of bots simultaneously, without the need to modify its BotMosaic detectors for different botnets. The BotMosaic watermarkers use different watermark signals for different instances of botnets. (3) Each client organization can detect not only the bot infected machines, but also the botmasters and stepping stones hosts residing inside their networks.

We analyze our scheme using simulations and experiments on PlanetLab (Bavier et al., 2004). We find that we can achieve a high rate of detection with few false positives using a watermark applied to captured/imitated bots that comprise a small fraction of the botnet, with a detection time of about a minute.

The rest of the paper is organized as follows: Section 2 describes previous work on IRC botnet detection and reviews past work on network flow watermarking. Section 3 describes the overall detection framework used by BotMosaic. Section 4 describes the detailed structure of the BotMosaic collaborative watermark. Simulations and implementation results are presented in Section 5. Section 6 offers a brief discussion of some additional issues, and Section 7 concludes the paper.

Section snippets

Related work and motivation

The primary goal of the BotMosaic is to detect bot-infected machines inside a network of interest, e.g., an ISP. The literature on this can be divided into host-based and network-based approaches. Host-based approaches analyze the information on hosts of the network; this is not easy to deploy on all hosts, especially in organizations where computers are not centrally managed. BotMosaic falls in the network-based category.

Network-based detection mechanisms aim to detect bot infected machines by

BotMosaic detection framework

In this section we describe the features of IRC botnets exploited by BotMosaic and its deployment scenarios.

BotMosaic watermarking scheme

In this section, we describe the watermarking scheme that we devise to be used for the BotMosaic botnet traceback system. The watermark is novel in being collaborative: the BotMosaic service provider uses multiple captured bots for watermark insertion, which makes the scheme specialized for the problem of botnet watermarking. Multiple captured bots allow us to spread the watermark power over a larger amount of traffic, compensating for the small amount of traffic each individual bot sends to

Simulations and experiments

In the simulations and experiments of this section we only consider the detection of BotMosaic watermarks being inserted into the traffic towards the botmaster. The detection of watermarks on traffic to bots is similar; however, botmaster detection is more difficult as the botmaster traffic is relayed through a number of stepping stones, resulting in more delays affecting the watermark pattern.

Discussion

We briefly discuss several other issues regarding the BotMosaic scheme.

Conclusion

We have presented a new botnet traceback scheme, BotMosaic, that detects bot infected machines and helps to track down the botmasters controlling the centralized botnets. BotMosaic uses a service-based approach where detector clients perform fast and low-cost watermark detection, which is much cheaper and easier to deploy than existing signature- and classification-based detectors. We presented a new collaborative flow watermarking structure, making it suitable for the botnet detection problem.

Acknowledgment

This work was supported in part by the National Science Foundation grant CNS 0831488 as well as the Boeing Trusted Software Center at the Information Trust Institute at the University of Illinois.

Amir Houmansadr received his PhD in electrical and computer engineering from the University of Illinois at Urbana-Champaign in August 2012. He is currently a postdoctoral researcher at the Computer Sciences department of the University of Texas at Austin. Amir's research revolves around various network security and privacy problems, including network traffic analysis, intrusion detection, covert channels, and anonymous communications. In particular, Amir has investigated the design and analysis

References (34)

  • P.Y. Lin et al.

    Distortion-free secret image sharing mechanism using modulus operator

    Pattern Recognition

    (2009)
  • G. Ulutas et al.

    Distortion free geometry based secret image sharing

    Procedia

    (2011)
  • Bavier, A., Bowman, M., Chun, B., Culler, D., Karlin, S., Muir, S., Peterson, L., Roscoe, T., Spalink, T., Wawrzoniak,...
  • S. Bhattacharya et al.

    A distortion free watermark framework for relational databases

  • S. Bhattacharya et al.

    A generic distortion free watermarking technique for relational databases

  • J.R. Binkley et al.

    An algorithm for anomaly-based botnet detection.

  • M.P. Collins et al.

    Using uncleanliness to predict future botnet addresses

  • J. Goebel et al.

    Rishi: identify bot contaminated hosts by IRC nickname evaluation

  • G. Gu et al.

    BotSniffer: detecting botnet command and control channels in network traffic

  • A. Houmansadr et al.

    SWIRL: a scalable watermark to detect correlated network flows

  • A. Houmansadr et al.

    Multi-flow attack resistant watermarks for network flows

  • A. Houmansadr et al.

    Rainbow: a robust and invisible non-blind watermark for network flows

  • Kalt, C., 2000. Internet Relay Chat: Server Protocol. RFC 2813 (Informational)....
  • I. Kamel et al.

    Distortion-free watermarking scheme for wireless sensor networks

  • A. Karasaridis et al.

    Wide-scale botnet detection and characterization

  • Kharouni, L., 2009. SDBOT IRC botnet continues to make waves. White paper, Trend Micro Threat...
  • N. Kiyavash et al.

    Multi-flow attacks against network flow watermarking schemes

  • Cited by (22)

    • A new Intelligent Satellite Deep Learning Network Forensic framework for smart satellite networks

      2022, Computers and Electrical Engineering
      Citation Excerpt :

      Through the clone-based analysis, parts of the malware code that were originally discovered in different malware are identified and as such, the portion of code that needs to be reviewed is reduced. A tool named BotMosaic was introduced by Houmansadr et al. [11], with its main functionality focusing on Botnet traffic detection and type identification. BotMosaic detects IRC Botnets by utilizes a watermarking technique for NetFlow traffic, which is non-distorting thus marking traffic without altering it for later identification.

    • Botnet detection via mining of traffic flow characteristics

      2016, Computers and Electrical Engineering
      Citation Excerpt :

      Botnets can be centralized, decentralized or hybrid according to their C&C channels and communication protocols like HTTP, P2P, IRC, IM, etc. IRC-based centralized C&C structure is the most commonly used botnet structure [3]. In this, all the bots in a botnet are connected to a single C&C channel to obtain the commands from the botmaster.

    • Digital Watermarking for Detecting Malicious Intellectual Property Cores in NoC Architectures

      2022, IEEE Transactions on Very Large Scale Integration (VLSI) Systems
    • Securing on-chip communication using digital watermarking

      2021, Network-on-Chip Security and Privacy
    View all citing articles on Scopus

    Amir Houmansadr received his PhD in electrical and computer engineering from the University of Illinois at Urbana-Champaign in August 2012. He is currently a postdoctoral researcher at the Computer Sciences department of the University of Texas at Austin. Amir's research revolves around various network security and privacy problems, including network traffic analysis, intrusion detection, covert channels, and anonymous communications. In particular, Amir has investigated the design and analysis of active network traffic analysis schemes, called flow watermarks, towards his PhD dissertation.

    Nikita Borisov is an Associate Professor of Electrical and Computer Engineering at the University of Illinois at Urbana-Champaign. His research interests include online security and privacy. He is the co-designer of the “off-the-record” (OTR) instant messaging protocol and was responsible for the first public security analysis of 802.11 security. He received the National Science Foundation CAREER Award in 2010. He graduated from the University of California, Berkeley with a PhD in Computer Science in 2005.

    View full text