1 Introduction

1.1 Background

A botnet refers to a group of compromised computers that are remotely controlled by a botmaster via C&C channels [1]. Based on botnets, multiple types of Internet attacks can be initiated, such as: DDoS (Distributed Denial of Service), Email Spam, Bitcoin or Monero Mining, etc. At present, the studies on botnets can be summarized into two aspects: attack technology and defense technology. The purpose of studying attack technology is predicting the attack trends and techniques of future botnets, so as to prevent the possible emerging botnet activities; and the purpose of studying on defense technology is improving the detection efficiency of botnets and discovering the botnets that are already in the cyberspace but not yet exposed in a timely manner, so as to reduce the actual harm caused by them.

In the early days, attackers usually controlled the bot based on the IRC [2, 3] or HTTP [4, 5] protocol. This centralized architecture is simple, efficient and highly interactive, but is vulnerable to single point of failure. Although a modified architecture based on the Domain-Flux [6] or Fast-Flux [7] protocol that appeared later can eliminate this problem certainly, it may be attacked by Sinkhole [8]. In order to make up for the deficiency of centralized botnets, botnets using P2P protocols as C&C channels have also evolved. In a P2P botnet, the botmaster can issue commands at any node, so it can hide the real address of the C&C server and effectively solve the single point of failure. However, P2P botnets are not perfect, which still have inherent weakness. For example, the structured P2P botnets, such as Storm [9, 10], are vulnerable to Index Pollution attack and Sybil attack, and its scale is easy to be measured by Crawler and Sybil nodes; the unstructured P2P botnets usually communicate by the way of random scanning or peer-list, the former has the inherent weaknesses of flow anomaly, and the latter is vulnerable to Peer-list Pollution attack.

In recent years, the new generation of botnets based on social network have been proposed, such as: Koobface [11], Stegobot [12], etc. Among the social botnet, each social account is a control node, which is equivalent to a C&C server in the traditional botnet, and is used to transfer the commands between the botmaster and individual bots. Although the social botnet can hide the malicious traffic within the normal legitimate traffic, the social platforms are generally only applicable for the botmaster issuing commands and cannot be used by bots to send back harvested information, especially file information. Moreover, for the botnet based solely on the social platform, its C&C channel is relatively simple, and it can be easily detected and destroyed by the defender. Therefore, in order to make up for the inadequacies of social botnets, this paper proposes an advanced botnet based on multiple publicly available resources.

1.2 Contribution

The goal of this paper is to study the development trends of future botnets, increase the defenders’ understanding of the advanced botnet, and promote more effective cyber defense to deal with the possible similar cases.

The contributions of this paper mainly include three aspects:

  1. (1)

    Based on the idea of “severless botnet”, an advanced botnet based on publicly available resources is designed. The system adopts a three-channel scheme, and each sub-channel can be supported by multiple publicly available resources and extended in the form of plug-in.

  2. (2)

    We have tested five categories and 37 websites, and the application scenarios of each website when constructing C&C channel are discussed. Meanwhile, the operation flow between the botmaster and individual bots is analyzed and described in detail to verify the feasibility of the proposed model.

  3. (3)

    We have analyzed the attributes and weaknesses of the PR-Bot, and propose a practical targeted defense scheme that covers detection, measurement, and tracing.

2 The Design of PR-Bot

2.1 System Overview

Definition 1 (Publicly Available Resource Botnet).

In this paper, we believe that all botnets that construct C&C channels based on the publicly available resources (including but not limited to: social network, URL shortener, image hosting, online clipboard or cloud disk, etc.) could be called the Publicly Available Resource Botnet (Fig. 1).

Fig. 1.
figure 1

Basic characteristics of publicly available resource Botnet

Full size image

The basic characteristics of this type of botnets are: the botmaster no longer relies on the self-built C&C server to control the bots, but uses the open and free website on the Internet to act as the C&C server. All communication flows are transferred through the Internet publicly available resources.

2.2 System Design

In botnets, no matter how complex the control model of botnet is or how powerful the bot program is, the interaction between the control end and the controlled end usually involves only the transfer of text information (that is: string content) or file information (that is: binary content), as shown in Table 1.

Table 1. Interactive information between the control end and the controlled end
Full size table

Although there are many kinds of publicly available resources on the Internet, due to the limitations of the nature of the publicly available resources, not all publicly available resources are suitable for issuing both text information and file information. For example, the social platforms used by social botnets are generally only applicable to store the commands issued by the botmaster, but not applicable to store the harvested information sent back by individual bots, making it a one-way communication channel.

The PR-Bot, an advanced botnet designed in this paper, takes into account the limitations of a purely social platform as C&C channels, so it combines multiple publicly available resources and uses their respective advantages to construct C&C channels. The PR-Bot is suitable for transmitting both text information and file information, as well as supporting two-way communication between the botmaster and bots. All in all, the PR-Bot adopts the three-channel scheme, and the interaction information in each channel is distributed in different locations of cyberspace, as shown in Fig. 2.

Fig. 2.
figure 2

The control model of PR-Bot

Full size image

2.3 System Architecture

The architecture of PR-Bot is shown in Fig. 3, whose communication between the botmaster and bots includes the following six stages:

Fig. 3.
figure 3

The system architecture of PR-Bot

Full size image

  • (1) Botmaster issues the content of command: The botmaster issues the content of command to the publicly available resources, such as an online clipboard or image hosting website, and records the URL of the website (abbreviated as PR_Address_A) where the command is located. For the online clipboard, the command is issued in the form of string; for the image hosting, the command is first converted into a picture and then issued in the form of picture.

  • (2) Botmaster issues the address of command: First, the botmaster selects a URL shortener or social network that supports customized URL, designs and runs the Username Generation Algorithm (UGA) [13], followed by selecting several candidate addresses (abbreviated as PR_Address_B) from the URL address pool. And then, the botmaster issues the address PR_Address_A as content to the website corresponding to the address PR_Address_B.

  • (3) Bot obtains the address of command: After the bot infects the controlled end, to establish the communication with the control end, it will first run the UGA algorithm consistent with the botmaster, and then traverse the URL address pool one by one. When an address (take PR_Address_B as an example here) is found to be accessible, it is considered that address of command is stored at this place, and then the address PR_Address_A will be extracted.

  • (4) Bot obtains the content of command: After obtaining the content of command from PR_Address_A in Stage 3, the bot first verifies its validity and availability, and only the command that passes verification can be executed. Otherwise, the control logic of the bot program will jump to Stage 3 and run again. Among them, “validity” refers to whether the command is within the validity period specified by the botmaster; “availability” refers to whether the command is signed by the private key of the botmaster.

  • (5) Bot feeds back the result information: For the command that needs to feed back the result information, the botmaster is required to specify the receiving address as a parameter in the issued command. The bot will feed back the relevant information based on this parameter in the command and the customized protocol with the control end.

  • (6) Botmaster downloads the result information: After a certain period of time, for the command that will feed back the result information, the corresponding result acquisition module will be run. During the operation of the module, it will download the text information (such as callhome information) or file information (such as stolen files) returned by the bot for further analysis, which is based on the address parameter in the issued command and the customized protocol with the controlled end.

2.4 The Standard for Selecting Publicly Available Resources

As shown in Table 2, this paper tests five major types of publicly available resources and analyzes the specific application scenarios of each when constructing C&C channel. Among them, PR-Bot stores the content of command based on online clipboard or image hosting website; stores the address of command based on a URL shortener or social network that can customized URL address; stores the stolen files based on public cloud disk website; stores the callhome information based on online clipboard that can customized URL address.

Table 2. Application scenarios of publicly available resources
Full size table
  1. (1)

    Publicly available resource for storing the content of command

In the CC channel, when selecting an online clipboard, as shown in Table 3, PR-Bot only considers two factors: one is the size of the space for storing information, and the other is the length of time for storing information. And at this stage, it does not consider whether the website supports customized URL. As long as the storage space can reach 200 KB and the storage time can reach 1 month, it means that the requirements is met. In addition, for the image hosting website, as shown in Table 4, PR-Bot only selects the websites that have no registration required and do not compress the pictures. And the size of signal picture allowed to upload should meet the requirements of 1 MB. Moreover, because the storage time supported by the image hosting website is usually unlimited, so this factor was not taken into account here.

Table 3. Online clipboard and its selection standard
Full size table
Table 4. Image hosting and its selection standard
Full size table
  1. (2)

    Publicly available resource for storing the address of command

In the CA channel, we test some services and show selection standard in Tables 5 and 6. For the URL shortener website, because it itself stores the mapping relationship between the long URL and the short URL, and the storage time is usually unlimited, PR-Bot does not have too many restrictions when selecting such publicly available resources, as long as it supports customized URL for dynamic addressing. Similarly, for social network website, in order to achieve the dynamic addressing, PR-Bot only selects the social platform that can customized homepage URL based on the user name. In addition, priority is given to websites that support temporary mailbox registration in order to avoid exposing too much real information about the botmaster.

Table 5. URL shortener and its selection standard
Full size table
Table 6. Social network and its selection standard
Full size table
  1. (3)

    Publicly available resource for storing the feedback result information

In the RF channel, for the online clipboard website, except for the storage space of up to 200 KB and the storage time up to 1 month, the website that supports customized URL is required, for making that the botmaster can find the result information fed back from bot by a certain rule (URL + numeric string). For the public cloud disk, as shown in Table 7, PR-Bot mainly considers two factors: one is the file size limit, and the other is the storage time, which is similar to the image hosting. If the website allows 10 M-sized files to be uploaded and storage time can be up to 1 month, it is enough. In addition, the public cloud disk is available without registration.

Table 7. Public cloud disk and its selection standard
Full size table

3 The Implementation of PR-Bot

3.1 CC Channel

PR-Bot mainly supports two types of commands, the “callhome” and “file stealing”, which parameters are shown in Table 8.

Table 8. The parameters of commands
Full size table

In order to prevent C&C hijacking and replay attacks, before issuing the command, the botmaster will specify the validity period of the command and sign the command based on the private key. The format of the command to be issued is as follows:

Base64 (Base64 (private key signature (original command ^ validity period))#original command ^ validity period)

Take the “callhome” as an example, its original content is a string in JSON format, which is as follows:

figure a

First, after specify validity period, private key signature, base64 encoding and string splicing, the corresponding content is as follows:

figure b

Next, if selecting the online clipboard website to store the command, it only needs to issue the above content in the form of text to the online clipboard website, which corresponding address looks like http://dpaste.com/0SV8NS5. But if selecting the image hosting website to store the command, the above content shall be converted into picture firstly and then issued, which corresponding address looks like http://h.hiphotos.baidu.com/image/pic/item/c8177f3e6709c93db7a0d055933df8dcd00054c6.jpg. In the process of storing data based on picture, PR-Bot does not embed the command into an existing picture, but directly converts it into pixels, and stores the original content in the form of pixels. The converted image style is shown in Fig. 4.

Fig. 4.
figure 4

Image style generated from text

Full size image

Normally, each pixel in a colored image is composed of three color information of RGB. Each color information occupies 8 bits, and the three colors are 24 bits, which means that each pixel can store 3 bytes of data. For a 500*500 RGB picture, it can store 75000 bytes of data, about 730 KB, which is enough to meet the space required for storing the command. Take the string “callhome” as an example: its hexadecimal representation is “0x63, 0x61, 0x6c, 0x6c, 0x68, 0x6f, 0x6d, 0x65”. First, it is divided into groups and each group consists of three units, if there are less than three, add 0 at the end. In this way, the original hexadecimal string is divided into three groups of {0x63, 0x61, 0x6c}, {0x6c, 0x68, 0x6f} and {0x6d, 0x65, 0x00}, and the corresponding RGB can produce three pixels. That is, the string “callhome” is converted into three pixel values. In addition, in the process of generating a picture, in addition to recording the content of the original data, the size of the original data also needs to be recorded to restore it normally. For PR-Bot, it uses two pixel units at the beginning of the picture, which is the six-byte space, for recording the data size. Finally, the pixel information in the picture consists of “data size (2 fixed pixels) + data content + 0 (may exist)”.

3.2 CA Channel

When issuing the address of command, the botmaster designs a UGA algorithm, which seed is based on the current date and hottest topics. This method is to prevent the address list generated by the bot from being predicted prematurely by the defender. The generated address list is as follows:

For the URL Shortener, when storing the address of command, only the URL address to be converted and the suffix of the alternative URL is needed, as shown in Fig. 5. And for the social network website, the user’s personal homepage address needs to be configured based on the alternative URL suffix, and then the address of command can be issued as a new message. Of course, all operations are automated by the program.

Fig. 5.
figure 5

How to use the URL shortener

Full size image

As shown in Fig. 6, for the bot, it undergoes two steps when obtaining the content of command, that is the “Secondary Addressing Mechanism” described above. First, the bot will traverse the generated address list based on the hard-coded UGA algorithm. When finding the address PS_Address_B, it will obtain the address PS_Address_A and further extract the content of command. The reason for adopting the “Secondary Addressing Mechanism” is to improve the flexibility and scalability of the PR-Bot. For some publicly available resources that are suitable for storing commands but not support customized URL, the jump relationship provided by the “Secondary Addressing Mechanism” can make it become “a publicly available resource that supports customized URL”. In addition, this mechanism can also improve the robustness and concealment of the C&C channel to some extent.

Fig. 6.
figure 6

Secondary addressing mechanism

Full size image

3.3 RF Channel

Callhome Module.

If all bots upload the information of controlled end to the unique URL specified in the command, there is a problem of information loss due to the limited storage space of the online clipboard website. In order to avoid this problem, this paper adopts the strategy of “URL + numeric string”.

As shown in Fig. 7, after the bot extracts the address parameter from the command, the bot will add a numeric string of the specified digits (according to the law from small to large) behind the address, and then it sends the device information to the new address. Take the address http://www.wepaste.com/abcde as an example: After the bot obtains the address parameter, it will add a numeric string from 000000 to 999999 behind it and then traverse the URLs from http://www.wepaste.com/abcde000000 to http://www.wepaste.com/abcde999999. The device information is not fed back until an address with empty content is found.

Fig. 7.
figure 7

The brief solution for result feedback

Full size image

However, if only the above brief solution is adopted, although the problem of the limited storage space can be solved, there is a problem of information coverage due to concurrent operation of bots. That is, if an address with blank content is found by two bots at the same time, they will upload device information to the address, no matter who comes first, there must be a case where one bot overwrites information uploaded by another bot. It is because the content on the online clipboard website is readable and writable.

In order to solve this new problem, this paper uses an enhanced solution, as is shown in Fig. 8. Specifically, after several minutes of uploading device information, the bot reads the uploaded information again and compares it with the locally stored information to verify whether the information is uploaded by itself. If the MD5 of the two are the same, it is considered that the device information is successfully uploaded. Otherwise, the new address is traversed sequentially and the device information is re-uploaded. In this way, even if multiple bots find an address with empty content at the same time, there will be no problem of information coverage.

Fig. 8.
figure 8

The enhancement solution for result feedback

Full size image

File Stealing Module.

Cloud disk website, also known as cloud storage website, is mainly classified into two categories: the public cloud disk and the private cloud disk. The public cloud disk can be used without registering, and the information on it is public. In the process of feeding back files, PR-Bot selects a public cloud disk to store the stolen files, which is shown in Fig. 9. First, the bot traverses the specific types of files in the controlled end and uploads them to public cloud disks one by one, and records the corresponding URL address at the same time. Then, all URL addresses are issued to the online clipboard website in the form of string. When issuing the address, it is basically the same as the flow of issuing callhome information, including the strategy of “URL + numeric string” and the mechanism of secondary verification.

Fig. 9.
figure 9

File feedback process based on public cloud disk

Full size image

Here, we will discuss the reason why PR-Bot does not choose private cloud storage to store stolen files. If PR-Bot selects the private cloud disk to store stolen files, it is necessary to specify the parameters required by the private cloud disk API interface in the command, which is used for authentication. It will cause two problems:

  1. (1)

    Once the defender has mastered the identification information, it is equivalent to obtaining the control authority of the account corresponding to the private cloud disk, so the files in the cloud disk can be viewed and deleted.

  2. (2)

    With the increase in the number of bots, the behavior of sharing the same account can easily cause abnormalities on the website, and may expose the entire botnet’s activities.

For the public cloud disk, the above two problems are inexistent. First, even if the defender obtains the command and masters the law of “URL address + numeric string” adopted by bots, it cannot locate the network location where the stolen file is located. It is because the uploaded address list is encrypted by the public key, and it can only be decrypted by the private key of the botmaster. Second, when a file is uploaded through a public cloud disk, each bot is equivalent to an independent user and has no necessary correlation, so there is no problem that multiple bots share the same account. Therefore, PR-Bot selects the public cloud disk to store stolen files, whose purpose is to ensure the security of the C&C channel.

4 The Defense Measures

Accurately finding botnets similar to PR-Bot and taking targeted measures to contain them is the ultimate goal of this paper. For the PR-Bot botnet proposed in this paper, this section describes the contents of PR-Bot defense from the aspects of detection, measurement and tracking, in order to take over the control of the botnet or reduce its availability.

4.1 Detection

In the CA channel, PR-Bot uses the UGA algorithm to generate an address pool, and the bot obtains commands by connecting the pseudo-random addresses. This process is similar to DGA. Therefore, some methods for detecting DGA also apply to UGA detection: (1) Character feature detection based on domain name [14]. There are still differences between the addresses generated by UGA and the normal addresses, such as: the use of a large number of URL Shortening services, the use of unusual user names, the use of fixed social network and etc. Therefore, the rules of distribution of domain name strings can be found by constructing the semantic rules and feature vectors, and they can be identified by the methods such as data mining and machine learning. (2) Detection based on domain name activity. In order to obtain commands, the bot will constantly address, and the addressing time will show some regularity, such as addressing at a fixed point of time even early in the morning. These features all show the non-human characteristics, so the domain name activity and spatio-temporal features can be used to detect the malicious addresses [15].

In the CC channel, PR-Bot mainly uses the online clipboard website and image hosting website. Therefore, the detection method based on communication content and network layer anomalies can be used. (1) For the commands issued on an online clipboard website, the bot will obtain the commands in the form of text, which are encoded and have the specificity, as well as identifiability. Among them, for the transmission content of the HTTP protocol, a feature matching rule may be configured in advance, such as Snort and other intrusion detection systems, to quickly and accurately discover such botnet. (2) For the commands issued on the map bed website, the bots will obtain commands from the downloaded pictures. It can also be detected through the abnormality of the transmitted information. However, the detection method based on the communication content is only applicable to botnets with specific characteristics. The disadvantage is that the unknown botnets cannot be detected, and the signature of the bot program needs to be continuously maintained and updated. The network layer anomaly detection method assumes that the communication mode between the botmaster and bots is quite different from the normal user communication, so that the trail of the botnet can be found through the flow analysis [16]. In the RF channel, PR-Bot uploads the text information according to the address specified by the botmaster, which is similar to the CC channel, so the detection method based on the communication content and the network layer anomaly may also be used.

In addition, public resource service providers should actively improve the security protection of the website to prevent normal services from being abused by attackers. PR-Bot needs to automatically register a large number of accounts and automatically issue control commands. Therefore, service providers can use the verification code-based or speed-limiting method to prevent the account from being registered in batches. Although this method will degrade legitimate users’ experience, it increases the cost of the attacker and can effectively avoid creating a potential target for attackers. Besides, the content of the account on the platform can be monitored in real time, and the release of the suspicious character string shall be further traced or handled by the security personnel.

4.2 Measurement

By measuring the PR-Bot botnet, it can portray its topological structure and corresponding scale, so that the defender can understand more about the outline and characteristics of the PR-Bot. However, due to the mechanism characteristics of PR-Bot itself, it is difficult to measure the PR-Bot, and the traditional measurement methods based on Crawler and Sybil cannot be applied. However, in the RF channel, the bot adopts the strategy of “URL + numeric string” to upload the callhome information or the address list of stolen files. The defender can find out the pattern adopted by PR-Bot through reverse analysis or flow monitoring, so that the entire scale of the botnet can be measured through the method of address traversal. Although the PR-Bot measurements are affected by various factors, such as time zone, startup/shutdown, it is difficult to accurately estimate the scale of the entire botnet, but it can estimate the number of bots as much as possible.

4.3 Tracking

If the defenders have mastered the botnet C&C channel, they can run the bot in a controlled environment or join the botnet in an infiltrated form to understand the internal activity of the botnet. In this section, we focus on how to track botnets by means of infiltration, and the infiltrating agent is called “Infiltrator”. Infiltrator can disguise as an infected controlled device to join the botnet and simulate the real communication protocol of PR-Bot to communicate with the botmaster to observe the internal activities of PR-Bot. Among them, in the RF channel, the infiltrator can intentionally submit a decoy file with tracking watermark or other payloads, so as to track the botmaster. For example: the infiltrator embeds a hidden remote picture URL in a Word document, so if the botmaster downloads and opens the file, it will actively request the URL and load the remote picture, and then the defender can trace the position of the botmaster based on the source of the request.

5 Related Work

To be well prepared for future botnet attacks, security researchers have done many works on studying advanced botnet models and defense technologies.

Sanatinia et al. [17] presented a robust, stealthy botnet that named OnionBots. The botnet use Tor privacy infrastructures for cyber-attacks by completely decoupling their operation from the infected host IP address and by carrying traffic that does not leak information about its source, destination, and nature. Ali et al. [18] presented ZombieCoin which used Bitcoin network for botnet C&C. ZombieCoin is robustness, because common takedown techniques of confiscating suspect web domains, seizing C&C servers or poisoning P2P networks, would not be effective. Yan et al. [19] proposed an anti-pollution P2P botnet called AntBot, which used a tree-like structure to propagate commands in P2P networks. The tree-like structure with the randomness and redundancy in its design, renders it possible that individual bots, when captured, reveal only limited information.

Besides, there are a number of botnet designs are based on publicly available resources. Artturi et al. [20] explores the multitude of ways in which modern malware abuses third-party web services as C&C channels, including Google Docs, Tumblr, Twitter and so on. Lee et al. [21] explore botnets based on USS, and propose alias flux methods that frequently change shortened URLs of C&C servers to hide their existence, which is similar to the domain flux method. Nagaraja et al. [12] exploit image steganography techniques to set up a communication channel within the social network, and use it as the botnet’s C&C channel. However, none of these research works have studied how to design a resilient and efficient bidirectional communication channel. Our study focuses on constructing a three-channel botnet based on multiple publicly available resources and is complementary to the existing research works to some degree.

On the defensive side, there have been many types of approaches to detect botnets, including signature-based, anomaly-based, DNS-based and data mining, machine learning techniques. For public service-based botnets, Chen et al. [22] design an unsupervised system to detect Twitter spam campaigns that use botnets to send duplicate content with embedded URLs. The unsupervised detection approach allows to build a blacklist of malicious email addresses, URLs and Twitter accounts, and to share threat intelligence with the research community in real-time. Guo et al. [23] explore the currently typical C&C server finding schemes as three types: dedicated IP address, Internet infrastructure and third-party service from a new perspective. Their work indicates that third-party service based C&C presents a better approach in terms of complexity, flexibility, traffic covertness and scale. In this paper, for PR-Bot, we propose the targeted defense strategies from the perspective of detection, measurement and tracking, so as to achieve the goal of combating against such botnets.

6 Conclusions

This paper introduces an advanced botnet based on publicly available resources, which is named PR-Bot. The PR-Bot is constructed by a three-channel scheme, which includes three sub-channels: CC channel, CA channel and RF channel. Each sub-channel can be supported by multiple publicly available resources and can be extended in the form of plug-in. Meanwhile, PR-Bot also uses the technologies, such as information hiding, content encryption and digital signature, to improve the robustness and concealment of C&C channels. In addition, in the face of new challenges, this paper proposes the defense strategies against PR-Bot in terms of detection, measurement and tracking to deal with possible similar cyber threats. We believe that it is of great practical significance to study how to construct a highly antagonistic botnet from the perspective of the attackers and propose the effective defense strategies before the attackers deploy them in practice. In the next step, we will conduct an in-depth study on this type of botnets, and design a rapid and effective detection system.