Abstract
With the continuous evolution of the Internet, as well as the development of the Internet of Things, smart terminals, cloud platforms, and social platforms, botnets showing the characteristics of platform diversification, communication concealment, and control intelligence. This survey analyzes and compares the most important efforts in the botnet detection area in recent years. It studies the mechanism characteristics of botnet architecture, life cycle, and command and control channel and provides a classification of botnet detection techniques. It focuses on the application of advanced technologies such as deep learning, complex network, swarm intelligence, moving target defense (MTD), and software-defined network (SDN) for botnet detection. From the four dimensions of service, intelligence, collaboration, and assistant, a common bot detection evaluation system (CBDES) is proposed, which defines a new global capability measurement standard. Combing with expert scores and objective weights, this survey proposes quantitative evaluation and gives a visual representation for typical detection methods. Finally, the challenges and future trends in the field of botnet detection are summarized.
1. Introduction
A botnet is an overlay network formed by many hosts (bots or zombies) infected by bots and controlled by an attacker (botmaster) for the purpose of malicious activities [1, 2]. Botmaster can control the server to control the bot and initiate various types of cyberattacks, such as distributed denial of service (DDoS), spam, phishing, click fraud, and information theft, which is one of the most serious security threats facing the Internet [3].
Given the security problems due to the continuous development of botnets, accurately identifying and detecting botnets, particularly unknown botnets in the incubation period, are the main challenging issues in academic and industrial research. Firstly, the C&C mechanism of botnets shows diversified and intelligent characteristics. Public service resources such as 5G, Internet of Things, smart terminals, cloud platforms, and social platforms have gradually emerged as fertile ground for botnets. Botnets use technologies such as zero-day vulnerabilities, P2P networks, phishing, fast flux, anonymous networks, bitcoin networks, and lightning networks as their means of utilization and spread [4–6]. Secondly, compared with conventional network security threats, botnets spread faster, have more infection channels, are more concealed, have a higher technical content, and have greater destructive power. Finally, because botnets are mostly in the silent state, they only maintain the connection state through C&C channels, without attacking and intruding, and often do not have conventional attack characteristics. Therefore, most intrusion detection systems cannot effectively identify botnets.
Deep learning theory has been rapidly developed, with significant advancements in related theoretical research and practical applications, particularly in speech recognition [7] and image recognition [8], etc. Deep learning methods can be used to solve conventional zombies. The low accuracy problem in the case of multiclassification task detection and the complexity of feature engineering in network detection technology have become research hotspots. The characteristics of blockchain technology such as decentralization, anticensorship, and concealment, as well as smart contracts, digital signatures, and incentive mechanisms, provide a new paradigm for the construction of botnets and distributed detection. The community mining algorithm in the complex network discipline provides new ideas for behavior-based botnet analysis. Swarm intelligence algorithms, SDN, MTD, and integrated methods, are some of the new methods for botnet detection.
In the field of botnet detection in recent years, there is a lack of a comprehensive overview on the latest detection technologies. This survey is divided into six major parts: first, we analyze previous surveys; second, we study botnet background and new development of botnet construction mechanism; third, we classify botnet detection technologies from a new perspective; fourth, we analyze the latest and most advanced botnet detection technologies; fifth, we propose the common bot detection evaluation system (CBDES); sixth, we summary the challenges and future trends in the field of botnet detections.
The main contributions of this article are as follows:(1)A novel summary of new developments in the construction mechanism of botnets(2)A novel classification of botnet detection technologies(3)A comprehensive analysis of the latest and most advanced botnet detection technologies, such as deep learning, complex networks, swarm intelligence, SDN, MTD, and blockchain.(4)A common bot detection evaluation system is proposed, from the four dimensions of service, intelligence, collaboration, and assistant, drawing on the ideas of analytic hierarchy process (AHP)(5)A new global capability metric € is defined, combined with expert scores and objective weights, to quantitatively evaluate eight typical detection methods and use spider diagrams to give a visual representation
The rest of the paper is organized as follows. Section 2 describes and analyzes previous botnet detection surveys. Section 3 studies botnet background and new development of botnet construction mechanism. Section 4 proposes a novel classification of botnet detection methods. Section 5 analyzes the most the latest and most advanced detection technologies. Section 6 proposes the common bot detection evaluation system. Section 7 discusses the challenges and prospects of the area, and Section 8 presents the conclusions.
2. Previous Surveys
There have been several surveys on botnet detection techniques in recent years, which are analyzed in this section.
The IoT botnet detection technologies are divided into host-based and network-based in [9]. Network-based detection is further divided into signature-based, DNS-based, traffic-based, anomaly-based, and mining-based methods. However, this review is not comprehensive enough because it targets one dimension of IoT botnet.
A detailed statistical analysis of IoT attack literature in recent years is summarized in [10]. The review outlines the existing proposed contributions, datasets utilized, network forensic methods utilized, and research focus of the primary selected studies. But it does not introduce the specific detection technology and compare and analyze the detection methods.
DNS-based botnet detection technologies are classified into five categories in [11]: flow-based, anomaly-based, flux-based, DGA-based, and bot infection-based. Essential attributes of a smart DNS-based botnet detection system are proposed. But the survey did not provide context for the botnet’s construction mechanism.
A comprehensive botnet detection is analyzed in [12]. This survey classifies botnet detection techniques into four classes: signature-based, anomaly-based, DNS-based, and mining-based. Unfortunately, the summary is too simple and does not cover the introduction of the latest technology.
For botnet detection technologies based on DNS traffic analysis, the technologies are classified into two categories in [13]: honeypot-based and IDS-based. It mainly introduces passive technologies, including graph theory, statistical analysis, clustering, decision tree, and neural network. This literature is comprehensive, but it is old and has not been evaluated.
Evasion and detection techniques of DNS-based botnets are focused on [14]. This survey introduces Fast-Flux and DGA botnet detection technology. Also, the dimensions are relatively single and there is no evaluation.
The detection is divided into four categories in [15]: honeypot analysis, communication signature, anomaly, and log. The literature introduction content is relatively few, not comprehensive enough.
Each survey emphasized different aspects of the papers. The analysis of the surveys shows some limitations: The surveys use different taxonomies and terminologies A lot of surveys focus on one type or function, such as DNS and IoT, with a single dimension and lack of comprehensive analysis of new construction mechanism of botnets Most surveys do not cover the most advanced method, and there is a lack of systematic introduction to the latest technologies Most of the data lack a comprehensive evaluation of the detection methods
A comparison of our survey with other surveys is presented in Table 1. Our survey aims at knowing and understanding botnet detection and eliminates these limitations.
3. Background
Based on an in-depth understanding of the working mechanism and behavior characteristics of botnets, this section introduces the latest development in botnet construction mechanisms in terms of the botnet architecture, life cycle, and C&C channel, as shown in Figure 1.
3.1. Architecture
The botnet C&C system architecture is mainly divided into the following three categories: centralized, distributed, and hybrid.(1)Centralized. The centralized botnet architecture adopts a client-server model generally. The bot mainly obtains control commands from the control server in a polling manner, and the botmaster sends the control commands and resources to the zombie host through these servers. Centralized botnets have advantages such as simple implementation, high efficiency, and good coordination, but their control process is associated with a central node failure.(2)Distributed. To improve the robustness of botnet, an attacker can use a decentralized structure of the P2P (peer to peer) mode as its channel architecture. Any node can act as a client and a server simultaneously, and the communication process does not rely on public network reachable server resources. Although the P2P botnet command issuance delay is higher than that of a centralized structure, the distributed structure is difficult to be hijacked, measured, and closed.(3)Hybrid. A hybrid architecture typically means that botnets have both central and P2P structure. This can be divided into two categories: an overall central structure and a partial P2P structure. From an overall perspective, it still belongs to the centralized structure of the C/S model; however, a P2P structure is present between the service nodes. One is the overall P2P structure and the other is a local center structure. This type of network structure is conducive to the realization of regional differentiation management and control, and it is difficult for defenders to detect all the key nodes and the overall scale of the botnet.
3.2. Life Cycle
The life cycle of a botnet mainly includes the stages of propagation, rally, interaction, and malicious activities. Propagation. As an independent runnable program, the spread of bots includes the use of conventional malicious code. The main propagation methods include shared media spread, vulnerability exploitation spread, social engineering spread, and password guessing. Rally. Rally refers to the behavior of bots in locating and controlling the server and its resources. Implementation methods are mainly divided into two categories: static and dynamic. Static addressing means that the C&C resources that bots try to access are static and unchanging. These resources are typically hard-coded in the body of the bot or stored in a hidden path of the infected machine, such as the registry. Dynamic addressing means that the access address is not fixed but needs to be dynamically generated based on the specific algorithm used. Interaction. When the zombie host successfully discovers the available command control server or resource, it will establish a connection with the controller and begin to interact. This process is also called the command control phase, which mainly includes four activities: registration, file download, order distribution, and result feedback. Malicious Attack. The main purpose of an attacker to build and control a botnet is to control many victim hosts to launch a variety of attacks. Common attack activities include DDoS, spam, spreading malware, information leakage, click fraud, phishing attacks, information collection, virtual currency mining, and encrypted blackmail [16].
A product development model was used to define the life cycle in [17], including concepts, recruitment, interaction, motivation, and attack execution (CRIME). Literature [18] proposed a fine-grained, hidden Markov model-based botnet life cycle model, describing the state transition of botnets from propagation to extinction and dividing the typical botnet life cycle into nine types of hidden states: infection/initialization/idle/propagation/attack/maintenance/offline/isolation/dead. The model used “state” instead of “stage” to describe the evolution of botnets and broke the conventional irreversible and abstract timing relationship. The model could better represent the migration and changes of botnets.
3.3. Command and Control Channel
3.3.1. Classical C&C Channel
The core of the botnet is communication, and the classical C&C channel is mainly implemented through IRC, HTTP, SMB, P2P, or other custom protocols [2]. Using IRC service as a centralized C&C channel is easy to implement and has low latency and good real-time performance; however, the centralized topology can easily be detected and blocked [19]. Bots that use the HTTP protocol to construct C&C channels can periodically access the botnet controller, obtain command files, parse, and perform corresponding operations, and can penetrate IDS and firewalls, with good versatility and concealment [20]. The Server Message Block (SMB) protocol is a protocol that hides communication under the typical traffic patterns in home and enterprise networks and is mainly used for communication in local area networks [21, 22]. The P2P protocol is used to construct distributed botnet control channels, which solves the single-point failure problem of botnet controllers, and has good robustness, stealthiest, and self-organization capabilities. The disadvantage is that it is vulnerable to index poisoning and Sybil attacks and initial vulnerability. Botnets that use custom protocols to communicate are stealthier, and the communication process is less likely to be detected.
3.3.2. New C&C Channel
(1) Diversified Platforms. The decentralization and concealment of public service resources represented by cloud platforms, social networking sites, and blockchains present natural advantages and have become fertile ground for botnets. Cloud Botnet. The multitenant feature of cloud computing can provide computing resources to anyone. Botnet controllers pretend to be legitimate tenants of cloud services and use virtual machines of cloud service providers to quickly construct botnets and use them to launch attacks. At the RSA2014 conference, Ragan introduced a cloud botnet construction method [23], which can realize the mining of electronic money by controlling massive cloud computing resources. IoT Botnet. The Internet of Things (IoT) has been implemented in various fields such as agriculture, healthcare, food supply management, drug supply management, environmental monitoring, and smart homes. IoT has heterogeneous environments and resource-constrained devices, i.e., low memory, low computing power, and low original security performance, which increases the risk of infection. Mirai is a common IoT botnet, the main objective of which is to perform DDOS attacks, with a strong scale and attack capability [24–26]. Social Botnet. Botnets use social media sites such as Facebook, Twitter, or WeChat to build transmission channels or spread messages in social networks. The Flashback botnet [27] uses Twitter to construct a backup C&C channel. Once the main channel fails, the bot will search for the C&C domain name by searching for a specific identifier that is dynamically generated to restore communication with the controller. A botnet that is parasitic on social networking sites can imitate normal users to complete a variety of online social actions [28]. Attackers can control social bots to achieve rumor dissemination, advertisement push, and personal information collection [29]. Mobile Botnet. The portability of mobile devices and the increasing popularity of applications also have an impact on the threat pattern of botnets. Geinimi can steal IMEI [30], geographic location, SMS, address book, and other information and send spam SMS and install malware; in 2012, Dexter, the first POS machine botnet, uses a memory reading technology to steal users’ payments card data; Zhao et al. [31] proposed a mobile botnet network based on Google Cloud to Device Message (C2DM). Based on Blockchain. The literature [32] uses bitcoin blockchain floating C&C servers to propose a new type of resilient botnet. The literature [33] uses the bitcoin transaction propagation mechanism as the C&C infrastructure and proposes the use of subliminal channels [34, 35] to create a concealed method to repeatedly create signatures on transactions. Fbot botnet, one of the Mirai variants, uses Emercoin [13] domain name system based on a distributed blockchain to solve a key problem that conventional DGA-based botnets are easily detected by reverse engineering. The literature [36] proposes a new generation of hybrid two-layer botnet LNBot, which uses lightning network (LN) infrastructure to communicate between bots and C&C servers. The off-chain concept [37, 38] realizes almost instant bitcoin transactions.
(2) Covert Communication. The covert communication technology mainly includes information hiding and C&C channel hiding. Information Hiding. This method modulates the secret information into the protocol redundancy field through various modulation methods, mainly including encryption, compression, and obfuscation, and steganography. Nagaraja et al. [39] proposed a botnet to use existing social networks as C&C channels and hide communications in JPEG pictures. Cui et al. [40] proposed a three-channel botnet model. The core idea was to use Domain-Flux, URL-Flux, and Cloud-Flux as subchannel protocol for the entire C&C based on registration, command issuance, and data return functions. The C&C Channel Hiding. The covert channel technology based on DNS protocol is one of the mainstream network covert channel realization methods. The common methods are Domain-Flux and Fast-Flux [41]. Casenove et al. [42] introduced scalable and stealthy botnets based on anonymous networks and could observe the C&C traffic at the Internet Service Provider (ISP) level.
(3) Intelligent Control. The concept of complex systems, with their self-organization, resilience, and adaptability, can greatly help botnet communication protocol and architecture design. A heuristic algorithm based on ant colony optimization was proposed to construct a botnet C&C [43]. This could ensure spontaneous and intelligent collaboration between independent bot agents, which improves network fault tolerance and the ability to dynamically adapt to the network environment.
4. Classification
Conventional detection methods are no longer suitable for new botnet detection. The industry has a more in-depth understanding of the working mechanism and behavior characteristics of botnets, and various botnet detection methods have been proposed. This section divides the key technologies of botnet detection into three categories based on honeypot analysis, communication signatures, and abnormal behavior. We focus on the application of deep learning, complex networks, swarm intelligence, MTD, SDN, blockchain, and other cutting-edge technologies in botnet detection.
Botnet detection technology classification standards are different, and there are multidimensional classification methods. This article classifies the key technologies as shown in Figure 2.
5. Methods
5.1. Based on Honeypot Analysis
Based on the honeypot analysis and detection method, many malicious code samples can be obtained through honeypot trapping, i.e., the botnet binary files of the existing botnet, and the monitoring and analysis can be performed in a controlled environment, and the bots and their malicious behaviors can be discovered [44]. It is an active detection behavior. Representative honeypots include Snort, Ntop, Argos, Nepenthes, Sebek, and the Goddess of Hunting project led by Peking University Zhuge Jianwei [45]. The darknet is quickly becoming a popular alternative to using honeypots and is essentially a derivative of the honeynet.
Although the honeypot-based method has a high accuracy rate for known botnets, it cannot effectively identify encrypted traffic and detect unknown attacks. Moreover, it cannot easily find botnets that spread through social engineering and is useless for real-time systems. Because of the lack of user operations, it can be easily recognized by bots with an anti-honeypot function.
5.2. Based on Communication Signature
The method based on communication signature detection is a commonly used defense method, which detects bot activities based on predefined patterns and signatures retrieved from well-known bots [46]. Common methods include regular expressions [47], whitelists (or blacklists) [48], and N-gram models [49]. By configuring feature matching rules in advance, conventional intrusion detection systems such as Snort have a rich signature database, which can help quickly and accurately discover botnet activities.
The communication signature-based method is suitable for botnets with definite features, which helps further understand the communication mechanism and potential vulnerabilities of botnets. The disadvantage is that robots can avoid signature-based detection by using code obfuscation technology and cannot detect botnets with unknown features. The method needs to maintain and update the signature knowledge base continuously, which increases the cost of detection.
5.3. Based on Abnormal Behavior
Anomaly-based detection is an important research field in botnet detection. The basic idea is based on host behavior or network traffic abnormalities, such as the high network latency, large amounts of traffic, traffic on abnormal ports, and abnormal system behaviors and based on established systems. The deviation in the benign behavior or the similarity with the behavior of bots can be detected.
5.3.1. Deep Learning
In the past few decades, researchers have used various conventional machine learning methods to detect botnets [50–52] and have made great progress, such as Naive Bayes [53], support vector machines [54], random forests [55], and clustering algorithm (such as DBSCAN [56] and X-means [57]), based on a variety of characteristics to establish a model that can identify malicious network traffic. The characteristics are typically set by researchers through the experience before the model is established. Common dimensions include network flow properties, such as the number of data packets, the average byte of data packets, and the average interval between two adjacent data streams, and behavior, such as whether to access the same server. These detection models were found to have a low false-negative rate and false-positive rate in an experiment. However, there are some shortcomings. First, manual selection has higher requirements on the prior knowledge of the designer. The second is that fixed features also provide an opportunity for attackers. Attackers can use anti-machine learning ideas to change the characteristics of botnet traffic in a targeted manner, thereby evading model detection. The botnet shape and command control mechanism are gradually changing, and artificial feature selection is becoming increasingly difficult. With the rapid development of deep learning technologies, neural networks, reinforcement learning, knowledge graphs, and other methods are gradually being applied to the field of botnet detection, which represents new approaches.
(1) Neural Network. The basic idea is to extract network traffic features based on temporal and spatial similarities. This method involves mapping network traffic into a grayscale image or feature vector and sending it to a neural network model, extracting distinguishable features and patterns from the space and time dimensions, and automatically learning network traffic features.
CNN. The convolutional neural network (CNN) mainly learns spatial features from the spatial dimension through network traffic. Aiming at BotCloud, Guang et al. [58] first extracted basic features from the network stream and mapped them onto grayscale images. Subsequently, the CNN algorithm LeNet-5 was used for feature learning, and more abstract features were extracted to express the hidden patterns and structural relationships in the network stream data; the algorithm was finally applied to detect BotCloud. There is no significant difference in traffic in the early stages of IoT botnet. Most botnet detection systems are not suitable for resource-constrained IoT devices. The literature [59] used side-channel power consumption information, such as power consumption, etc., to distinguish whether IoT devices are affected by malicious behaviors and proposed a CNN-based deep learning model to perceive the subtle differences in power consumption data. The literature [60] proposed an extensible framework that uses LSTM to collect DNS traffic data at the ISP level and detect DGA-based malware in real time. Deep learning ImageNet model was used to classify domain names generated by DGA [61].
RNN. Recurrent neural network (RNN) mainly learns the characteristics of network traffic in time series from the time dimension. The literature [62] applied RNN to detect botnets by modeling network communication behavior as a sequence of time-varying states. The behavior model of each flow is established based on four parameters: source and destination IP addresses, destination port, and protocol. The literature [63] proposed a solution to detect botnet activities in consumer IoT devices and networks, four attack vectors of Mirai were used as eigenvectors, and a detection model was established based on the RNN and bidirectional long- and short-term memory (BiLSTM-RNN). The literature [64] proposed an anomaly detection system for 5G adaptive real-time deep learning system, which includes two modules: abnormal symptom detection (ASD) and network anomaly detection (NAD). DBN (Dynamic Bayes) was used to realize the ASD time measurement process, and the LSTM network model was used to realize NAD. The literature [65] proposed a malicious domain name detection method based on knowledge graph. For DNS traffic, TransE took the embedded model of the system as input and completed the storage and representation of information in the knowledge graph, which not only included the embedding of entities and relationships, but also the embedding of attribute values. The advantage of combining the BiLSTM neural network to extract features for detection was that it can learn the context relationship of vectors in the sequence and better extract features for classification.
CNN + RNN. Reference [66] proposed a deep learning-based botnet detection system Bot Catcher, which extracted network traffic characteristics from the two dimensions of time and space automatically. Spatial feature learning was based on the application CNN LeNet-5 structure, which was used in the field of image recognition, and each stream data was converted into a 2D gray image. The data stream of 1024B (32 × 32) data was intercepted before each data stream. Typically, the data in front of a data stream mainly includes connection information (such as the three-way handshake in TCP connection and key exchange in TLS connection), and less part of the content exchange could better reflect the main characteristics of the entire data stream. To mine deeper into the characteristics of the data stream in the time series, Bot Catcher used the BiLSTM neural network in the RNN to learn the time characteristics and scanned each data stream in both forward and reverse directions. In [67], for Fast-Flux botnets, combined with convolutional neural network (DenseNet) and recurrent neural network (BiLSTM), the DNS data response packet in the network traffic based on analysis, a fast-flux botnet detection method based on the temporal and spatial characteristics of traffic was proposed.
GAN. The literature [68] proposed a botnet detection framework based on generative confrontation networks (Bot-GAN), which was different from other variants of generative confrontation networks. The framework focused more on discriminative models instead of generative models.
DNN. The literature [69] proposed a DGA domain name detection method that does not require extracting specific features, based on word-hashing technology to map strings to a high-dimensional space, and used deep neural network DNN to classify domain names. The literature [70] proposed a two-level deep learning framework for real-time detection of botnets. In the first-level framework, the Siamese network is used to estimate the similarity measure of DNS queries. In the second level, a domain generation algorithm (DGA) based on deep learning architecture is proposed to classify normal and abnormal domain names. The literature [71] constructed a deep learning framework based on dual-stream network (TS-ASRCaps), used multimodal information to reflect the characteristics of DGAs, and proposed an attention sliced recurrent neural network (ATTSRNN) to automatically mine the underlying semantics. A capsule network with dynamic routing (CapsNet) was used to model high-level visual information.
FNN. The literature [72] proposed a real-time online Fast-Flux botnet filtering system, aiming to improve the detection of unknown “zero-day” online fast-flux botnets. Fast-Flux botnet domain was distinguished from the legal domain in an online mode based on new rules, characteristics, or classes. Using the adaptive evolutionary fuzzy neural network algorithm, the first stage preprocessing includes feature analysis and stemming, and the A-Import feature represents the Fast-Flux botnet, and the second stage uses an evolutionary fuzzy neural network (EFUNN) algorithm to establish FF Hunter (FFH) system.
(2) Reinforcement Learning. Reinforcement learning is an algorithmic method used to solve sequential decision problems, in which agents (or decision makers) interact with the environment to learn to respond under different conditions. Reinforcement learning is used for botnet detection in three ways: firstly, combined with NN neural network for new feature extraction, the agent learns a strategy to maximize the total number of bots detected over time. Secondly, it is used for the deployment of distributed detectors to make intelligent decisions. The agent estimates the system status and operation rewards by monitoring the network activities in different network segments. Thirdly, deep reinforcement learning is used for evading machine learning detection.
The literature [73] proposed a method that combines reinforcement learning technology to detect botnets as early as possible in the propagation stage or before any malicious activities are initiated by the bot. It included four stages: network traffic capture and packet filtering, feature extraction, malicious activity detection, and bot behavior detection using reinforcement learning. The network traffic feature extraction was carried out in three levels: data packet level, data flow level, and connection level. Malicious activity detection included three stages: offline stage (training), online detection stage, and reinforcement learning stage. The literature [74] proposed an online clustering-enhanced botnet detection method using reinforcement learning PRCL, which could detect botnets in real time with high accuracy. Research on adversarial machine learning has shown that botnet attackers can bypass the detection model by constructing specific samples, and many algorithms are susceptible to less input disturbances. The literature [75] proposed a new anti-botnet traffic generator framework based on deep reinforcement learning (DRL), which could effectively generate reverse traffic flow through the RL algorithm and Markov decision process (MDP). The agent could add disturbances to the flow and changed the spatial and temporal properties of the network traffic and automatically added disturbances to the samples to try to fool the target detector. This research could help inspectors to find defects and improve the robustness of the system.
The summary of typical botnet detection techniques based on deep learning is shown in Table 2.
5.3.2. Complex Network
Botnet communication is associated with both similarity and stability. The relatively frequent communication activities based on the heartbeat mechanism will form a correlation graph. For abnormal behaviors, complex network methods are used to conduct community mining to detect botnets. The methods used can be typically divided into two categories: graph methods and community mining algorithms.
(1) Graph Method. There are two main ideas for graph-based methods. One is for the behavior of executable files, such as control flow graphs, call graphs, and code graphs, to model graphs. The other is based on the behavior of nodes in network traffic, e.g., the IP-domain name mapping relationship is modeled on the graph, and then classified and detected on this basis.
The literature [83] proposed a new high-order subgraph feature based on PSI (printable characters) extracted from malicious code to detect large-scale botnets. These features had precise behavior descriptions and less space requirements. Aiming to Large Scale Spamming, BotGraph [84] revealed the correlation between botnet activities by constructing a large user graph, including two components: detector registration and behavior connection. The first component ensures that the total number of bots was limited, and the second component was based on constructing a user-user random undirected graph to detect invisible robot users, and then it detected bots through abnormal behavior [85]. Based on the topological features of the nodes in the graph, a novel botnet detection method was proposed, which extracts in-degree, out-degree, weight, degree weight, clustering coefficient, internodes and feature vector centrality, and based on these features, a self-organizing map clustering method SOM was used to establish the clustering of nodes in the network. This method can isolate bots in small clusters. In literature [86], for high-speed networks, it correlates NetFlow-related data and uses host-dependent models for advanced data mining, extends the popular link analysis algorithm PageRank [87] for cluster processing, and uses P2P communication infrastructure to effectively detect invisible botnets without a significant amount of traffic. Aiming at the anonymity of botnets, based on the DNS query response, a mapping relationship between its domain name and IP was extracted through the DNSmap tool to construct a DNS association map in [88] named XIONG. Moreover, the authors analyzed the structural characteristics, FQDN (full domain name was called fully qualified domain name) node and IP node characteristics, and connection edge characteristics of the graph and integrated the blacklist statistical characteristics to realize the multifeature analysis of the graph components and selected the light GBM algorithm to complete diagram component classification. It also proposed a prototype system for Fast-Flux and Domain-Flux botnet detection under high-speed networks. The architecture was analyzed from the vertical perspective of data flow transmission, which was divided into data access layer, data storage layer, processing unit layer, and user interface layer.
(2) Community Mining. The literature [89] considered three types of community behavior: traffic statistics characteristics, digital community characteristics, and structural community characteristics, and proposed an early method based on community behavior analysis, PeerHunter, which uses complex networks for community detection. Louvain method algorithm could detect botnets communicating through the P2P structure. In [90], advanced features were extracted from network traffic to detect P2P botnets in real time. By jointly considering flow-level traffic statistics and network connection patterns, a dynamic group behavior analysis (DGBA) was applied to distinguish P2P bot-infected hosts from legitimate P2P hosts, and a new dynamic group behavior analysis was conducted to extract the collective and dynamic join patterns for each group. Wang and Paschalidis [91] proposed a new two-stage method for detecting the existence of botnets and identifying damaged nodes. The first stage detected anomalies by using large deviations in the empirical distribution. Two methods for creating empirical distributions were proposed: the flow-based method to estimate the histogram of the quantized flow and the graph-based method to estimate the degree distribution of the node interaction graph, including the ER graph and the unscaled graph. The second stage used the idea of social network community detection in the network to detect robots. This graph captured the relationship between the interactions between nodes over time and conducted community detection by maximizing the modularity metric in this graph.
The summary of typical botnet detection techniques based on complex networks is shown in Table 3.
5.3.3. Swarm Intelligence
The swarm intelligence optimization algorithm mainly simulates the group behavior of insects, herds, birds, or fish, which search for food in a cooperative manner. Each member of the group constantly changes the search direction through learning experience. The main idea of this type of botnet detection method is to use heuristic biological behavior to search and find abnormal points, perform feature extraction, and then combine with classifiers for detection.
(1) PSO. The literature [95] proposed a botnet detection method (BD-PSO-V), which was a hybrid particle swarm algorithm and voting system. The PSO algorithm was used for feature selection of network stream data. The feature was considered as particles, and the birds found the best particles. The voting system, including a deep neural network algorithm, support vector machine (SVM), and decision tree C4.5, based on the maximum number of votes, was utilized to identify botnets and classify samples. Six well-known adversarial attacks, including Fast Gradient Sign Method (FGSM), were evaluated on the ISOT and Bot-IoT datasets. The literature [96] proposed a detection model based on multiobjective particle swarm optimization (MOPSO) to identify malicious behaviors in bulk network traffic. In [97], a smart adaptive particle swarm optimization support vector machine (SAPSO-SVM) algorithm was proposed for Android botnet detection application. The algorithm used the changes in each stage of the execution process of the personal best and the global best to specify a new evolution factor value and then eliminated the interference of the inertial weight interval.
(2) GWO. The literature [98] proposed a new unsupervised evolutionary IoT botnet detection method. By using the latest Grey Wolf Optimization (GWO) swarm intelligence algorithm to optimize the hyperparameters of one-class support vector machine (OCSVM), it detected botnet attacks launched from compromised IoT devices.
The summary of typical botnet detection techniques based on swarm intelligence is shown in Table 4.
The summary of typical botnet detection techniques based on statistical analysis is shown in Table 5.
5.3.4. Statistical Analysis
The statistical method is mainly based on the data modeling of its statistical attributes to find outliers and estimate whether the test sample is a bot. The literature [102] proposed a spatial snapshot rapid flux detection system (SSFD), which relied on spatial distribution estimation and spatial service relationship evaluation. The space of distinction and information entropy were combined to measure the equivalent distribution of the nodes in each time zone. The benign areas tended to be distributed in the same time zone, whereas the fast-flux nodes were widely distributed in multiple time zones. [103] Aiming at the Internet of Things botnet DGA, a lightweight system was proposed to detect IoT-based botnets through the flow of the rapid recognition algorithm generation domain (AGDS). Threshold random walk (TRW) was used to quickly classify NXDOMAIN (a large set of random nonexistent domain names) query flows to create opportunities to interrupt C&C connections.
Botnets can hide periodic behavior characteristics by changing the communication interval. When the interval is too large, a time-series analysis cannot detect a periodic communication behavior. Based on the periodic communication detection method along with sequential hypothesis testing, Wang et al. [104] proposed a botnet periodic communication behavior detection algorithm and introduced a fast quantum search algorithm called the Grover quantum state to better realize parallel processing and improve the algorithm speed. This method can complete botnet detection with less query time and improved detection speed.
5.3.5. Distributed Approach
To increase the detection accuracy and improve the flexibility of the detection system, some literature studies have designed the distributed detector to collect massive and multidimensional data for detection.
(1) MTD. The literature [105] proposed a new type of botnet defense mechanism based on a combination of honeypot and network-based strategies, MTD (moving target defense), and reinforcement learning technology. The MTD method is used to periodically change the position of the detector, constantly reshaping the attack surface of the system and increasing the complexity and cost of the attacker. Using reinforcement learning to optimize and dynamically deploy detectors in an iterative manner, the agent learns a strategy to maximize the detection and removal of bots over time.
(2) SDN. SDN technology realizes the separation of control plane and data planes, and the visibility and programmability provided are typically used to implement various security detection and attack mitigation schemes. The basic idea of the SDN-based botnet detection mechanism is based on Open vSwitch, a virtual switch that implements the OpenFlow protocol, combined with classifiers to detect bots and identify malicious traffic. OFX [106] proposed by Sonchack et al. can deploy security functions in the existing OpenFlow infrastructure, allowing control applications to dynamically load security modules directly into unmodified SDN-compatible switches. Zha et al. [107] proposed an SDN-based scalable, accurate, and online data center bot detection framework BotSifter, which distributed detection tasks across the edge of the network in Open vSwitch, and the use of centralized learning (DNN) and distributed detection enhances the robustness of detection. The literature [108] proposed a lightweight real-time botnet detection scheme BotGuard under SDN, using the idea of graph matching algorithm, and proposed a convex lens imaging model graph to describe the topology feature of the botnet. It allowed the SDN controller to independently locate the attack location while reducing the network load. The Mininet platform was used for simulation evaluation.
(3) Blockchain. The basic idea of using blockchain technology for botnet detection is to the use of smart contracts, digital signatures, incentive mechanisms, and other technologies, based on proxy or collaborative detection, to achieve trust information exchange or voting among different detectors. In [109], AutoBotCatcher used BFT (Byzantine Fault Tolerant) to perform dynamic and collaborative botnet detection on large networks and used the community detection algorithm Louvain method to detect communities. The literature [110] proposed a blockchain trust model (BTM) for malicious node detection in wireless sensor networks, which used blockchain smart contracts and WSN quadrilateral measurement and positioning methods to achieve malicious node detection in the 3D space, with good traceability. Based on a consensus mechanism blockchain, Spathoulas et al. [111] used lightweight agents installed at multiple IoT locations to collaboratively detect DDoS attacks carried out by a botnet of IoT devices. The literature [112] proposed an incentive platform SmartRetro driven by blockchain smart contracts and PoW consensus schemes, which could incentivize and attract more distributed detectors to participate in traceable vulnerability detection and contribute their detection results.
The summary of typical botnet detection techniques based on distributed approach is shown in Table 6.
5.3.6. Combination Method
The evolution of botnets presents characteristics such as diversified platforms, concealed communications, and intelligent control. A single abnormal behavior detection method cannot meet the actual requirements. Multidimensional, multiagent, and multitechnology combined detection methods have therefore emerged.
Multidimensional refers to the combination of multiple detection objects, mainly referring to the combination of network traffic and signature detection. The literature [113] proposed a hybrid botnet detection method HANABot based on a host-side and network analysis; this is a general technology that can detect new botnets in the early stage. The system contains three components: network analysis component, host analysis component, and a test report. The document [114] proposed an effective two-stage traffic classification method based on a non-P2P traffic filtering mechanism and session feature-based machine learning technology to detect P2P botnet traffic. In the first stage, non-p2p packets are filtered, and network traffic was reduced through well-known ports, DNS queries, and flow counting. In the second stage, the session features were extracted based on the data stream characteristics and stream similarity, and the P2P botnet was successfully detected by the machine learning classifier. The literature [115] proposed a signature generation method based on the similarity of HTTP botnet header information, which could automatically generate high-quality network signatures. This method combined the advantages of network traffic and data packet detection, and TCP flow was used as the object to extract the size statistical characteristics of the HTTP first request packet and the first response packet (referred to as “a question, one answer packet”) and combine the HTTP header field content. Statistical analysis could detect “silent” state bots. The literature [116] proposed a multistage detection method for domain fluxing, fast-flux service network (FFSN), and domain generation algorithm (DGA). The first stage used NX domain and server failure errors to detect DNS tunnel C&C server calls. In the second stage, a signature matching technology was used to detect the DNS tunnel SSH handshake between BOTS and C&C server.
Multiagent (Agent) refers to the combination of multiple agents [117]. A multiagent Robot Detection System (MABDS) [118] was a hybrid technology that associates an event log analyzer with a host-based intrusion detection system (HIDS). It used multiagent technology to combine management agent, user agent, honeypot agent, system analysis, and knowledge database.
Multitechnology refers to the combination of multiple technologies or algorithms. The literature [119] used the characteristics of a graph and a neural network for detection. They generated network communication graphs at regular intervals by modeling the graph features over time, extracted the graph-based statistics and central features, assembled a time series of the features for each host (identified by IP address), and trained the time-series classification model. Ten graph features were extracted for each node: out-degree, in-degree, adjacency, neighbor, PageRank centrality, intermediate centrality, feature vector centrality, authority, and hub centrality, using the graph tool library [120] local clustering coefficient. Using the time-series data in the network is suitable for a real-time detection.
The summary of typical botnet detection techniques based on combination method is shown in Table 7.
We mainly focus on the comparison of botnet detection techniques based on abnormal behavior. The basic ideas, advantages, and disadvantages of various methods are summarized in Table 8.
6. Evaluation
The classification of botnet detection systems has more dimensions. This section draws on AHP, from the four dimensions of , , , and , a general botnet detection system performance evaluation system CBDES (common bot detection evaluation System) is designed. By constructing the judgment matrix and checking the consistency, the performance of the subsystems is independent of each other, and the weight of each index is calculated. Combining expert scores and objective weights, a new global capability metric € is defined, and eight typical detection methods are quantified and evaluated. Finally, a visual representation is given using spider graphs.
6.1. Index System
The performance index system of the botnet detection system is mainly divided into four dimensions: , , , and , as shown in Figure 3.
6.2. Evaluation Index System
(1) refers to the basic performance of the botnet detection system. The indicators are divided into three subindices: accuracy, scenes, and stage. The accuracy refers to the accuracy of botnet detection. The scenes refers to which scenes the method is suitable for. Stage refers to which stage of the botnet’s life cycle is detected. In the dimension, the weight is used to give the importance of each indicator, the quantitative formula of this dimension is obtained, and the result is normalized to [0,1]:(2) refers to the degree of automation of the detection system, which is divided into three subindexes: automation, adaptability, and real time. Automated refers to the degree of automation of feature extraction in the detection process. Adaptive refers to whether the detection model can detect unknown types of botnets. Real-time refers to whether the detection system can be performed in real time. In the dimension, the weight is used to assign the importance of each indicator to get the quantitative formula of this dimension, and the result is normalized to [0,1]:(3) refers to the system’s synergy and scalability and is divided into subindices such as architecture, content, and integration. refers to the organizational structure of the detection system, which is divided into centralized and distributed. Content refers to the category of detection content, which is divided into single type, such as host log or network traffic. Diversity refers to the detection of host and network combination, or code and traffic combined with multiple data. Integration refers to the use of multiple types of detection methods. In the dimension, the weight is used to give the importance of each indicator, the quantitative formula of this dimension is obtained, and the result is normalized to [0,1]:(4) refers to some other indicators, mainly latent, cost, and visualization. Latent refers to whether it can detect deep latent botnets. Cost refers to the consumption cost of the detector, such as GPU and bandwidth consumption. refers to the visualization of data information or botnet detection through visualization methods. In the dimension, the weight is used to assign the importance of each indicator to get the quantitative formula of this dimension, and the result is normalized to [0,1]: According to the calculated value of the four dimensions, it can be abstracted as a polygonal area. This paper uses Gauss’s area formula to define a global measure. A new global competency metric € is defined with expert ratings and objective weights as follows:
6.3. Quantitative Assessment
(1)AHP (analytic hierarchy process) is used to determine various weight indicators Step 1: construct a judgment matrix. According to the AHP hierarchical structure, the judgment matrix of the criterion layer to the target layer is constructed from top to bottom. Generally, the factors of the lower layer are used to evaluate the factors of the upper layer. The judgment matrix is composed of the results of the comparison of the factors of the lower layer. As shown in Table 9, the scale is 1–9. First construct the judgment matrix of criterion layer and index layer: Taking the criterion layer as an example, the calculation method of weight value is introduced in detail. Using the L1 paradigm, the W elements are normalized by column to get Step 2: calculate the weight of each performance index. This paper uses the maximum eigenvalue max corresponding to the components of the standard eigenvector as the weight of each factor. = 4.245, The feature vector is . Step 3: the consistency correction of the judgment matrix. In addition to human factors, the consistency of the judgment matrix is different in the acceptance range of the consistency of the judgment matrix of different orders. By using the consistency index, revises to achieve its goal. The value is shown in Table 10. it passed the inspection. Therefore, the weight vector of the criterion layer is . Step 4: use the same method to calculate the weight of the indicator layer, and then multiply the weight of the above layer to get the value of all weights, as shown in Table 11.(2)Description of various indicators of botnet detection system is shown in Table 12.(3)Evaluation of typical botnet detection system According to the botnet detection methods introduced in Section 3, eight typical detection methods are evaluated. Step 1: expert scores are as shown in Table 13, where EV represents evaluation vector. Step 2: calculate the four-dimensional indicators according to the weight values obtained by the analytic hierarchy process and the calculation of the global metrics. Table 14 is obtained and sorted. According to the quantitative evaluation proposed in this article, the top four methods with good detection performance are PRCL, BotSifter, PeerHunter, and Bot Catcher. Step 3: use spider diagrams for visual representation of the top four as shown in Figure 4.
7. Challenges and Prospects
7.1. Challenges
In the process of attacking security organizations and overcoming government supervision, botnets are constantly evolving. To solve problems, such as concealment, survivability, and survivability, new terminal botnets, such as IoT and smart mobile devices, have also become the main source of various types of Internet security threats. Nevertheless, the industry has a more in-depth understanding of the working mechanism and behavior characteristics of botnets, and a variety of botnet detection methods have been proposed. This article summarizes the challenges faced by detection methods:(1)Multisource information collection and fusion: because of the concealment and cross-platform nature of botnets, its traces are often hidden in various information scattered at different dimensions, such as personal hosts, regional networks, and backbone networks, and stored in different formats. Multisource information contains various redundant information. Data collection must have the characteristics of collaboration, distribution, and intelligence. Any combined method should be highly accurate and have a low complexity, provide unified data representation and storage, and then perform data processing. It should also dynamically adjust the collection based on the actual scene strategy.(2)Deep latent command and control channel: botnet detection is typically in the communication and attack stage of its life cycle. In the communication stage, the focus is on traffic data. The centralized structure of botnets shows strong similarity and correlation characteristics, and the detection effect is evident. However, for third-party channels, such as P2P, cloud platform, and blockchain technology, there is a lack of effective detection methods. Detecting deep latent botnets is challenging in the early stages, such as the spread and infection stages.(3)High-speed network real-time detection: the backbone network has the characteristics of high bandwidth, large traffic, and limited storage. These factors have led to the slow development of real-time detection technology. Lightweight real-time detection is an important content of future research.(4)Detection system structure coordination: the existing botnet detection system architecture has some problems. First, the centralized structure is unsuitable for large-scale network environments; the second is that the feature extraction method is not flexible, the structure is single, and multiple methods cannot be integrated; the third is the lack of coordination in the architecture function, and although some systems implement distributed detection, they lack effective information sharing and cooperation, and the coordination method is single, and they cannot respond quickly to botnet activities. The detection system framework must meet the requirements of distributed, scalable, and extended models and should realize the coordination between the detection system and other security systems.
(a)
(b)
(c)
(d)
7.2. Prospects
As an evolution of conventional malicious code, botnets provide controllers with a flexible and efficient command control mechanism, which is an ideal platform for DDoS, spam, information theft, click fraud, and malware distribution. With the convergence of the network era with the advent, botnets have seen changes in terms of infection targets, management and control technologies, and malicious behaviors, which pose a greater threat to future Internet security. Future research directions and technical difficulties in the field of botnet detection include the following:(1)Botnet multidimensional data representation: based on the knowledge graph technology, for DNS traffic, the entity-relationship and entity-attribute modeling embedding vectors are simultaneously performed, and the collected data are represented in multiple dimensions.(2)A method based on the combination of code and traffic analysis: this is conducive to the detection of botnets in all directions, improves the detection accuracy, and is suitable for the actual environment.(3)Lightweight deep learning model: existing traffic feature extraction methods that combine the spatiotemporal characteristics are commonly used in CNN + RNN. The model is more complex, and the processing speed is not high [104]. A fast quantum search algorithm called Grover and a cutting-edge lightweight deep neural network can help improve the feature extraction speed.(4)Efficient community mining algorithm and visual detection technology: the visualization of botnet behavior and efficient data mining algorithms in the field of social networks help botnet group detection.(5)Lightweight real-time detection: the DNS traffic in the network traffic is relatively less. For zombie DNS queries, community mining or deep learning methods are used for real-time detection.(6)Distributed elastic detection framework: based on blockchain, MTD, reinforcement learning, and other technologies, the detectors can be dynamically deployed to realize information sharing between detectors and improve the flexibility of the detection system.
8. Conclusion
This survey introduces the new construction mechanism of botnet, summarizes the latest technologies in the field of botnet detection, and makes a comparative analysis of the key technologies based on anomaly. One of the contributions of this paper is to propose an evaluation system for the comprehensive evaluation of detection techniques. New botnets emerge one after another, and new technologies and their comprehensive applications will be the research focus on the field in the future. This survey is of great significance for security personnel to analyze and defend botnets, and it may help the research community to produce better tools and techniques for mitigating the threat of botnets.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This paper was supported by the National Key Research and Development Project (2016YFB08011601). The authors would like to acknowledge the support.