Abstract
Network traffic classification technologies could be used by attackers to implement network monitoring and then launch traffic analysis attacks or website fingerprint attacks. In order to prevent such attacks, a novel way to generate adversarial samples of network traffic from the perspective of the defender is proposed. By adding perturbation to the normal network traffic, a kind of adversarial network traffic is formed, which will cause misclassification when the attackers are implementing network traffic classification with deep convolutional neural networks (CNN) as a classification model. The paper uses the concept of adversarial samples in image recognition for reference to the field of network traffic classification and chooses several different methods to generate adversarial samples of network traffic. The experiment, in which the LeNet-5 CNN is selected as a classification model used by attackers and Vgg16 CNN is selected as the model to test the transferability of the adversarial network traffic generated, shows the effect of the adversarial network traffic samples.
1. Introduction
As a basic technology for enhancing network controllability, network traffic classification technology helps researchers understand traffic distribution, optimize network transmission, and improve network service quality; however, it is often leveraged by attackers for monitoring network traffic against the network targets and classifying the application types (such as mail, multimedia, and websites) the network traffic belong to. Based on the classification results, network traffic interception is implemented and a possible website fingerprint attack may be followed [1]. In particular, the network traffic classification, in which area machine learning and deep learning are applied, provides attackers easier conditions that result in extremely high classification accuracy. A typical scenario for a network traffic classification method based on deep learning that is used by attackers is shown in Figure 1.
Although the application of deep learning in network traffic classification can improve the accuracy of classification and has demonstrated huge potential in areas such as image recognition and natural language processing, adversaries against the deep learning models including the convolutional neural networks (CNN) have raised the interest of scholars on the concept of “Adversarial Sample” that was introduced to the area of computer vision by Szegedy et al. [2].
In the study of image recognition, Szegedy has found that CNN tends to give an error output with high confidence degrees when intentionally adding some undetectable and tiny perturbations to the input samples of the learning models. For deep learning models, these are called “Adversarial Samples” that are crafted by these tiny perturbations to the original dataset. From the perspective of attack, the most direct application of adversarial samples is in the area of computer vision, including face identification and automatic driving. By adding perturbation undetected by eyes to the image, failures in face identification and traffic signs [3] are triggered and damages from misclassification are then caused. In the area of information security, it can also lead to detection avoidance [4] by deceiving the malware detection models based on neural network. However, on the contrary, the adversarial samples, from the perspective of defense, are also of high value. First, it can improve the robustness of deep learning models in responding to possible adversarial sample attack by being trained with adversarial samples generated in advance [5]. Second, the adversarial samples can be leveraged to deceive the classification models by attackers using the deep learning network, which results in misclassification and increase of attack cost, thus cancelling the attacks. From the second view above, this paper is designed for defenders to trigger errors in attackers’ network classification by crafting adversarial samples for network traffic with the addition of perturbation and thus forming deceptive network traffic against attackers’ network traffic classification attacks.
In this paper, the concept of adversarial samples is introduced to defend the network traffic classification attacks initiated by attackers. Adversarial samples of network traffic are generated to deceive network traffic classification models based on deep learning network used by the attackers, resulting in misclassification and attack failure. The contributions of the paper are as follows: firstly, the concept of adversarial samples is introduced into network traffic as a view of active defense, and deceptive effects initiated by different adversarial samples are compared. Secondly, contrary to the fact that attackers initiate attacks with adversarial samples in other areas, the adversarial samples of a network are considered as a defensive way to confuse the attackers’ classification models, that can be regarded as “attacks in active defense.” Finally, the LeNet-5 CNN is selected as a network traffic classification model used by attackers to be deceived, and Vgg16 CNN is chosen as the model to test the transferability of the adversarial network traffic generated.
2. Related Work
2.1. Network Traffic Classification
Based on the granularity of network traffic, the study in network traffic classification is mainly for the following three levels [6]: packet, flow, and stream. In the three levels mentioned above, the flow level includes five types of flow network traffic according to different granularities [7] as shown in Table 1, which are the most widely used.
In this paper, flow network traffic is used as the original data. By crafting adversarial samples of network traffic, the defenders deceive the attackers who use deep learning methods as their classification models. The classification methods based on deep learning assume that the statistical characteristics (such as flow duration distribution) of the network layers for some types of applications are unique. These methods, including Decision Tree, Naive Bayes, Support Vector Machine, Association Rules Learning, Neural Network, and Genetic Algorithm, are applied in the classification model’s construction to classify, with such characteristics as broad scenarios, high classification accuracy, and ability in encrypted data traffic classification.
For studies of traffic classification based on machine learning, the main idea is to construct united statistical attributes of traffic as the fingerprint to classify. Ref. [8] applies for the first time machine learning into network traffic and assumes the fact that the bytes in flow can be regarded as pixels in images, and the deep learning method with excellent performance in image recognition can be used for network classification. Ref. [9] integrates feature extraction, feature selection, and classification into an end-to-end framework and calculates the load bytes of different behaviours by first-order CNN to construct fingerprints. Ref. [10] leverages characteristics of anonymized TOR (The Onion Router) network and applies the direction of the length sequence as the input for deep learning networks including SAE (Stacked Auto Encode), CNN, and LSTM (Long Short-Term Memory), to classify the webpage access. Ref. [11] applies for the first time the method of representation learning into the area of malicious network traffic classification, which regards the original traffic data as images, then it conducts classification with CNN that does well in image classification tasks, and finally, it achieves the purpose of classifying the malicious network traffic. These studies have proven the feasibility of deep learning in traffic classification and at the same time, provided targets for adversarial samples of network traffic classification based on deep learning.
For studies in adversarial network traffic classification based on deep learning, Ref. [12] proposes a defense method loading background network traffic and validates the Tor and JAP (Java Anon Proxy) anonymized network. Ref. [13] has validated the effects of encrypted network traffic classification adversary filled by encrypted protocol bytes. Ref. [14] applies different real traffic as noise during website access. Ref. [15] proposes that Walkie-talkie loads a website in simplex mode to confuse the burst feature. The abovementioned studies have mainly achieved the goal of modifying the communication characteristics of the traffic and have proposed methods that mostly focus on how to avoid being detected, which are of limited ability to disguise and deceive, and with an insufficient adversary. At present, the studies close to our work are those on network traffic disguise and confusion in the area of privacy protection, in which TOR releases obfsproxy, an obfuscated proxy software [16] that makes the encrypted traffic of SSL (Secure Socket Layer) or TLS (Transport Layer Security) look like unencrypted HTTP or instance communication traffic. Ref. [17] releases TOR’s transmission layer plug-in, SkypeMorph, to fill the communication traffic between a TOR client and a network bridge to Skype video communication traffic for statistical analysis of adversarial traffic. Ref. [18] proposes the method of analysis of traffic classification rules in a black box, which can infer traffic analysis identification rules through tests and thus modify the communication packet to avoid being detected. However, all these studies neither apply the concept of adversarial samples into those on adversarial traffic analysis nor discuss it as a method of defense for defenders, which, however are the focus of this paper.
2.2. Adversarial Samples
The key of adversarial samples is to craft adversarial perturbation. In the area of computer vision, it is essential for perturbation to meet the requirement of being invisible to human eyes after addition of original images and be able to confuse original classification models. In this paper, the deception for traffic classification models still have to meet certain requirements (e.g., bandwidth), though it is not necessary for the perturbation being crafted to meet the requirement of “being invisible to human eyes.”
Now, the majority of studies are focused on crafting the adversarial perturbation to misclassify an image. Szegedy et al. [2] discovered the weakness of the deep neural network in the area of image classification, proposed the concept of adversarial samples, and described the perturbation crafting as an optimized issue for the first time. Goodfellow et al. [19] proposed an optimal method of max norm constrained perturbation, which is called the Fast Gradient Sign Method (FGSM), to improve the computational efficiency and proved that high dimension linearity is the primary reason to make adversarial samples better. Kurakin et al. [20] proposed a basic iteration method that leverages FGSM in iteratively crafting perturbation. Moosavi-Dezfooli et al. [21] discovered adversarial perturbation irrelevant to particular images in image classification models, that is, the existence of universal perturbations, which can lead the classification models to misclassify any image with the addition of this perturbation. Athalye et al. [22] have discovered that the deep network classifier could also be deceived by objects in the real world printed by 3D printers. DeepFool [23] further improved the effectiveness of adversarial perturbation. Metzen et al. [24] introduced Universal Adversarial Perturbation (UAP) for semantic segmentation tasks and extended the iterative FGSM attack of [21] and changed the labels for prediction of each pixel. Mopuri et al. sought data-free universal perturbation without any sample data distribution. They proposed a new algorithm without target data to craft universal adversarial perturbation called FFF [25]. Their later work, GDUAP [26], has improved the attack effect to cause misclassification for different structures and parameter classification models and validated the validity of the method in tasks across computer visions. Furthermore, attacks in other areas are studied besides those on classification and recognition tasks in computer visions, and there is presently no research on attacks against network traffic classification.
3. Security Models
3.1. Attack Model
This paper assumes that attackers can observe the flow-level network traffic between host nodes and extract features such as packet size and internal packet arrive time. By training on classification models with these network traffic, attackers could infer the application types to thus conduct classification. From [27, 28], we build an attack model, which is specifically described as follows: attackers attempt to classify the network traffic being observed into type of application set , in which:. The feature set of network traffic is , . is the classification function of the model, and the output value is the probability of type in . Related notations are shown in Table 2.
3.2. Defense Model
Defenders, according to network traffic , generate network traffic adversarial samples by adding perturbation . This paper will generate different adversarial samples of network traffic by different methods of crafting perturbation, from which the feature set is extracted, which will make the output of the attackers’ classification function different from the original output . That is to say, the attackers will misclassify the traffic into the type th rather than type th. Related notations are shown in Table 3.
3.3. Methods of Generating Perturbation
Ref. [29] summarizes the perturbation crafting into full-pixel perturbation and partial-pixel perturbation, on the basis of which there are three secondary types including target/nontarget, black box/white box, and visible/invisible. In collaboration with characteristics of network traffic classification, the methods of crafting perturbation introduced in this paper are just like those of the full-pixel perturbation in image classification; that is, adversarial samples are crafted under the context that the parameters and internal structure of the classifier used (such as LeNet-5) by attackers are known. These adversarial samples are required to lead the attackers’ classifier to misclassify into not only target label but also nontarget label. Based on the abovementioned, the four perturbation crafting methods introduced in this paper are as follows: (1)L-BFGS
L-BFGS is introduced by Szegedy [2] when he proposed the concept of adversarial samples. L-BFGS generates adversarial samples based on optimization, and is described as follows: (2)FGSM
As one of the basic methods in crafting adversarial samples, FGSM, proposed by Goodfellow et al. [19], induces a network to misclassify the image generated by adding increments into the direction of a gradient based on the principle of gradient descent. FGSM calculates perturbation by using the following: (3)JSMA
JSMA is a typical white box and targeted attack algorithm constrained by norm proposed by Papernot et al. in 2016, which is aimed at computing a direct mapping from the input to the output to achieve an explicit adversarial goal. JSMA algorithm mainly includes three processes: calculating forward derivative of a deep neural network, calculating adversarial saliency maps, and modifying samples by adding perturbation [30]. (4)C&W Method
C&W is proposed by Carlini and Wagner [31] based on FGSM, L-BFGS, and JSMA, which improves greatly in norm , , . The method with norm as an example is shown in equation (3). C&W can produce strong adversarial samples, enhance its adversarial transferability, and achieve the ability of black box attacks.
where
3.4. Adversarial Samples of Network Traffic Crafting Algorithm
Based on the abovementioned analysis and perturbation crafting algorithm, this paper has designed the adversarial samples of a network traffic crafting algorithm. The details are as follows:
| ||||||||||||||||||||||||||||||||
In Algorithm 1, the real traffic needs to be preprocessed and normalized first. And then, each characteristic value of multiple types of characteristics is reshaped as a grey value in 0-255 and the network traffic is visualized. Next, the CNN model selected by attackers is constructed and trained. The function Training (TF, mode) enables it to classify the traffic data visualized and test the accuracy of classification. At the same time, different methods of crafting perturbation are used to generate perturbation, which will be overlayed with original traffic to be adversarial samples of network traffic . Finally, by comparing and , adversarial samples of traffic network will be evaluated.
4. Experiments
This paper constructs an attack and defense scenario shown in Figure 2, in which attackers are assumed to be able to observe the flow-level network traffic of different applications between host nodes and use the classification models based on deep learning for further attacks. Defenders use the adversarial samples of network traffic crafting method proposed in this paper to add a different perturbation to lead attackers to misclassify during network traffic classification and thus achieve the purpose of defense.
Environment and parameters required by the experiment are shown in Table 4.
4.1. Dataset
The USTC-TFC2016 dataset [11] used in this paper is as the flow traffic observed by attackers which is commonly used by network traffic classification. This dataset includes ten types of malware traffic captured from the real network environment by CTU researchers from 2011 to 2015 and ten types of normal traffic data simulated by professional tools. To reflect more kinds of traffic as possible, ten kinds of traffic contain eight classes of common applications. The size of the USTC-TFC2016 dataset is 3.71 GB, and the format is pcap.
4.2. Data Preprocessing
In this part, with the toolkit USTC-TK2016, raw traffic data (pcap format) is converted to CNN’s input data (idx format). The whole process includes traffic split, traffic clear, image generation, and IDX conversion [11]. After preprocessing, 20 types of different applications of network traffic are formed, including 10 types of 243761 normal traffic flows and 10 types of 179252 malicious traffic flows, in which 90% (379812 flows) are used as training dataset and 10% (42201 flows) as test dataset. The statistical chart of dataset distribution is showed in Figure 3.
Each of the 20 types of network traffic can be visualized to grey image with 784 () bytes. The visualization results are shown in Figure 4. In Figure 4, the left group shows the visualization result of all 20 types of traffic and the right group shows the consistency in the same traffic type. It is obvious that these images visualized from network traffic have obvious discrimination degree, and each type of traffic has high consistency.
4.3. Attacker Classification Model
It is assumed that the attacker uses LeNet-5 CNN as his classification model, which is widely used in classification of network traffic applications [33]. Ref. [34] has improved the LeNet-5 CNN model with its network structure, and the network structure and parameters of LeNet-5 CNN are shown in Figure 5 and parameters in Table 5.
To validate the transferability of the adversarial samples of network traffic generated, the Vgg-16 CNN model is selected as the classification model to test adversarial samples crafted for LeNet-5. The parameters of the network structure of Vgg-16 are shown in Table 6.
4.4. Classification Test
Without the defense of adversarial samples of network traffic, the effect of classification of LeNet-5 and Vgg-16 used by the attacker is perfect. The classification test is shown in Figure 6 and Table 7 with three evaluation metrics: accuracy, precision, and F1 score.
4.5. Perturbation Crafting
With Algorithm 1 and four methods of perturbation crafting, adversarial samples of network traffic of the defender model are crafted for LeNet-5 CNN. Taking with Geodo type as the example, perturbations crafted by different methods are shown in Figure 7. In Figure 7, the column “Perturbation” shows the difference of perturbations generated by four methods, in which the brightness of the perturbation pixel corresponds to the value of the heat map. The value “1” and value “-1” of the heat map mean the strongest “positive” and “negative” perturbations after standardization. For example, perturbations generated by JASM are stronger than the perturbations generated by C&W.
4.6. Comparison and Analysis
The comparison of the experiment consists of two parts: (1) The defender carries out untargeted defense to the attacker, which means the purpose of defender is make the attacker misclassify the application class to another but no particular class. For example, the attacker misclassifies the Outlook network traffic to any other class such as Gmail and FTP. (2) The defender carries out targeted defense to the attacker, which means the purpose of the defender is to make the attacker misclassify the application class to a particular class. For example, the attacker misclassifies the Outlook network traffic to MySQL. In each part, after the test of the effect of the adversarial samples for LetNet-5 CNN, the transferability is validated, which means the defender uses adversarial samples generated for LeNet-5 CNN to deceive Vgg-16 CNN. To evaluate the effect of deceiving in untargeted defense, deceiving rate (DR) is used as shown in equation (4). And in targeted defense, matching rate (MR) [35] is defined as that which describes the percentage of the adversarial examples generated for the source model that is misclassified as the target label by the target model.
To evaluate the quality of adversarial samples generated, norm, norm, and Structural Similarity Index (SSIM) are used. The comparison of the experiment is shown in Table 8 and Table 9:
From the comparison in Tables 8 and 9, we can validate the effect of adversarial samples of network traffic generated by four different methods. In the untargeted defense group, the adversarial samples crafted by C&W perform best on deceiving LeNet-5 CNN with low change to original but with disadvantages of slow to crafting perturbation and low transferability to other CNN models, which could be used in an application field that could provide high computation ability and demand for high deceiving rate. FGSM could craft perturbation quickly and transfer the deception to other CNNs. However, the change is to the original image of perturbation by FGSM which is much more than other methods. In the targeted defense group, C&W is also the best to perform the ability of deceiving LeNet-5 but has no effect of transferability to Vgg-16, and neither are other methods. Contrary to L-BFGS performing better in this part than in the untargeted part, FGSM performs worse in contrast to the performance in the untargeted part. About the transferability of the adversarial samples of network traffic, only JASM performs a little bit of transferability.
5. Conclusion and Further Work
This paper first describes the current research status in the area of network traffic classification. Then, from the perspective of defenders and based on researches related, it introduces the concept of adversarial samples to network traffic and raises a novel way to generate adversarial samples of network traffic. After the models of attack and defense are described, experiments are conducted with four methods of crafting perturbation. In the experiments, LeNet-5 CNN is considered as the classification model used by the attacker to be deceived. By adding perturbation generated by different methods to grey images transformed from normal network traffic, the adversarial samples of network traffic are formed, respectively, to confuse the target model. The experiments not only compared the effect of adversarial samples generated on LeNet-5 CNN but also validated the transferability of adversarial samples of network traffic on Vgg-16 CNN.
There are three limitations and related future work about this work. First, the main goal of this paper is to show the effect of adversarial samples of network traffic, so only the basic methods of crafting perturbation are used and compared. The effect of other methods should also be considered. Secondly, it is assumed that the classification model used by the attacker in the experiment is LeNet-5. However, in the real attack and defense, other CNNs may be selected as the classification model too. So, the effect on other CNNs will be validated next. Lastly, our work in this paper only performs the transformation from the network traffic to grey images, but how to change the image to network traffic and how to keep the integrity of the original network traffic during transforming need to be studied carefully in further work.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare no conflicts of interest.
Acknowledgments
This work was supported by the Foundation of Science and Technology on Information Assurance Laboratory (No. KJ-15-108).