Abstract

Intrusion detection system (IDS), the second security gate behind the firewall, can monitor the network without affecting the network performance and ensure the system security from the internal maximum. Many researches have applied traditional machine learning models, deep learning models, or hybrid models to IDS to improve detection effect. However, according to Predicted accuracy, Descriptive accuracy, and Relevancy (PDR) framework, most of detection models based on model-based interpretability lack good detection performance. To solve the problem, in this paper, we have proposed a novel intrusion detection system model based on model-based interpretability, called Interpretable Intrusion Detection System (I2DS). We firstly combine normal and attack samples reconstructed by AutoEncoder (AE) with training samples to highlight the normal and attack features, so that the classifier has a gorgeous effect. Then, Additive Tree (AddTree) is used as a binary classifier, which can provide excellent predictive performance in the combined dataset while maintaining good model-based interpretability. In the experiment, UNSW-NB15 dataset is used to evaluate our proposed model. For detection performance, I2DS achieves a detection accuracy of 99.95%, which is better than most of state-of-the-art intrusion detection methods. Moreover, I2DS maintains higher simulatability and captures the decision rules easily.

1. Introduction

With the rise of technologies such as the Internet of Things and cloud computing and the advent of the era of big data, the network security environment has become even worse. The increasingly frequent network attacks have caused security researchers to refocus on network intrusion detection. At present, the network intrusion detection system can be divided into two types according to the detection method, one is the abnormal intrusion detection system and the other is the misuse of the intrusion detection system [1]. In terms of the ability to detect new attacks, anomalous intrusion detection systems have a more prominent effect than misuse of intrusion detection systems, which make anomaly intrusion detection systems necessary for intrusion detection [2]. At the same time, the important role of machine learning in anomalous network intrusion detection has made machine learning the mainstream of constructing anomalous intrusion detection systems.

The establishment of an effective intrusion detection system first requires a dataset that conforms to the current network environment. Since 1999, the KDDCUP99 dataset and the NSL-KDD dataset have been widely used in the construction of network intrusion detection systems. Many intrusion detection models use these two datasets, such as the ANN and fuzzy clustering model [3], and the model combined misuse and anomaly detection for intrusion detection [4]. However, a study explains the reasons why these two datasets cannot reflect the output performance of the network intrusion detection system [5]. (1) The attack types of these two datasets are only a small part of modern network attack methods. (2) These two datasets were established in 1999 and are quite different from modern network traffic benchmarks. (3) The different distribution of the training and test datasets in the data type will cause the deviation of the classifier and the accuracy will decrease.

In addition, building an intrusion detection system requires a dynamic detection model. In recent years, many powerful intrusion detection models have been proposed, which makes the detection effect better than before, especially when deep learning becomes the mainstream. For example, the detection accuracy of DL-IDS using Convolutional Neural Network (CNN) and Long Short-Term Memory Network (LSTM) reaches 98.67% in the CICIDS2017 dataset [6]. In addition, some hybrid detection models based on traditional machine learning are performing well, for example, the detection performance of IDS based on decision tree and rules-based concepts reaches 96.67% in the CICIDS2017 dataset [7]. However, although the predictive accuracy of intrusion detection models is very high, most of them lack great interpretability. Some of them achieve higher detection accuracy but use a black box model or lower interpretable model, and some of them use an effective model-based interpretable model such as a decision tree and rule-based model but have lower detection accuracy than some state-of-the-art methods.

In this paper, we have proposed a novel intrusion detection model based on model-based interpretability, which can achieve higher detection accuracy and interpretability. In the proposed model, AddTree, a model-based interpretable method is used as classifier that can provide remarkable predictive performance and obtain admirable interpretability, and AE is used as feature rebuilder then can improve the predictive accuracy. Our contributions to this research are as follows:(1)We have proposed an intrusion detection model using AE and AddTree, in which we use AE to highlight the normal and attack features, respectively, and use AddTree and voting machine to detect whether a sample belongs to normal or attack.(2)We have conducted comparative experiments on UNSW-NB15 datasets, illustrating that I2DS outperforms most of the state-of-the-art methods.(3)According to the PDR framework, our proposed model has outstanding predictive accuracy and higher descriptive accuracy and relevancy. AddTree, as a kind of decision tree, has superior simulatability because of its high prediction performance and user-friendly and visual decision process. Combining with AE, I2DS achieves prominent predictive performance.

In this paper, the other sections are as follows: Section 2 proposes a review of related work to intrusion detection and interpretability of machine learning and its application. Section 3 supplies the details of our proposed detection model. Section 4 presents experimental details and the comparison of our model and other machine learning algorithms and the interpretability analysis of our model. Section 5 indicates the conclusion and an overall review of our research results.

2.1. Intrusion Detection

A number of studies have been conducted on intrusion detection from the network traffic perspective (Table 1). Gharaee and Hosseinvand proposed IDS based on genetic algorithm (GA) and support vector machines (SVM) [8]. They built new fitness function based on true positive rates, false positive rates, and computation time using SVM. Using KDDCUP-99 and UNSW-NB15 datasets, the detection performance of the model reached 99.26% and 93.24%, respectively. Huang and Lei proposed a novel Imbalanced Generative Adversarial Network (IGAN) and applied it into intrusion detection system [9]. The IGAN module was composed of CNN and fully connection network (FCN), and it was used to balance the percentage between normal and attack samples. For NSL-KDD, UNSW-NB15, and CICIDS2017 datasets, the detection rates of the model were 84.45%, 82.53%, and 99.79%, respectively. Liu et al. proposed a detection system based on feature representation and data augmentation [10]. The model was mainly divided into three parts: feature extraction, data augmentation, and detection. First, the data set is converted into an image set through the steps of feature encoding, feature reduction, standardization, and recirculation pixel permutation strategy. Then, least squares generative adversarial network was used to balance the training dataset and the convolutional neural network was used as classifier. NSL-KDD and UNSW-NB15 datasets were used to evaluate the proposed model. The model accuracy reached 98.80% and 94.90%, respectively. Zhang et al. proposed an intrusion detection model based on conditional Wasserstein Generative Adversarial Network (CWGAN) and cost-sensitive stacked autoencoders (CSSAE) [11]. CWGAN was used to generate the minority samples to reduce the class imbalance. CSSAE was used to extract deep feature representation of the data and detect attacks by utilizing cost-sensitive loss function. KDDTest+, KDDTest-21, and UNSW-NB15 datasets were used to evaluate the proposed model. The accuracy of model achieved 90.34%, 80.78%, and 93.27%. Ferrag et al. proposed rules and decision tree-based intrusion detection system [12]. This model consisted of three classifiers. The first classifier was REP tree, a binary classifier used to detect normal and attack. The second classifier was Jrip, detecting benign and one of the different categories of attacks. The third classifier was Forest PA, and its input was the result of the first two classifiers and the entire training set. For CICIDS2017 and BoT-IoT datasets, the detection rate of RDTIDS achieved 96.66% and 96.99%, respectively.

2.2. Interpretability of Machine Learning and its Applications

In recent years, deep learning models have very splendid performance in many fields (Table 2), such as face recognition, image classification, and natural language processing, but this performance is more dependent on the model’s highly nonlinear and parameter adjustment technology [16]. From the perspective of human beings, if the decision-making process of the model is incomprehensible, the model is unexplainable [17]. Therefore, interpretability can be defined as users have enough understandable information to understand the decision-making process and the decision result of the model. In machine learning, interpretability is divided into two categories: (1) model-based interpretability that constructs a model which is interpretable in nature and (2) post hoc interpretability that applying an interpretability method after training a black box model.

There are lots of researches that focus on it. Zhang et al. used the decision tree to quantitatively explain the logic of deep neural network prediction at the semantic level [13]. They used the decision tree to interpret the neural network prediction results for each input image, determined which parts of the object are used for prediction, and quantified the contribution of each object part to the prediction. Luna et al. proposed a new decision tree model called Additive Tree [14]. They combined Gradient Boosting with Classification and Regression Trees (CART) to build a more accurate tree that improved the predictive performance and maintained the characteristic of the decision tree. Murdoch et al. proposed a PDR framework for interpretability [15]. They defined interpretability of three categories: predictive accuracy, descriptive accuracy, and relevancy.

3. Materials and Methods

3.1. Overview of Approach

The main contribution of the proposed model is to detect network traffic and label them as benign or attack. This detection process can be divided into two phases (Figure 1): (1) we do data preprocessing for data sets, including data standardization and data cleaning. (2) We use the proposed model to detect the dataset and then get the detection result.

3.2. Dataset

In this paper, the dataset we use is UNSW-NB15, which is developed by the IXIA PerfectStorm tool in the Cyber Range Lab of the Australian Centre for Cyber Security [18]. The Argus and Bro-IDS tools are used, and twelve algorithms are developed to generate totally 49 features with the class label. The created features can be classified into five categories: flow features, basic features, content features, time features, and additional generated features (Table 3). The dataset contains 9 attack types, including Fuzzers, Analysis, Backdoor, DoS, Exploits, Generic, Reconnaissance, Shellcode, and Worms [19]. However, the attributes in UNSW-NB15 training and testing dataset (UNSW_NB15_training-set.csv and UNSW_NB15_testing-set.csv) [20] contain only 44 features, including 42 attributes and 2 categories. Table 4 illustrates the description of training-set and testing-set of UNSW-NB15.

3.3. Data Processing

In this section, the UNSW-NB15 data processing operations are as follows:(1)We split UNSW-NB15 into two datasets by label category(2)We remove the columns named “id” and “attack_cat”(3)We encode the columns named “proto,” “service” and “state”(4)We normalize the data with the minimum-maximum normalization method which is defined as

3.4. Proposed Model

In this section, we describe I2DS we proposed dealing with problem of network intrusion detection. It combines an unsupervised approach using two AEs with a supervised machine learning model using Additive Tree. The architecture of the proposed model is described in Figure 2.

3.5. Feature Selection Using Autoencoder

Different from classical multilayer perceptron (MLP), an autoencoder (AE) is a particular neural network which tries to copy the input to the output. In other words, the task of AE is an attempt to the best to make the output content the same as the input content. In particular, AE contains two parts: encoder and decoder . For input data , AE can make the be approximately equal to .

In this paper, we use two AEs to learn features of attack and normal, respectively. The loss function is defined as . The architecture of AE is described in Table 5. In our proposed model, AE reestablishes normal and attack samples, respectively, which can emphasize the normal or attack features. The benefit of this is that the original data can enhance the main features and weaken unrelated features by combining with data generated by AE.

3.6. Binary Classifier Using Additive Tree

In order to build a decision tree model with higher predictive accuracy, Luna et al. proposed a new decision tree model, called Additive Tree (AddTree) [14]. The Additive Tree walks like CART but learns like Gradient Boosting. In other words, it is an algorithm that builds a single decision tree, similar to CART, but the training is similar to boosting stumps (a stump is a tree of depth 1). More specifically, the steps of AddTree are similar to those of Gradient Boosting. Its iterative steps are as follows: (1) calculate the negative gradient, (2) fit the weak learner by minimizing the product of the square error and the instance weight, (3) find the optimal scale by minimizing the product of the instance weight and the square of the difference between the estimated function, the weak learner factor, and the original data, (4) update the current function estimate, and (5) finally calculate the weights of the left and right subtrees, until the node field and the classifier partition are empty. In other words, unlike CART, AddTree uses weight to measure the partitioning effect of the dataset not the Gini coefficient. This improves accuracy without compromising interpretability [14].

3.7. Model Design

The algorithm steps of our proposed model are as follows:(1)We use the attack-set and normal-set obtained in data processing to train the AE (attack-AE and normal-AE).(2)We put the UNSW-NB15 testing-set into attack-AE and normal-AE to obtain reconstructed datasets (re-attack-set and re-normal-set).(3)We combine UNSW-NB15 Testing-set with re-attack-set and re-normal-set, respectively (attack-addtree-set and normal-addtree-set).(4)We use attack-addtree-set and normal-addtree-set to train AddTree and then put the results into voting machine to get the final detection result.

4. Results and Discussion

4.1. Performance Metrics

To evaluate our proposed model, we use recall, precision, F1-score, and accuracy as the primary metrics. Accuracy is the proportion of all correct predictions to the total. Precision refers to the proportion of true correctness that is positive for all predictions, while recall refers to the proportion that is really correct and accounts for all the actual positives. F1 score is the harmonic mean of recall and precision, considering both the classifying ability and the detection rate.

Four statistical standards are defined as follows:

4.2. Predictive Accuracy

In this section, we compare our proposed model I2DS with IDS-AddTree which not use AE (Table 6). It can see from the results that, in the primary metrics, I2DS is better than IDS-AddTree.

In addition, we also compare our proposed model with other state-of-the-art classifiers (Table 7). It can be seen that the accuracy of our proposed model is the best, achieving 99.95%. For other evaluation metrics, the precision, recall, and the F1-score of our proposed model are higher than those of other classifiers. Therefore, the proposed method achieves satisfactory performance across evaluation metrics when compared to other classifiers using the UNSW-NB15 dataset.

4.3. Interpretability

Predictive accuracy, descriptive accuracy, and relevancy (PDR) framework is a method to define interpretable machine learning [15]. According to the PDR framework, if a model has exceptional interpretability, this model should have higher predictive accuracy, descriptive accuracy, and relevancy. Decision tree is considered as an interpretable model because it has higher simulatability. Human beings can use input data and decision tree model parameters in the appropriate time to make predictions through each calculation step [28]. In the decision tree, each leaf node of the tree represents a class and is interpreted by the path from the root node to the leaf node in terms of a rule such as “If A1 and A2 and A3, then class C1,” where A1, A2, and A3 are the clauses involving the attributes and C1 is the class label [29]. However, although the traditional decision tree has good simulatability, it is very poor in prediction performance. Therefore, AddTree, which is proposed by Luna et al., makes up for the poor prediction performance of traditional decision tree [14]. It maintains the advantages of the decision tree that it can be visualized directly which means it can help to understand the influence of different variables on decision-making results by using the criteria of information theory and provides the same great predictive performance as ensemble learning and neural networks.

In terms of predictive accuracy, I2DS reaches 99.95% and outperforms most advanced intrusion detection models. In terms of descriptive accuracy and relevancy, AddTree, as a classifier to classify a certain sample, can be visualized directly and easily (Figure 3), and in the AE and voting machine, the architecture is based on fully connect network, which can be seen as the process of matrix linear transformation. In conclusion, in the model-based interpretability, I2DS obtains higher interpretability.

5. Conclusion

This paper proposes a more accurate and interpretable Intrusion detection model using AutoEncoder and Additive Tree, called I2DS. In our proposed model, two autoencoders are learned from normal and attack flows, respectively. They can highlight the main features of traffic flows during reconstruction. AddTree is used as classifier to classify which class a certain sample belongs to. Additionally, the UNSW-NB15 dataset is used to evaluate the proposed model.

In terms of predictive accuracy, we use recall, precision, F1-score, and accuracy as the primary metrics. The results demonstrate that the primary metrics of I2DS is better than most of state-of-the-art intrusion detection methods. In terms of descriptive accuracy and relevancy, AddTree of I2DS maintains the characteristic of a decision tree that can be easily visualized and capture the interactive information between features. Autoencoder of I2DS is composed of fully connect network, which means the AE can be regarded as linear matrix transformation.

In general, according to the PDR framework, the interpretability of I2DS can be said to be super-excellent. I2DS has the excellent predictive performance and the high-level descriptive accuracy and relevancy. We come to the conclusion that I2DS provides a first-class predictive performance and credible interpretation of intrusion detection system.

Data Availability

The dataset can be obtained in the Kaggle (https://www.kaggle.com/wenfengxu/i2ds-for-addtree), and the code can be obtained in the Github (https://github.com/Xuwenfeng-GUET/I2DS).

Conflicts of Interest

The authors declare that there are no conflicts of interest.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 61762026 and 61462018, in part by Guangxi Natural Science Foundation under Grant 2017GXNSFAA198278, and in part by the Innovation Project of GUET Graduate Education under Grant 2019YCXS056.