Abstract

As an important part of power infrastructure, a power monitoring system provides real-time data acquisition, state detection, and remote control of power equipment for the power grid and can deal with sudden anomalies in time. The operation and maintenance of the power monitoring system are very important to ensure the stable operation of power grid. The current mainstream remote operation and maintenance mode has internal threats such as misoperation of operation and maintenance personnel or malicious damage caused by attackers stealing operation and maintenance authority. Meanwhile, the existing operation and maintenance audit has the problems of high human resource cost and limited supervision of operation and maintenance personnel. To solve this problem, this paper proposes a collaborative filtering method for operation and maintenance behavior of power monitoring system called CFomb. Exploiting a keyword matching algorithm, CFomb determines the power resources accessed by operation and maintenance users from multiple operation instructions and extracts operation and maintenance behaviors. Referring to the collaborative filtering idea, the feature matrix decomposition scheme is introduced to train the access probability model based on the historical normal behavior of multiple operation and maintenance users, which provides a basis for real-time prediction of the access behavior probability of target operation and maintenance users. The OTSU binarization technique is used to determine the probability threshold of abnormal operation and maintenance behaviors, identify abnormal behaviors through threshold comparison, and send real-time alarms to operation and maintenance audit. The simulation experiment results show that the method in this paper can effectively identify the abnormal behavior of operation and maintenance users, reduce the overhead of manual audit, and help improve the power monitoring system’s ability to respond to internal threats of operation and maintenance.

1. Introduction

As an important part of power infrastructure, the power monitoring system provides reliability support for the stable operation of power grid. In order to ensure the normal operation of the power operation system, it is necessary to carry out routine operation and maintenance for the system function, related equipment, hardware and software, and other internal resources, to deal with emergencies and abnormalities in time. The power monitoring system is distributed and deployed in different power stations, power distribution stations, and dispatching centers at all levels. Therefore, in case of employing on-site operation and maintenance, on the one hand, it will be difficult to deal with the emergencies anywhere in the system; on the other hand, it will be hard to control the operation and maintenance of power monitoring system. As a result, the operation and maintenance platform of power monitoring system needs to realize both remote and centralized operation and maintenance management.

The current typical operation and maintenance platform of the power monitoring system is generally built based on bastion host [13], whose technology can manage operation and maintenance accounts and assets uniformly and set the buffer to allow the assets of the power monitoring system to realize remote operation and maintenance without direct exposure to the outside. The administrator of bastion can configure the access strategy of operation and maintenance users. When logging in to the power monitoring system with fortress machine technology to implement operation and maintenance, the operation and maintenance personnel can record the operation and maintenance process of operation and maintenance personnel in real time, and the auditors can audit the operation and maintenance process of operation and maintenance personnel according to the audit rules, so as to achieve the supervision of operation and maintenance personnel. In addition, the bastion can isolate the internal resources of the power monitoring system from external exposure, centralize the identity of operation and maintenance users, and achieve centralized access control of operation and maintenance work. However, the remote operation and maintenance mode of the bastion host power monitoring system neglects the protection against internal threats, resulting in the high human resource cost of security audit.

To solve this problem, this paper proposes a collaborative filtering method (CFomb) for the operation and maintenance behaviors of power monitoring system. Exploiting keyword matching algorithm, CFomb determines the power resources accessed by operation and maintenance users from multiple operation instructions and extracts operation and maintenance behaviors; by reference to the collaborative filtering idea of the recommended system, the feature matrix decomposition scheme is introduced to train the access probability model based on the historical normal behavior of multiple operation and maintenance users, which provides a basis for real-time prediction about the access behavior probability of operation and maintenance users; OTSU binarization technique is used to determine the probability threshold of abnormal operation and maintenance behaviors, identify abnormal behaviors through threshold comparison, and send real-time alarms to operation and maintenance audit. Finally, the behavior that multiple operation and maintenance users of the power monitoring system access multiple power resources was simulated, and an experiment was carried out. The experimental results show that the methods proposed in this paper can help build a behavior probability prediction model for users and resources based on user behavior patterns and determine whether random behavior of users is abnormal in accordance with the thresholds generated automatically.

The overall structure of the paper is shown in Figure 1.

Various information security issues are introduced in the informatization development of the power industry. Among others, the internal threats of information system, as a hot issue in the research on current general information system security, have drawn increasing attention [47]. In terms of internal threat recognition methods, the traditional way was to audit the historical access logs of operation and maintenance users and detect the internal attacks that occurred by afterward examination. For example, Liu et al. [8] proposed the Log2vec method to detect the abnormal behavior of system. Based on the log information, this method extracts multiple factors, such as the sequential relationship between user behavior sequences within a day, the relationship between behavior sequences on different dates, and the behavior topology relationship of resource access by users, to carry out behavior modeling; the logging behavior is transformed into a vector using graph neural networks; the behavior vector is separated from abnormal behavior with the clustering algorithm, and then the insiders responsible for the abnormal behavior are traced. Gu and Guo [9] proposed an internal threat detection method based on role abnormal behavior mining, which mines the role abnormal behaviors using the sequence pattern and carries out pattern matching with KMP algorithm to recognize abnormal users. However, these methods require studying the historical data offline, failing to analyze new data in real time and to locate the malicious behaviors in real time. Therefore, it is difficult to avoid the losses caused by internal threats.

Rashid et al. [10] simulated the weekly normal behaviors of each user using the hidden Markov model and then applied them to the detection of the significant deviation between abnormal behaviors and normal behaviors. Happa [11] used the EM algorithm to train a GMM for the behaviors of each user in the first month, to simulate the normal behaviors of the user. The trained GMM is applied to computing the likelihood of input observations to indicate the possibility of the input. If the likelihood is smaller than the threshold, the observation will be detected as abnormal.

However, the above methods merely consider modeling the normal behaviors of users using the sequential features of a single user’s behaviors, failing to consider the correlation between user behaviors. As recommendation systems analyze and model user behavior data, they predict and recommend products that users do not use but are likely to be interested in. Collaborative filtering algorithm is a key algorithm in recommendation systems. Collaborative filtering can use user behaviors similar to those of the target user to infer the target user’s preference for a specific product and then make recommendations accordingly based on this preference. Therefore, by reference to the collaborative filtering idea, in the abnormal behavior detection of internal threats on the operation and maintenance platform of the power monitoring system, this paper not only considers the historical longitudinal features of operation and maintenance users but also integrates the transverse impacts among similar operation and maintenance users.

Compared with the existing work, the main innovations of this paper are as follows. (1)The introduction of the feature matrix decomposition method to train the access probability model, which is simple to compute, easy to obtain training data, and does not require complex processing, can be applied to more power monitoring system O&M scenarios(2)Combining collaborative filtering idea and OTSU binarization method to achieve probability prediction of real-time access behavior of O&M users and adaptive selection of probability threshold of abnormal O&M behavior, supporting more efficient and safe development of power monitoring system O&M(3)Simulation experiments are carried out based on the OTSU access behavior dataset for power monitoring system multiple O&M users to access multiple power resources behavior, and the results show that the proposed method can effectively identify the abnormal behavior of power monitoring system O&M users and can reduce the cost of human O&M audit

3. Design of the CFomb Method

3.1. Overall Design of the CFomb Method

The overall architecture of the CFomb method is shown in Figure 2. The method mainly consists of three modules, i.e., operation and maintenance behavior extraction, behavior analysis, and behavior alarm. The operation and maintenance behavior extraction module and the behavior alarm module are deployed in the bastion host on the operation and maintenance platform of the power monitoring system, while the computation nodes of the behavior analysis module can be deployed separately. In the CFomb method, the operation and maintenance behavior extraction module provides real-time behavior input for the behavior analysis module, which recognizes abnormal behavior and outputs abnormal behavior alarm information to the behavior alarm module, while the behavior alarm module displays the alarm information to the auditor and feeds back to the behavior analysis module.

3.2. Extraction of Operation and Maintenance Behaviors

As the processing object of the CFomb method, the description of an operation and maintenance behavior requires defining the specific operation and maintenance users and objects.

The operation and maintenance user can be a user with the operation and maintenance permission for the server of the power operating system. The user may carry out operation and maintenance based on the server of the power monitoring system within the preset time in accordance with the requirements of operation and maintenance [12]. When an operation and maintenance user carries out the operation and maintenance, the user needs to determine the target power resources to be accessed firstly and then input the corresponding operating instruction into the operation and maintenance platform of power monitoring system based on the determined target power resources. The operation and maintenance platform will obtain the corresponding operating instruction and determine the target power resources in accordance with the operating instruction, so that the target operation and maintenance user can smoothly carry out the corresponding operation and maintenance.

The operation and maintenance users and the power resources shall have unique identification information on the operation and maintenance platform of the power monitoring system. In CFomb method, the operation and maintenance behavior extraction module inserts the instruction extraction points into the bastion host to extract the real-time operation and maintenance instructions of operation and maintenance users. Through comparing and analyzing the instructions and keyword database of operation and maintenance objects, the module determines the operation and maintenance users and the resource objects accessed and outputs the operation and maintenance behaviors. For each instruction input by users, the algorithm indicated in Algorithm 1 is called to extract the user behavior in the instruction. As the input of the behavior analysis module, the behavior information output is used to recognize the current abnormal behavior of a user.

Input: a real-time operating instruction character string I of a user, the current user id user_id, and resource keyword database source_db
Output: behavior information{user_id, source_id} containing user id and resource id
1 Divide instruction I into multiple words
2for word in each words. /Traverse the words in the instruction/
3   use the word to query the keyword database to obtain the resource id source_id
4   if (successful query)
5      output the behavior information {user_id, source_id}
6   end if
7end for
3.3. Analysis of Operation and Maintenance Behaviors

As the core of CFomb method, the behavior analysis module needs to collect the historical access records of users to construct the behavior matrix and obtains the user model and resource model via behavior matrix decomposition. When the behavior analysis module obtains the real-time behavior information input of the behavior extraction module, it can extract the corresponding features from the user model and resource model, predict the probability of occurrence of such behavior, and determine whether the behavior is normal or abnormal based on the probability threshold classification. In addition, the behavior buffer will continuously record user behavior and take it as the training set adjustment information of follow-up iterative training to update the learning model.

3.3.1. Construct the Training Set

Stemming from the statistics on historical behaviors of operation and maintenance users, the training set employs the frequency of resource access by users to describe the probability of resource access by users. Since the scope of historical access behavior and resource access by users is limited, the training set merely contains the information of resource access frequency of a few users.

The training set is represented in the form of frequency matrix of rows and columns, among which the number of matrix row represents the total quantity of users on the operation and maintenance platform, while the number of matrix column represents the total quantity of resources on the operation and maintenance platform, and the behavior space (i.e., the scale of frequency matrix) is . Since the training set is a sparse matrix, it can be stored in the form of triple , among which represents the element () in the th row and the th column.

Based on the statistics on the historical behaviors of operation and maintenance users, the construction of the training set is updated continuously with the new access behavior of operation and maintenance users. If the record on user behaviors is unavailable in the initial construction stage of operation and maintenance platform, it is necessary to collect the user behaviors for a period of time before enabling the CFomb method for the follow-up work.

3.3.2. Matrix Decomposition

With the aim of predicting the probability of random behavior in behavior space, the CFomb method introduces the collaborative filtering algorithm based on matrix decomposition and carries out matrix decomposition of the training set to construct the feature model of users and resources [13]. The user model represents the matrix of rows and columns, the resource model represents the matrix of rows and columns, among which represents the implied feature dimension, whose value is much smaller than any one of and .

Matrices and represent the feature distribution of users and resources in dimensional feature space. The th column in matrix and the th column in matrix represent the dimensional eigenvector of user and resource , respectively. The similarity of these eigenvectors can be obtained through inner product computation, which includes similarity between users and resources, similarity among users, and similarity among resources. Among these, the similarity between users and resources represents the prediction of the probability of resource access by users.

Therefore, the user model and resource model obtained by the matrix decomposition algorithm shall guarantee that the product of matrix and matrix approaches, as much as possible, the frequency matrix indicated in the training set, i.e., frequency matrix can be represented by and in the form of the following formula.

For the user and the resource designated randomly in the behavior space, the th column and the th column can be taken, respectively, in the user model matrix and the resource model matrix to obtain dimensional column vectors and . Therefore, the predictive value of the element in the th row and the th column of matrix can be obtained by Formula (2). In the formula, represents the estimated value of the element in the th row and the th column of the frequency matrix by user model and resource model .

For the decomposition of matrices, the CFomb method employs ALS (alternating least squares).

First, the loss function is constructed as formula (3), among which represents the set of numbers of rows and columns corresponding to the recorded users and resources. represents the coefficient of regular terms for preventing overfitting.

Formulas (4) and (5) represent the gradient descent iteration formulas for this loss function.

The algorithm that computes user model and resource model by ALS is shown in Algorithm 2.

Input: Behavior frequency matrix PM×N of resource access by users
Output: User matrix XM×X and resource matrix YN×k
1 Construct the loss function Loss
2 Randomly initialize user model X and resource model Y
3while (Loss does not converge)
4   for i from 1 to M
5      for j from 1 to N
6        if (pi,jis recorded in the training set)
7           e = XiYj - pi,j /Predictive value of model- Corresponding value of training set/
8        end if
9xi,j = xi,j-2 alpha(eyi,j+lambadaxi,j)/alpha represents the learning rate/
10yi,j = yi,j-2 alpha(exi,j+lambadayi,j) /delta represents the regular terms/
11end for
12end for
13 Output Xi and Yjas user matrix XM×k and resource matrix YN×k
3.3.3. Abnormal Behavior Recognition

The abnormal behavior recognition by the CFomb method is divided into two steps: the first step is to quickly predict the probability of real-time behavior, and the second step is to discriminate whether the current behavior is abnormal based on the abnormal threshold.

The user model and resource model can be obtained by the matrix decomposition method provided in 3.3.2. For the behavior that random user accesses random resource , the eigenvector of user can be obtained by extracting the th column of , and that of resource can be obtained by extracting the th column of . The inner product of these two eigenvectors represents the probability prediction of such behavior.

The recognition of abnormal behavior can be completed by determining the abnormal behavior discrimination threshold after the behavior prediction probability is obtained. The CFomb method has introduced the OTSU algorithm to adaptively determine the abnormal behavior discrimination threshold.

First, the statistics on the prediction probability of all behaviors can be made in behavior space via user model and resource model. The prediction probability of abnormal behavior will be concentrated in the lower probability interval, while that of normal behavior will be concentrated in the higher probability interval. If an existing probability threshold divides the behavior into two maximum sets of between-cluster variance in the behavior space, the threshold can be taken as the probability threshold for abnormal behavior discrimination.

Formula (6) provides the computation of between-cluster variance in operation and maintenance behavior in the CFomb method.

In the formula, represents the proportion of behaviors with a prediction probability not more than the threshold in the behavior space, represents the proportion of behaviors with a prediction probability above the threshold in the behavior space, represents the mean probability of behaviors with a prediction probability not more than threshold , and represents the mean probability of behaviors with a prediction probability above the threshold .

The specific algorithm steps for computing threshold based on OTSU algorithm are shown in Algorithm 3.

Input: Set S of all probability prediction values of behavior space
Output: Abnormal behavior discrimination threshold T
1 Obtain the behavior probability set S
2maxScore =0
3for t = min(S) to max(S) by 0.0001/t represents the candidate threshold, traverseall the probabilities between min(S) and max(S), with an increased step size of 0.001 (0.1%)/
4S1={s|s∈S,s≤t}S2={s|s∈S,s>t}
5 Obtain the mean values of m1and m2 in S1 and S2
6 Obtain ratios p1 andp2 ofS1 and in S
7 Obtain the between-cluster variance of S1 andS2 with Formula (6)
8if (maxScore<)
9     T = t
10   end if
11end for
12 Output the final threshold T

Furthermore, the real-time operation and maintenance behavior can be realized in accordance with operation and maintenance user model , power resource model , and abnormal behavior discrimination threshold , with the specific algorithm shown in Algorithm 4.

Input: User modelX, resource model Y, abnormal behavior discrimination threshold T, and the extracted user behavior informationi and j (User i accesses Resource j)
Output: 1 /0; 1 indicates the normal behavior, and 0 the abnormal behavior
1 Extract vector Xi in the ith column of user model and vectorYjin thejth column of resource model
2p = Xi(Yj)T /p represents the prediction probability, which is obtained by computing the inner product of vectors XiandYj/
3if(p>T)
4 Output 1
5else
6 Output 0
7end if
3.3.4. Construction of Behavior Buffer

The CFomb method will continuously collect the behavior information of users during operation. In order to implement the dynamic adjustments to the user model and resource model, the CFomb method designs a behavior buffer in the behavior analysis module for receiving the behavior information of operation and maintenance users and feeding back new operation and maintenance user behaviors to the training model, thereby adjusting the user model and resource model.

The behavior information to be written in the behavior buffer will be stored in the form of triple (, , ), indicating that the number of times stored in the buffer for accessing resource by an operation and maintenance user is . Moreover, the behavior buffer maintains a counting sequence , in which , indicating that behavior information of the th operation and maintenance user is recorded in the buffer. Formula (7) represents the computational formula in which the access behavior of an operation and maintenance user to resource in the behavior buffer is integrated into the training set.

where represents the writing weight of user buffer. The higher the writing weight, the higher the change rate of training set. The specific values will not be specified in this paper. Algorithm 5 provides the workflow of operation and maintenance behavior buffer.

Input: The extracted user behavior informationiand j (user i accesses resource j),the audit tag,and the BUFFER_SIZE
Output: update the training set
1 Obtain the behavior information i, j
2 Set the default integration ratio as alpha=1
3if (tag=1) /audit feedback enters the buffer/
4   alpha = BUFFER_SIZE
5end if
6action[i][j] += alpha /statistics on behaviors in the buffer/
7user[i] += alpha/update the amount of user behavior/
8if (user[i]>BUFFER_SIZE)
9   Employ Formula (7) to update the information corresponding to user i in the training set
10     user(i)=0
11     action[i][j]=0
12end if

4. Experimental Evaluation

4.1. Data Preparation

In order to verify the recognition efficiency of the CFomb method on the users’ abnormal behaviors of access to the resources on the operation and maintenance platform of power monitoring system, the behavior of resource access by operation and maintenance users is simulated under the background of operation and maintenance [1415], and the dataset of resource access by operation and maintenance users is generated, based on which the recognition capability of the CFomb method on abnormal behaviors is verified. The dataset was obtained from the internal network data of China Southern Power Grid Corporation, which contains a total of 107,670 records on 1,170 users’ access to 7,455 resources. Moreover, 1,759 access records are generated by simulating the malicious user access behavior to verify the recognition effect.

4.2. Influence of Feature Dimension on Model Training

In the implementation of the algorithm of user behavior matrix decomposition, it is necessary to first determine the values of feature dimension in user model and resource model. The value of represents the dimension describing the features of users and resources when predicting the probability of resource access by users, which affects the prediction accuracy of the training model, model size, and training time. In order to verify the influence of on the prediction accuracy of training model, the predictive RMSE of model for 2 to 10 rounds of training is recorded, respectively, when the values of are 3, 5, and 10, and the RMSE decline curve was drawn. The experimental results are shown in Figure 3. It is obvious that the larger the value of in the training model is, the smaller the model error is; and all the training models converge to the steady state after the fifth round.

In order to further determine the influence of the values of on model training, the number of training times is set as 5, and the training time and RMSE with the value of ranging from 3 to 20 are recorded, as shown in Figures 4 and 5, respectively. It can be seen that when the value of ranges from 3 to 20, the increase in training time and the decline in RMSE are linear and variable.

Since it is necessary to continuously update the training set to implement iterative training in the context of operation and maintenance and quickly output model on the premise of ensuring the accuracy of the training model. The value of in this experiment is set as 10.

4.3. Threshold Sensitivity Analysis Based on the Test Set

The user model and resource model are obtained via the training on the behavior dataset in 4.2. Through the models, the probability prediction on all behaviors in behavior space can be implemented. In order to realize the abnormal behavior recognition, the abnormal behavior discrimination threshold shall be determined. In order to verify whether the probability prediction value of user behaviors obtained from the training model can distinguish the normal behavior from the malicious behavior, 100 malicious access behaviors and 5,000 normal behaviors are extracted to form a test set to test the training model.

In the experiment, the probabilities of all behaviors in the test set in the user model and the resource model are predicted, and the sensitivity of the data in dataset on the prediction probability threshold is verified using ROC (receiver operating characteristic) curve in combination with the tags indicating whether the behaviors are abnormal. The ROC curve can depict the relationship between false positive rate (FPR) and true positive rate (TPR) when test data classify the abnormal behaviors at different thresholds. TPR is the proportion of correctly identified positive data to the total positive data, i.e., the recall rate, while FPR indicates the percentage of negative data predicted to be positive when the actual value is negative. The specific formulas (8) and (9) of FPR and TPR are as follows:

In the formula, , , , and represent true positive, false positive, true negative, and false negative, respectively. Since it is generally assumed that behaviors equal to or lower than the probability threshold are abnormal, the behaviors lower than the threshold are determined as positive in this experiment [16, 17].

The ROC curve of test set data in this experiment is shown in Figure 6. The threshold sensitivity of data in test set can be represented by AUC (area under the curve) [18, 19], and the AUC of the test data is 82.80%. The ROC curve indicates that the probability threshold has a certain capability to classify normal and abnormal behaviors in the data of test set [2022]. However, a higher accuracy rate of abnormal behavior recognition will be accompanied by FPR increase to some extent [2325].

4.4. Recognition Effect of Abnormal Operation and Maintenance Behavior

In practical application, it is necessary to adaptively generate the abnormal behavior discrimination threshold in accordance with the probability distribution in behavior space; therefore, the CFomb applies the OTSU algorithm to the computation of the abnormal behavior discrimination threshold. In the experiment, the probabilities of all behaviors in behavior space are calculated, and the threshold of 0.0927 is obtained with the OSTU algorithm.

Afterward, the effect of abnormal behavior recognition proposed in this paper is further tested at the threshold of 0.0927. There are a total of 5,100 behavior records in the experiment. The confusion table (as shown in Table 1) is obtained based on the statistics on recognition results, with the corresponding TPR and FPR of 94% and 30.26%, respectively. The data indicates that 94% of malicious behaviors are recognized at the threshold of 0.0927, with 30.26% of normal behaviors determined as abnormal behaviors. Moreover, of all the behaviors, 31.51% are determined as normal behaviors, indicating that CFomb can exclude 68.49% of behavior records in manual audits, which significantly reduces the workload of security audit.

5. Conclusion and Future Work

Based on the matrix decomposition collaborative filtering method, this paper proposes the CFomb-based abnormal behavior collaborative filtering method under the background of internal attacks against the operation and maintenance platform of the power monitoring system. This method obtains a user model and resource model via training in global data access by users, combined with collaborative filtering idea and OTSU binarization method to realize the probability prediction of real-time access behavior of O&M users and adaptive selection of probability threshold of abnormal O&M behavior to support more efficient and safe development of power monitoring system O&M. The experiment indicates that the recognition effect is obvious, which can effectively prevent internal threats to the operation and maintenance platform of the power monitoring system and significantly reduce the workload of manual audits. The method proposed in this paper overcomes the inefficiency of traditional methods and provides new ideas for the operation and maintenance mode of power monitoring systems. Due to the limitation of the data in the training set, the probability prediction model has certain errors. Based on this paper, we will lower the misjudgment rate in the future in combination with other behavior features of operation and maintenance users as well as the feedback iteration of operation and maintenance personnel on the training model, to further study more effective methods for recognizing abnormal behaviors.

Data Availability

The labeled dataset used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare no competing interests.