Abstract
Reliable operation of distributed internet of things (IoT) devices is essential for supporting two-way interaction between energy flow and information flow in a distribution power grid. Traditional reliability evaluation methods suffer from several challenges including complex evaluation indicator selection and inadaptability of fixed weights with actual reliability requirements. In this paper, we first establish a multilevel reliability evaluation indicator system, where the indicators are selected through joint consideration of comprehensiveness and effectiveness. Then, we propose a fuzzy analytic hierarchy process- (FAHP-) based reliability evaluation method, where a three-layer reliability evaluation architecture is established. Specifically, a 0.1-0.9 scaling method is adopted to establish the fuzzy judgment matrix and scale the important relationship among state-layer elements, which can be dynamically adjusted to generate the weights adapting with actual operation state. The scores of distributed IoT devices and corresponding reliability levels can be obtained through the weighted summation of the scores of each layer. Simulation results verify the effectiveness and accuracy of the proposed method through comparing with actual reliability value and two existing evaluation methods.
1. Introduction
With the integration of distributed renewable energy sources, flexible loads, and energy storage devices into the distribution power grid, two-way interaction between energy flow and information flow is essential to improve energy-supply balance and promote renewable energy consumption [1, 2]. Massive distributed internet of things (IoT) devices with strong interaction and perception capabilities are deployed in the distribution power grid to collect operation status and electrical parameters in real time, which provide data support for the optimization of distributed energy management [3]. Therefore, the reliability of distributed IoT devices is vital to the safe and stable operation of a distribution power grid. It is necessary to extract reasonable and valid analysis data from numerous evaluation indicators and build a reliability evaluation system to improve the operation reliability of distributed IoT devices [4, 5].
However, the performance evaluation indicators of distributed IoT devices span numerous categories, and the correlations among different categories of indicators are diverse [6, 7]. Several major critical technical challenges are summarized as follows.
Challenge 1. Difficulty in selecting evaluation indicators. Numerous evaluation indicators exist for distributed IoT devices in a distribution power grid, and the selection of evaluation indicators has a significant impact on the construction of the evaluation indicator system. Selecting insufficient indicators result in inaccurate reliability assessment, while selecting excessive indicators increase the complexity of the whole evaluation process. Therefore, it is a challenge to appropriately select the evaluation indicators to achieve accurate and concise reliability evaluation of distributed IoT devices.
Challenge 2. Difficulty in calculating weights of indicators. Various services of a distribution power grid impose different reliability requirements of distributed IoT devices. When fixed weights are used to evaluate the reliability of the same indicator under different services or scenarios, the evaluation results cannot reflect the real reliability performance of the IoT devices. Therefore, how to determine the appropriate weights of indicators under different scenarios is another challenge.
There exist some works on the performance evaluation of IoT devices. In [8], Yuwen proposed a comprehensive evaluation indicator system to evaluate the safety performance of a communication system in a distribution power grid. In [9], Xiao et al. proposed a principal component analysis-based sensor performance evaluation method to achieve online monitoring of sensor faults. However, these works do not consider reliability evaluation, which are not suitable for a distribution power grid with high-reliability requirements on distributed IoT devices. Moreover, the expert evaluation method determines indicator weights according to individual expert preferences in performance evaluation, which cannot well handle the ambiguity of the indicators due to experts’ personal subjective preferences.
The fuzzy analytic hierarchy process (FAHP) eliminates the influence of experts’ personal subjective preferences on weight determination by fuzzy processing of indicator weights. It provides a solution for quantitative and qualitative analysis of multiobjective decision-making problems. In [10], Yuan proposed a comprehensive evaluation method of power quality based on incentive and punishment mechanism. FAHP was utilized to calculate the comprehensive value to evaluate power quality. In [11], Zeng et al. proposed a FAHP-based intelligent evaluation method for IoT devices in distribution station. A hierarchical structure model was established to quantitatively evaluate the importance of performance indicators. In [12], Li proposed a FAHP-based comprehensive evaluation system to evaluate multidimensional performance of navigation IoT devices, such as economy, reliability, and security. However, these works only consider a few evaluation indicators and ignore the impact of the order of magnitudes of different indicators, which are not suitable for the reliability evaluation of distributed IoT devices in a distribution power grid.
Motivated by the above challenges, we construct a reliability evaluation indicator system of distributed IoT devices in a distribution power grid and propose a FAHP-based reliability evaluation method. First, a three-layer reliability evaluation architecture including a target layer, indicator layer, and state layer is established [13]. Secondly, state-layer elements are classified according to the relationship between them and reliability, and corresponding membership functions are constructed to obtain the scores of state-layer elements. Then, the scores of indicator-layer evaluation indicators can be obtained through establishing the state-layer fuzzy judgment matrix, performing consistency check and consistency transmission, calculating the weights of state-layer elements, and calculating the weighted sum of weights and scores of elements. Similarly, the target-layer score can be obtained, which is utilized to determine the reliability level of distributed IoT devices and complete the reliability evaluation. According to the reliability levels and scores of three layers, the abnormalities of IoT devices can be inferred to provide guidance for maintenance [14]. The main contributions are introduced as follows.
Contribution 1. Multilevel reliability evaluation indicator system. A multilevel reliability evaluation indicator system for distributed IoT devices is proposed. The system includes four categories of first-level indicators and thirty-six second-level indicators, which are selected through the comprehensive consideration of the ability to reflect the actual operation state and the actual characteristics such as the difficulty level of data collection and the calculation complexity.
Contribution 2. FAHP-based weight calculation method. Considering that the importance of each state-layer element for reliability evaluation is constantly changing in the actual distribution power grid scenario, the proposed FAHP-based weight calculation method can dynamically adjust the relative importance of elements through adopting the 0.1-0.9 scaling method to establish the fuzzy judgment matrix, which improves the adaptability of element weight with actual reliability requirements.
The remainder of this paper is organized as follows. Section 2 introduces the reliability evaluation indicator system of distributed IoT devices in a distribution power grid. Section 3 presents FAHP-based reliability evaluation. The simulation results are presented in Section 4. Finally, Section 5 concludes this paper.
2. Reliability Evaluation Indicator System of Distributed IoT Devices in Distribution Power Grid
The reliability of distributed IoT devices in distribution power grid is mainly manifested in the ability of survival, effective communication, and accurate monitoring in a specific application environment and within a specified time period [15]. It also emphasizes the ability to deal with emergencies within a specific range and ensure its stable operation. The reliability evaluation indicator system needs to accurately describe all kinds of indicators affecting the reliability of distributed IoT devices [16]. At the same time, it should also specify the details of the hierarchical relationship and indicator weight of the indicator system [17]. Moreover, the evaluation accuracy of the reliability evaluation system is also directly affected by the selection strategy of evaluation indicators [18, 19]. On the one hand, the selected indicators are required to reflect the actual operation of IoT devices as much as possible, so that important indicators cannot be omitted [20]. On the other hand, the actual characteristics such as the difficulty level of data collection, the effectiveness of information, and the calculation complexity should be considered to improve the evaluation accuracy and reduce the evaluation complexity as much as possible.
There is no unified reliability evaluation standard of distributed IoT devices in a distribution power grid [21–23]. Following the principles of objectivity and comparability, we establish a multilevel reliability indicator system, including four first-level indicators and thirty-six second-level indicators. As shown in Figure 1, the first-level indicators are divided into four categories, i.e., technicality evaluation indicator, energy efficiency evaluation indicator, security evaluation indicator, and operation evaluation indicator. The second-level indicators contained in each first-level indicator are provided as follows: (i)Technicality evaluation indicator. The technicality evaluation indicator contains nine second-level indicators, i.e., node redundancy, channel redundancy, sampling frequency, signal transmission performance, average voltage deviation, quality quantization, device duty cycle, maximum number of retransmissions, and flow benefit(ii)Energy efficiency evaluation indicator. The energy efficiency evaluation indicator contains nine second-level indicators, i.e., power factor, load factor, conductor temperature rise, rated no-load loss, rated load loss, link energy availability, three-phase load imbalance rate, configuration and change management, and medium management(iii)Security evaluation indicator. The security evaluation indicator contains nine second-level indicators, i.e., drift deviation fault, precision degradation fault, complete failure fault, patch security, denial-of-service impact, network attack frequency, channel packet loss ratio, security mechanism perfection, and encrypted transmission(iv)Operation evaluation indicator. The operation evaluation indicator contains nine second-level indicators, i.e., sensor accuracy, applicability of protocol distance, end-to-end delay, real-time received data, service success rate, mean time between failures, node connectivity probability, node capacity, and link capacity
The multilevel reliability evaluation indicator system proposed in this paper can be applied to more evaluation indicator scenarios by adjusting the categories of first-level indicators and second-level indicators.
3. FAHP-Based Reliability Evaluation of Distributed IoT Devices in Distribution Power Grid
3.1. Three-Layer Reliability Evaluation Architecture of Distributed IoT Devices
Based on the reliability evaluation indicator system proposed in Section 2, we establish a three-layer reliability evaluation architecture of distributed IoT devices, including the target layer, indicator layer, and state layer as follows: (i)Target layer. The target layer is defined as the distributed IoT devices considering that the ultimate target is to evaluate the reliability of distributed IoT devices(ii)Indicator layer. The indicator layer is composed of four types of evaluation indicators corresponding to the first-level indicators defined in Section 2, i.e., technicality evaluation indicator, energy efficiency evaluation indicator, security evaluation indicator, and operation evaluation indicator(iii)State layer. The state layer consists of thirty-six elements corresponding to the second-level indicators defined in Section 2. Each type of evaluation indicator in the indicator layer includes nine state-layer elements, and the details are elaborated in Section 2
Based on the established three-layer reliability evaluation architecture, we denote the four types of evaluation indicators in the indicator layer as , respectively. The state-layer elements included in the evaluation indicator type are denoted as . For instance, represents technicality evaluation indicators, and , represent the node redundancy, channel redundancy, sampling frequency, signal transmission performance, average voltage deviation, quality quantization, device duty cycle, maximum number of retransmissions, and flow benefit, respectively.
3.2. State-Layer Scoring
The scoring method for state-layer elements in the three-layer reliability evaluation architecture of distributed IoT devices is illustrated as follows. Membership function is adopted to realize the mapping from the element values to the element scores, where the membership of an element to a certain fuzzy evaluation is calculated based on the element value, and then, the element score is calculated based on fuzzy evaluation scores and memberships. Membership function-based scoring method for state-layer elements can effectively reduce the influence of human subjective factors on the reliability evaluation of distributed IoT devices [24]. Considering the differences in the units and order of magnitudes of state-layer elements, membership functions with different parameters are constructed for different elements to realize unified scoring.
Based on the requirements on the operation state of different distributed IoT devices in the actual distribution power grid, the fuzzy evaluations for the reliability of a state-layer element are defined as poor, medium, and good. The membership functions corresponding to three fuzzy evaluations are denoted as , , and for the state-layer element [25]. Particularly, when the element value helps to improve the reliability of distributed IoT devices, the membership grade to the good fuzzy evaluation will be larger. Considering the relationship between element value and reliability, the state-layer elements are classified to positive correlation elements and negative correlation elements. Specifically, a larger positive correlation element value represents the higher reliability of the distributed sensor devices, and negative correlation elements are the opposite. We propose two membership functions for the above two kinds of elements, which are shown in Figure 2 and introduced as follows:
(a)
(b)
Membership function : the membership function for positive correlation elements.
Membership function : the membership function for negative correlation elements.
Here, represents the actual element value, and . , , and are the parameters to describe the boundaries of poor, medium, and good fuzzy evaluations for element , the values of which are determined by the relationship between reliability and element values [26]. Take the negative correlation element end-to-end delay as an example. We assume that the device is considered to be reliable when the end-to-end delay is lower than ms and unreliable when the end-to-end delay exceeds ms. Then, , , and can be set as ms, ms, and ms, respectively.
The specific procedures of the state-layer element scoring for distributed IoT devices are summarized in the following: (1)Determine the membership function based on the state-layer element types. Specifically, membership function is utilized for positive correlation elements, and membership function is utilized for negative correlation elements(2)According to the selected membership function, calculate the membership grades of the current element value to the good, medium, and poor fuzzy evaluations, i.e., , , and (3)Score the current element value as
where , , and are the scores corresponding to good, medium, and poor fuzzy evaluations, the values of which are determined according to the actual operation environment of the distribution power grid
3.3. Indicator-Layer Scoring
The scoring procedures for the indicator-layer evaluation indicators in the three-layer reliability evaluation architecture of distributed state IoT devices consist of establishment of state-layer fuzzy judgment matrix, consistency check, and consistency transformation, as well as weight calculation of state layer and scoring of indicator layer, which are elaborated as follows.
3.3.1. Establishment of State-Layer Fuzzy Judgment Matrix
The state-layer fuzzy judgment matrix is defined to describe the important relationships between the state-layer elements belonging to a certain evaluation indicator of the indicator layer. Specifically, the state-layer fuzzy judgment matrix for the evaluation indicator is denoted as and is given by
where represents the number of state-layer elements contained in . represents the importance of the element relative to the element in , where .
In this paper, the 0.1-0.9 scaling method is adopted to scale the important relationship . The specific scaling method is shown in Table 1.
For example, when is extremely important compared with , .
3.3.2. Consistency Check and Consistency Transformation
In the process of reliability evaluation of distributed IoT devices, the weight calculation of state-layer elements requires the consistency of the fuzzy judgment matrix. Consistency check and consistency transformation are introduced as follows:
Consistency check: if the fuzzy judgment matrix satisfies the consistency condition, i.e.,
the fuzzy judgment matrix is called fuzzy consistency judgment matrix, which is denoted as .
Consistency transformation: if the consistency condition is not satisfied, perform consistency transformation to transform the fuzzy judgment matrix to the fuzzy consistency judgment matrix , where
3.3.3. Weight Calculation of State Layer and Scoring of Indicator Layer
The weights of state-layer elements are calculated according to the fuzzy consistency judgment matrix obtained by (6), which are integrated with the scores of state-layer elements obtained in Section 3.2 to calculate the scores of evaluation indicators in the indicator layer.
For the state-layer elements contained in the evaluation indicator , the relationship between the weights and the fuzzy consistency judgment matrix is given by
where and represent the weights of state-layer elements and , respectively. is a random parameter which satisfies .
Then, the least square method is adopted to calculate the weights of the state-layer elements. Specifically, the weight of is given by
Finally, based on the weights and scores of state-layer elements, the score of the evaluation indicator is calculated as
3.4. Target-Layer Scoring and Reliability Evaluation
Similar to the indicator-layer scoring, the procedures of target-layer scoring include the establishment of indicator-layer fuzzy judgment matrix, consistency check, and consistency transformation, as well as weight calculation of indicator layer and scoring of the target layer. The reliability evaluation of distributed IoT devices is performed based on the target-layer score. The details are introduced as follows.
3.4.1. Establishment of Indicator-Layer Fuzzy Judgment Matrix
According to the scores of the evaluation indicators obtained in subsection 3.3, establish the indicator-layer fuzzy judgment matrix based on variable weight method to describe the important relationship between the indicator-layer evaluation indicators, which is given by
where , and represents the number of evaluation indicators.
3.4.2. Consistency Check and Consistency Transformation
Perform consistency check on based on (5). If does not satisfy the consistency condition, transform to the indicator-layer fuzzy consistency judgment matrix , where
3.4.3. Weight Calculation of Indicator Layer and Scoring of Target Layer
Similar to (8), the weight of the evaluation indicator can be calculated as where is a random parameter which satisfies .
Based on the weights and scores of indicator-layer evaluation indicators, the target-layer score is calculated as
3.4.4. Reliability Evaluation
The reliability level is determined based on the target-layer score to achieve the reliability evaluation of distributed IoT devices. According to the characteristics of the operation environment of the distribution power grid, the reliability levels include excellent, good, medium, and poor [27]. The corresponding relationship between the target-layer score and the reliability levels is shown in Table 2. In other practical application scenarios with higher requirements on the reliability of distributed IoT devices, the reliability levels can be further finely divided. The on-site inspectors can judge the reliability levels of the distributed IoT devices according to the final target-layer score [28]. In addition, for the devices with poor reliability level, the state-layer and indicator-layer scores can also provide maintenance guidance for on-site inspectors.
3.5. Implementation Process of FAHP-Based Reliability Evaluation
The implementation process of FAHP-based reliability evaluation of distributed IoT devices is shown in Figure 3 and is introduced as follows: (i)Step 1. Establish the three-layer reliability evaluation architecture of distributed IoT devices, and determine the composition of each layer(ii)Step 2. Select different membership functions for different types of state-layer elements. Collect the current operation data of state-layer elements, calculate the membership grades to different fuzzy evaluations, and calculate scores of the state-layer elements based on (3)(iii)Step 3. Establish a state-layer fuzzy judgment matrix based on the 0.1-0.9 scaling method(iv)Step 4. Perform consistency check and consistency transformation based on (5) and (6) to obtain the state-layer fuzzy consistency judgment matrix(v)Step 5. Calculate the weights of state-layer elements based on (8), and calculate the score of the indicator-layer evaluation indicators based on (9)(vi)Step 6. Judge whether all evaluation indicators of the indicator layer have been scored. If not, return to Step 2 to calculate the next evaluation indicator. If yes, proceed to the next step(vii)Step 7. Establish indicator-layer fuzzy judgment matrix based on (10), and carry out consistency check and consistency transformation to obtain the indicator-layer fuzzy consistency judgment matrix(viii)Step 8. Calculate the weights of evaluation indicators based on (12), and calculate the target-layer score based on (13)(ix)Step 9. Determine the reliability levels and complete the reliability evaluation of distributed IoT devices based on Table 2. According to the reliability levels and scores of three layers, analyze the operation state and abnormal faults, and provide guidance for on-site maintenance and scheduling
4. Simulation Results
To verify the accuracy and practicability of the proposed FAHP-based reliability evaluation method, the on-site operation data of an IoT device in the distribution power grid of Shandong Province, China, are collected to perform reliability evaluation.
4.1. Simulation Verification
In this subsection, we introduce the simulation verification process of the FAHP-based reliability evaluation method. Due to the space limitation, we only take the technicality evaluation indicator and its contained state-layer elements as an example.
4.1.1. Membership Function Selection and State-Layer Scoring
Judge the type of each state-layer element of the technicality evaluation indicator and select the corresponding membership function. For instance, the signal transmission performance is directly proportional to the reliability, which is a positive correlation element, and thus, membership function 1 is selected. The node redundancy is inversely proportional to the reliability, which is a negative correlation element, and thus, membership function 2 is selected. The boundary parameters are set based on the characteristics of each element [29]. Then, the scores of state-layer elements of technicality evaluation indicator can be obtained based on (3), as shown in Table 3.
4.1.2. State-Layer Fuzzy Judgment Matrix of Technicality Evaluation Indicator
Based on the 0.1-0.9 scaling method, the state-layer fuzzy judgment matrix of is given by
According to (5), does not meet the consistency condition. Therefore, it is transformed into a fuzzy consistency judgment matrix based on (6), which is given by
4.1.3. Weights of State-Layer Elements of Technicality Evaluation Indicator
The weights of state-layer elements of technicality evaluation indicator are calculated based on (8). The results are shown in Table 4.
4.1.4. Scores of Evaluation Indicators
The score of the technicality evaluation indicator can be calculated based on (9), which is given by .
Similarly, the scores of the energy efficiency evaluation indicator, security evaluation indicator, and operation evaluation indicator can be calculated. The specific results are shown in Table 5.
4.1.5. Reliability Evaluation
The indicator-layer fuzzy judgment matrix is given by
According to (5), does not meet the consistency condition. Therefore, it is transformed into a fuzzy consistency judgment matrix based on (6), which is given by
Calculate the weights of evaluation indicators based on (12). The results are shown in Table 6.
Then, the target-layer score is calculated as . According to Table 2, the reliability level of this distributed IoT device is excellent. However, the scores of the energy efficiency evaluation indicator and security evaluation indicator are relatively low, which indicate that the device may be abnormal. Through maintenance, it is found that the three-phase load of this device is unbalanced, and a small probability of drift deviation exists, which is consistent with the reliability evaluation results and proves the accuracy of the proposed method.
4.2. Simulation Comparison
In this section, the on-site operation data of ten distributed IoT devices in the distribution power grid of Shandong Province, China, are collected. We perform reliability evaluation on these ten devices and compare the evaluation results with the actual reliability value to verify the effectiveness. In addition, the proposed method is compared with two evaluation methods, i.e., principal component analysis (PCA) [30] and analytic hierarchy process (AHP) to verify the accuracy [31]. PCA performs correlation analysis on the indicators to obtain comprehensive indicators and complete reliability evaluation. AHP establishes a judgment matrix based on qualitative analysis to calculate the indicator weights and conduct reliability evaluation.
4.2.1. Reliability Evaluation of Distributed IoT Devices
Figure 4 shows the actual reliability values and the evaluation values, i.e., target-layer scores, of ten distributed IoT devices. It can be seen that the proposed method can achieve accurate reliability evaluation. The difference between the evaluation values and actual values is small, and the obtained reliability levels based on the evaluation values are accurate according to Table 2. The evaluation results indicate the superiority of the proposed method in the processing of indicators, which can effectively reduce the influence of the evaluator’s subjective factors and objectively and truly reflect the reliability of the distributed IoT devices.
4.2.2. Accuracy Comparison
Figure 5 shows the reliability evaluation accuracy of different evaluation methods. Compared with PCA and AHP, FAHP can increase the evaluation accuracy by 9.86% and 6.96%, respectively. The reason is that FAHP utilizes membership functions to obtain the scores of state-layer elements, which avoids the influence of subjective factors caused by artificial scoring. In addition, FAHP can flexibly adjust the boundary parameters of membership functions to adapt to different state-layer elements, different operation requirements, and different environments. AHP cannot get rid of the influence of subjective factors, resulting in a lower evaluation accuracy. The accuracy of PCA is the lowest because PCA can hardly distinguish the difference among multiple indicators.
5. Conclusions
In this paper, we addressed the reliability evaluation problem for distributed IoT devices in the distribution power grid and proposed the FAHP-based reliability evaluation method to overcome the influence of subjective factors and realize accurate reliability evaluation. Compared with PCA and AHP, simulation results indicate that the proposed FAHP-based method increases the evaluation accuracy by 9.86% and 6.96%, respectively. In the future work, we will further consider establishing more accurate membership functions based on the refinement characteristics of state-layer elements.
Data Availability
The [data type] data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported by the Science and Technology Project of State Grid Shandong Electric Power Company under grant number 2020A-010.