Abstract

This study proposes a large-scale group decision-making (LSGDM) consensus model considering the experts’ adjustment willingness based on the interactive weights’ determination, which can be applied to an LSGDM problem through a case of earthquake shelters. The main contributions of our research are of three aspects as follows. First, the determination method of the interactive weight, which obtains the DMs’ attitude towards the decision-making results, is presented. The subgroups’ weights are calculated, and the unit adjustment cost for each DM is defined. Second, we introduce an objective consensus threshold by the mean and variance of the consensus level for each subgroup. Subsequently, an identification rule is performed to determine the DM to be adjusted with the large difference and the low adjustment cost. And we developed a novel consensus adjustment method that takes the DMs’ adjustment willingness into account to retain as much original information as possible. Thirdly, in order to reduce the subjectivity of the preset consensus threshold and the maximum number of iterations, an objective consensus termination condition that combines the current group consensus level and the consensus adjustment rate is put forward. Finally, the proposed model has demonstrated its effectiveness and superiority based on the comparative and sensitive analysis through a practical example.

1. Introduction

Group decision-making (GDM) is a process in which, under certain constraints, some experts or decision-makers (DMs) obtain the optimal from several feasible alternatives by expressing their opinions or preferences [1]. With the development of social, science, and technology, the complexity of decision-making events has becoming increasingly high [1], the ambiguity and uncertainty of the decision-making environment and context have become increasingly high, and the number and diversity of DMs participating in decision-making issues have increased rapidly. GDM has developed into large-scale group decision-making (LSGDM) [26], multiattribute LSGDM [710] and so on. Compared with GDM, the number of experts and DMs involved in LSGDM is larger, usually more than 20 [4, 9, 11, 12]. The differences in DMs’ backgrounds and knowledge are greater, and thereby the consensus level among DMs is lower.

In recent years, LSGDM and fuzzy mathematics, game theory, computers, information technology, and other theories are being integrated and developed. The research of multiattribute LSGDM mainly focuses on the expression of DMs’ or experts’ preference information [13, 14], clustering [12], aggregation of group preference information [2, 1316], determination of weight [3, 17, 18], and consensus reaching process (CRP) [2, 3, 9, 14, 19]. Many methods of expression of experts’ preference information have proposed, such as fuzzy preference information, linguistic preference information, and random preference information. Liu et al. [20] transformed interval-valued intuitionistic fuzzy numbers into single-valued numbers and then proposed a two-stage regularized generalized canonical correlation analysis decision-making method based on multiblock analysis to address the MALSGDM problem in the interval-valued intuitionistic fuzzy environment. Bai et al. [11] developed an LSGDM model with cooperative behavior based on social network analysis, considering the propagation of decision-makers’ preferences by considering the propagation of DMs’ preferences. Zhen et al. [21] proposed a computational model based on the use of extended linguistic hierarchies and used multigranularity linguistic distribution to provide interpretable aggregate linguistic results to experts in order to maximize information retention.

To reduce the dimension and complexity of the decision-making process, many clustering methods have been proposed and applied, such as the k-means clustering method [4, 12], fuzzy c-means clustering method [22], vector space-based clustering method [23], the and transitive closure clustering method [5]. Several researchers have proposed some novel clustering methods that can be used in LSGDM problems from different perspectives. For instance, Du et al. [6] developed a new clustering method considering both trust relationships and opinion similarity in a social network context. In this study, the k-means clustering method is utilized to reduce the dimension of the LSGDM problem.

However, despite a number of LSGDM methods having been proposed, these methods are used in specific situations. For instance, in Liu et al. [24]’s research, DMs cannot give complete and accurate evaluation information at once, and it is therefore difficult for decision-making groups to reach a consensus at once. In addition, some proposed LSGDM models include the determination process of the DMs’ weight, but the weight of the DM is relatively simple, that is, the dynamics of the decision-making process are not taken into account. Next, CRP, a rather critical and essential process, reduces and even eliminates the conflicts of group and further improves the effectiveness and rationality of decision results. But in the existing literature, the threshold determination of many CRP is very subjective, which is not conducive to the objectivity of decision-making results. Therefore, our research intends to answer the following questions:(a)How can we not only ensure that more DMs participate in the decision-making process but also make the results represent the opinions and attitudes of more DMs?(b)How to obtain a more scientific and accurate consensus threshold?(c)Under what conditions can the consensus adjustment process terminate automatically?

Based on the above analysis, an LSGDM consensus model considering the experts’ adjustment willingness based on the interactive weights is proposed in this study. Its innovations and contributions are shown as follows:(1)A novel method of weight determination has been developed, which considers the DMs’ attitude towards the decision-making results, thereby ensuring the effective participation of DMs. Moreover, to improve the rationality of LSGDM, a harmonious degree is taken into account in the calculation of subgroups’ weight.(2)In the consensus measure process, a more reasonable consensus measure method is introduced, which considers both the differences between the DMs’ opinion and the group opinion and the harmonious degree. In the consensus feedback process, an identification process is presented, which considers the unit adjustment cost for each DM. Then, a new adjustment process is constructed, which the evaluation information is less distorted or lost by considering the experts’ adjustment willingness.(3)An objective calculation method of the consensus threshold is presented from the perspective of the mean and variance of the consensus level, and then a termination condition that considers both the current consensus level and the consensus adjustment rate is designed to objectively terminate the CRP. It not only compares the current consensus level with the consensus threshold but also compares the consensus adjustment rate. As a result, this method can address the subjectivity and unreasonableness of the preset consensus threshold and the maximum number of iterations to a certain extent.

The remainder of this paper is organized as follows: In Section 2, we briefly summarize the recent literature review from the following four aspects: the aggregation of group preference information, the interactive process, the weight determination method, and the CRP. Section 3 introduces the multiattribute LSGDM problems, and then the k-means clustering method is used to classify the DMs into several subgroups. The determination methods of the DMs, the attributes, the subgroups, and the unit adjustment cost are shown in Section 4. Section 5 presents the proposed consensus model, which includes the consensus measurement process and the consensus feedback process and explains how its framework is conducted. In Section 6, the proposed model is applied to a case study to illustrate its effectiveness and rationality. The comparative analysis and the sensitivity analysis are performed in Sections 7 and 8, respectively, to further validate the proposed model. We draw the conclusions of this research in Section 9.

2. Literature Review

In this study, we summarize the existing literature from the following four perspectives: the aggregation of group preference information, the interactive process, the weight determination method, and the CRP.

2.1. Research on the Aggregation of Group Preference Information

Many scholars are interested in the aggregation of group preference information, which is mainly to obtain the results of group clustering. Xu et al. [19] presented a two-stage method to support the CRP. The first stage classifies and obtains several subgroups by utilizing the self-organizing maps, and then an iterative algorithm is proposed to obtain the group preference for each subgroup. The second stage treats the group preference of each subcluster as the representative preference and collapses each subcluster to form a smaller and more manageable group. Liu et al. [25, 26] utilized the idea of principal component analysis (PCA), regarded attributes and decision makers as interval-valued intuitionistic fuzzy variables, transformed them into several independent variables, and then combined them with the traditional preference aggregation operator to obtain a decision-making method to solve the complex multiattribute LSGDM problems. Chen et al. [27] developed a two-tier collective opinion generation framework integrating professional knowledge structure and risk preference to generate collective preference assessment, and thereby to obtain an accurate and reliable alternative ranking. In this paper, we utilize weighted arithmetic averaging (WAA) to aggregate each DM’s opinion.

2.2. Studies about the Interactive Process

In reality, DMs often cannot give complete and accurate evaluation information at once, it is difficult for decision-making groups to reach a consensus at once, and thereby, the evaluation information needs to be continuously supplemented and adjusted. Therefore, the interactive process is not only necessary but significant to avoid the limitations of the DMs’ opinions, improve the effectiveness of the DMs’ participation, and ensure the rationality of the decision-making results.

Zeng et al. [28] developed an interactive procedure for GDM based on intuitionistic fuzzy preference relations, in which the similarity measures between the collective preference relation and the intuitionistic fuzzy ideal solution are used to rank the alternatives. Liao and Xu [29] established an optimization model for determining the weight and an interactive model of a multiattribute decision-making problem with hesitant fuzzy information to make the decision more reasonable. Ding et al. [30] proposed an interactive method to deal with the probability hesitation fuzzy multiattribute GDM problem with incomplete attribute weight information, which can reflect the DMs’ subjective desirability and reduce the effect of unfair arguments on the decision results. Therefore, in our research, the interactive process is utilized for the experts’ weights determination to improve the rationality of the decision-making results.

2.3. The Aspects of the Weight Determination

The determination of weight is a hot issue in LSGDM problems, which includes mainly the weights of experts’ or DMs’, the weights of the attributes of the alternatives, and the weights of the subgroups. For the determination method of experts or DMs, Meng et al. [14] integrated objectively cooperation networks and references network of DMs to construct a directed and weighted social network, and then obtained the DMs’ relative weights. Wan et al. [31] show that DM’s weight can be obtained through a programming model by minimizing the distance between individual semantics and collective. Liu et al. [17] proposed a double weight determination method of experts by utilizing mathematical programming and information entropy for multiattribute LSGDM in an interval-valued intuitionistic fuzzy environment. Wan et al. [32] developed a similarity determination method to calculate the weight for each DM and constructed two programming models to obtain the optimal weight for each attribute.

Related to the studies of the attributes’ weight, Zhong et al. [4] developed an approach to determine the attributes and their weights based on the social media data relevant to decision-making problems by using the term frequency-inverse document frequency (TF-IDF) method. It considers both the experts’ opinions and the views of stakeholders. In this study, the interactive process is introduced in the determination process of the DMs’ weights. For the subgroups’ weights, Liu et al. [8] set an equal weight for different subgroups, while Xu et al.[1] determine the subgroups’ weights according to the size of the subgroup.

In this study, by considering the DMs’ attitude towards the decision-making results (i.e., satisfaction degree), the DMs’ weights are updated and obtained, and then the attributes’ weights are obtained within the subgroups. The weights of the subgroups are calculated and updated by considering the number within the subgroup and the level of the subgroup’s satisfaction degree (i.e., harmonious degree).

2.4. CRP Studies

CRP is a rather critical and essential segment in LSGDM problems [33, 34], which is reducing and eliminating the conflict of a group and improving the effectiveness and rationality of decision results. Zhong et al. [4] presented a multistage hybrid consensus-achieving model by integrating both cardinal consensus and ordinal consensus and applied it in the scene of the selection of a hotel for the centralized isolation of entry personnel during the COVID-19 epidemic. For the consensus feedback process in the CRP, experts may not tolerate their opinions being modified unrestricted during the CRP. Hence, all experts have an accepted modification for their opinions, which can be presented as the adjustment willingness. However, few studies have focused on the experts’ adjustment willingness. Zhong et al. [5] proposed a nonthreshold consensus model, which includes an objective termination condition for CRP. It can reduce the subjectivity of the predefined consensus threshold and the maximum number of iterations to a certain extent. In addition, Wan et al. [35] developed a novel two-stage CRP method considering DM’s willingness to modify preference information.

Therefore, our research presents an objective calculation method of the consensus threshold from the perspective of the mean and variance of the consensus level and then develops a termination condition that considers both the current consensus level and the consensus adjustment rate to objectively terminate the CRP.

2.5. Research on the Case Application

Many scholars have proposed many LSGDM approaches from the perspective of practice and application. Xiao et al. [36] established the civil engineering construction contractor selection framework in the LSGDM environment by considering the interaction within and between the management layers of the consensus model. Chen et al. [37] determined passenger demands and evaluated their satisfaction by using a combination of online review analysis and LSGDM based on a case study of a high-speed rail system in China.

3. Preliminaries

3.1. The Multiattribute LSGDM Problems Description

LSGDM is the process of selecting the best option from the opinion of many DMs, who express their opinion based on the decision-making information provided for alternatives [11]. Accordingly, let X = {x1, …, xp, …, xP}(P ≥ 2) be the set of alternatives, E = {e1, …, em,…, eM}(M ≥ 20) be the set of experts and DMs, and F = {f1, …, fn, …, fN}(N ≥ 2) be the set of attributes for each alternative.

First, the DM em provides his or her evaluation information Qm = ()P × N(m = 1, …, M), where represents the evaluation value of the attribute fn on the alternative xp for the DM em. Then, the DM em provides his or her allowed modification values and , which represents, respectively, the DM em is allowed to modify the positive and negative range of the evaluation information they provide. It is noting that the allowed modification values represent the DMs’ adjustment willingness and the values of and are both positive. For instance, a DM em provides the values of , and , the modification value of can be more acceptable in the interval [max(, 0), min(, 1)]. Note that the value of should be in the range [0, 1] before and after adjustment. The greater the value of  + , the lower the difficulty of adjusting the evaluation information, and the higher the concession degree of the DM em in order to reach group consensus. However, if the adjustment value exceeds the allowed modification range of the DM, it will pay an enormous adjustment cost, and the evaluation information of the DM will be forced to change, resulting in information distortion. Therefore, this situation is not considered in this paper.

In this paper, the weight vectors of attribute for each alternative are denoted as wm = [, …, , …, ], where represents a weight value of the attribute fn that the DM em provided according to his or her knowledge and experience, 0 ≤  ≤ 1, and . The set of the DMs’ weights is denoted as Wt = {m = 1, …, M}, where means the weight value of the DM em participating in the subgroup given at the t-th stage. Clearly, W1 is the initial set of the DMs’ weights.

Generally, the LSGDM process usually involves the following four stages:Stage 1. Clustering. In order to reduce the complexity of the LSGDM problem and the calculation process, a clustering method is generally utilized to divide all DMs to several subgroups according to some rules. In this study, the k-means clustering method is utilized according to the opinion similarity. The details are shown in Section 3.2.Stage 2. Aggregate the opinion. The weighted arithmetic averaging (WAA) operator is usually used to aggregate each DM’s evaluation information to a subgroup’s decision matrix and each subgroup’s decision matrix into the collective decision matrix [38]. For a LSGDM problem, suppose that is the em’s weight in the subgroup Gk, is the Gk’s weight. Then, the aggregation process can be derived aswhere meets 0 ≤  ≤ 1 and  = 1.where meets 0 ≤  ≤ 1 and  = 1.Stage 3. The CRP. Due to the large number and the complex background of DMs in the subgroup, the consensus level is lower. It aims to obtain an acceptable consensus level. If the consensus is not reached, then the consensus feedback process should be executed. The details are shown in Section 5.Stage 4. Selection process. After obtaining the collective decision matrix by equation (2), the collective opinion score s(xp) for each alternative xp is derived aswhere  ∈ [0, 1] is the weight of attribute fn for the group, and  = 1. ’s calculation equation is shown in Section 4.2 by equation (22).

3.2. The DMs Clustering Process

In this study, all DMs are classified as K subgroups by using k-means clustering method. The algorithm is given as Algorithm 1.

Input: The value of Qm for each DM and the number of subgroups K.
Output: The clustering results G1, G2, …, GK.
Step 1. Each evaluation information matrix Qm (m = 1, …, M) is transformed into one-dimensional vector. For instance, the transformed vectors are denoted as (m = 1, …, M),
Step 2. Set or Choose randomly K vectors as the initial cluster centers {, …, }, where
Step 3. Let each DM enter to the subgroup closest to him or her. In other words, calculate the distance d(ei, ) between the DM ei and the k-th cluster center. d(ei, ) can use the Euclidean distance given by
Then, we can obtain  = d(ei, ), and the DM ei therefore should enter to the subgroup .
Step 4. Recompute the cluster results by using the member information of the current subgroups. Suppose that the new cluster centers are {, …, }, where
Step 5. Set the boundary conditions of clustering process. Setting δ > 0, and the total differences before and after adjusting clustering are called TD, where
Step 6. The judgment process. If the condition TD < δ is satisfied, the clustering process is over, and then go to the next step; Otherwise, go to Step 4.
Step 7. Output the cluster results G1, G2, …, GK. End.

In this paper, K meets the condition 2 ≤ K ≤ . The number of the DMs is denoted as in the subgroup Gk.

4. Several Important Concepts in This Study

4.1. The Determination of the DMs’ Weights and the Attributes’ Weights

There are many factors that affect the determination of the DMs’ weights, such as the subjective degree of the DMs’ opinion, DMs’ attitude towards the overall or intragroup opinion, and so on. Apparently, the higher the subjective degree of the DM’s opinion or preference, the lower the objectivity of the DM’s decision results, the smaller the influence of the DM on the final results of decision-making, and the DM’s weight value therefore should be decreased to some extent. Again, the higher the DM’s attitude towards the overall or intragroup opinion or preference, the more satisfied the DM is with the clustering results, the greater the DM’s recognition of the results of group decision-making, and the DM’s weight value therefore should be enhanced to a certain extent. In this study, the determination methods of the DMs’ weights and the attributes’ weights are developed in Algorithm 2.

Input: The values of , , , and , and the subgroups G1, G2, …, GK.
Output: The final values of the DMs’ weights and the attributes’ weights.
Step 1. Calculate the evaluation information for each subgroup, k = 1, …, K, where
Step 2. Compute the evaluation information differences between each DM and his or her subgroup by using equation (4). For instance, the difference between the DM em and the subgroup Gk is
Step 3. The initial weight values for each DM are given. Note that the evaluation difference of the DM and his or her subgroup is an important basis for determining the initial value of the DM’s weight. The higher the difference between them, the lower the initial value of the DM’s weight. Conversely, if the DM’s opinion or preference are closer his or her subgroup, a higher weight should be given initially. Therefore, the determination of the initial weight value is given as follows.
Step 4. Calculate the weighted attributes’ weight values for each subgroup. The calculation formula is
Step 5. Calculate the distances of weight value between each DM and his or her subgroup. For instance, if the distance dH(em, Gk) between the DM em and the subgroup Gk is larger, the weight value of the DM em in the subgroup Gk should be decreased to a certain extent. On the contrary, the value of dH(em, Gk) is lower, it indicates that the subjective attitude of the DM em is closer to the collective attitude of the subgroup Gk, the weight value of the DM em in the subgroup Gk should be improved to some extent. dH(em, Gk) can utilize the Hamming distance given by
Step 6. Updating the weight value for each DM. The updating formula is:
where represents the importance of adjusting DMs’ weight for each time, the higher the value of , the larger the importance of updating DMs’ weight in this time, and is the deviation proportion of the DM em in the subgroup Gk, and the compute formula is shown as follows.
Step 7. Computing the proportion of the allowed modification range, and update the weight value for each DM. The greater the proportion of the allowed modification range  + , the more concessions the DM makes in order to obtain the subgroup consensus, and the larger the proportion should be given when adjusting the DMs’ weight. The calculation and modification equations are performed so that
where represents the proportion of the allowed modification value of the DM em in the subgroup Gk, and meets condition 0 ≤  ≤ 1. The normalization process should be then carried out.
Step 8. Recalculate the weighted attributes’ weight values for each subgroup by using equation (9), and the calculation results are presented that
Step 9. Show the weight values of the aggregated attributes in this time to each DM, and then obtain the satisfaction degree provided by em. The values of provided by em are chosen from the set {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}, which means {extremely dissatisfied, very dissatisfied, dissatisfied, slightly dissatisfied, average, slightly satisfied, satisfied, very satisfied, extremely satisfied}.
Step 10. Re-update the weight value for each DM through the satisfaction degree . The greater the values of , the more satisfied the DM em is with the current weight value, the higher the enthusiasm for the current decision results, and the weight value of the DM em in the subgroup Gk should be improved to some extent. Conversely, if the value of is smaller, the weight value of the DM em in the subgroup Gk should be reduced appropriately. The updating formula of the DMs’ weights is that:
The weight value in this time is the final weights for each DM. The final weight value of the DM em in the subgroup Gk is denoted as , that is,
Step 11. Compute the aggregated attributes’ weight values for each subgroup according to the DMs’ final weight values by using equation (9), that is,
The weight value in this time is the final attributes’ weights. The final weight value for each attribute in the subgroup Gk is denoted as , that is,
Step 12. Output the values of and . End.

It is noted that the purpose of the Step 10 is to obtain the attitude of DMs, i.e., the satisfaction degree, and then to adjust the weights of DMs according to it. The decision information is fed back to the DMs, and then their attitude towards the decision result is obtained. The step, which is necessary and significant, cannot only improve the participation of DMs in the decision-making process and better reflect the attitude of DMs, but make the decision-making results more scientific and reasonable.

4.2. The Weight Determination for Each SubGroup

There are several factors that affect the weight value of the subgroup, such as the number of the subgroup and the harmonious degree of the subgroup. If the subgroup has more DMs, the higher weight value should be given to the subgroup. Conversely, the fewer xDMs in the subgroup, the lower the weight value of the subgroup. Moreover, the greater the harmonious degree of the subgroup, the higher the support of the DMs in the subgroup for the decision results, the larger the satisfaction degree of the DMs in the subgroup with the clustering, and the weight value of the subgroup should be increased appropriately. The algorithm for the weight determination of the subgroup has been developed in Algorithm 3.

Input: The cluster results G1, G2, …, GK, and the satisfaction degree for each DM.
Output: The final weight value for each subgroup.
Step 1. Calculate the initial weight for each subgroup according to the number of the DMs in the subgroup. Obviously, the larger the number of the subgroup, the higher the weight value. The calculation equation is that:
where represents the initial weight value of the subgroup Gk, and satisfies 0 ≤  ≤ 1, and  = 1.
Step 2. Compute the mean value and variance of satisfaction degree for each subgroup. The mean value and variance of satisfaction degree of the subgroup is denoted as and , respectively, and the calculation formulas are shown as follows. ,
Step 3. Define and calculate the harmonious degree for each subgroup. Apparently, the harmonious degree is related not only to the mean value of the satisfaction degree but also to the variance in a subgroup. The calculation equation of the harmonious degree of the subgroup Gk is presented as follows.
 The properties of the harmonious degree is introduced as follows for the subgroup Gk:
(a) The value of meets condition 0 ≤  ≤1.
(b) =  when  = 0.
(c) ≤  at any time.
(d) It increases monotonically for , and decreases for .
Step 4. Update the weight for each subgroup according to the harmonious degree. Obviously, the greater the harmonious degree, and the weight of the subgroup should be improved to some extent. The updating formula is performed as follows.
where represents the importance of harmonious degree in the updating weight process. The value of is considered as the final weight value for each subgroup.
Step 5. Output ,…, . End.

According to the weight of each subgroup and the weight of the attribute for each subgroup, the collective attribute weight vector wG = [, …, ] is derived as

4.3. The Determination Method of the Unit Adjustment Cost

In the existing studies, the unit adjustment cost for each DM sometimes is given in advance [3941], while others is calculated according to some rules [42]. For instance, Labella et al. [42] developed an objective metric based on the cost of modifying experts’ opinions to evaluate CRPs in GDM problems, which is based on two novel minimum cost consensus (MCC) models that consider the distance of the DMs to the collective opinion and also ensure the minimum consistency among DMs.

However, in reality, the unit adjustment cost for each DM is many factors involved. DM may have different perspective for the same problem under the different context. Moreover, DM may have different expectation for the LSGDM result, and then have different attitude. Therefore, the unit adjustment cost must be related to the DMs’ individual characteristics. In this study, we determine the unit adjustment cost of each DM according to two factors: the DMs’ adjustment willingness and and the satisfaction degree . For the and , the larger the value of + , the higher the concession degree of the DM em in order to reach group consensus, the lower the difficulty of adjusting the evaluation information, the unit adjustment cost of em should be smaller. On the contrary, if the lower the value of  + , the unit adjustment cost of em should be larger. For the , the larger the value of , the more satisfied the DM em is with the current decision-making result, the less the difficulty of adjusting the em’s evaluation information to the subgroup collective decision result, and the unit adjustment cost of em should be smaller. Conversely, if the value of is smaller, the unit adjustment cost of em should be higher.

As aforementioned, in this paper, we define the unit adjustment cost for each DM as follows.

Definition 1. Suppose that the unit adjustment cost of em is cm, the allowed modification values of em are and , and the satisfaction degree of em is .where is a scale factor and represents the proportion of the allowed modification range. Apparently, cm increases monotonically with respect to (), and decreases with respect to .

Theorem 1. The value of cm is in the interval of [0, 1].
The proof of Theorem 1 is shown in Appendix A.

5. The Proposed Consensus Model

The CRP mainly includes two parts: consensus measure and consensus feedback. The details of the method of consensus measure are shown in Section 5.1, and Section 5.2 presents the consensus feedback process.

5.1. Consensus Measure

This procedure aims to judge whether an acceptable consensus level among group is reached or not. Clearly, in reality, the collective consensus level is related not only to the differences between the DMs’ and the subgroups’ opinion but to the attitude for each DM in a subgroup, i.e. the harmonious degree of a subgroup. Therefore, we define the consensus measure method as follows.

Definition 2. Suppose that Qm = ()P × N is the evaluation information of the DM em,  = ()P × N is the evaluation information of the subgroup Gk obtained by equation (1), and represents the harmonious degree for the subgroup Gk by equation (19). The differences between the DM em’s and the subgroup Gk’s opinion can be derived aswhere the greater the value of  ∈ [0, 1], the larger the deviation between em and Gk. And, the differences within the subgroup Gk’s opinion can be presented aswhere the larger the value of  ∈ [0, 1], the lower the opinion similarity in the subgroup Gk. The consensus level CLk in a subgroup Gk is derived asAccordingly, the group consensus level GCL can be calculated bywhere GCL meets 0 ≤ GCL ≤ 1. Obviously, the greater the value of GCL, the higher the consensus level between the group. If GCL = 1, it means that a complete consensus has been reached, however, is almost impossible. Hence, soft consensus is generally a rule for LSGDM problems [4345].
 ∈ [0, 1] is usually a preset consensus threshold. If GCL ≥ , then it means that an acceptable consensus level has been reached among the group. Otherwise, consensus feedback is an imperative process to improve the consensus level, and let GCL0 = GCL. Consequently, the determination method of is important for consensus process. Generally speaking, the consensus threshold is composed of the level of each individual, which indicates that the overall threshold must reflect the willingness of the individual. Therefore, without losing generality, the value of is obtained from the perspective of mean and variance.

Definition 3. Suppose that and are the reasonable thresholds, the calculation process is shown aswhere  =  CLk and  =  represent the mean and standard deviation of the consensus level CLk(k = 1, …, K), respectively. We define that the consensus threshold is the average value of and , namely,The determination method of the consensus threshold is to make the setting of the consensus threshold more objective from the perspective of the current consensus level. In reality, the consensus level is related to the experts’ evaluation information, which means the consensus level is uncertainty in the group. Therefore, to reduce the chance, the calculation of the consensus threshold is by weighting two reasonable thresholds in this study, which can reduce the subjectivity of the pre-defined it to a certain extent.

5.2. Consensus Feedback

Consensus feedback aims to obtain a high-consensus level in the group. Generally, the feedback process includes two parts: identification and adjustment.

5.2.1. Identification Process

It aims to determine the subgroup and the DM to be adjusted, which have the maximum differences. First, the determination of the subgroup to be adjusted is necessary. Due to the consensus level involves the difference between opinions and harmonious degree and harmonious degree does not change, the determination method of the adjusted subgroup focus on the difference between opinions, and the equation is  = max{}(k = 1, …, K). Then, the DM em to be modified is obtained in the subgroup Gk by the cm = min{em ∈ Gk}. Therefore, the DM em of the subgroup Gk should be modified in the adjustment process. It should be noted that a DM can be adjusted at most once in the feedback process.

5.2.2. Adjustment Process

This process goals to modified the DMs’ evaluation information according to some advices. Most of existing studies generally are given on adjustment strategies based on mathematical analysis. In this study, the DMs’ evaluation information is modified according to both the subgroup’s opinion and the DM’s adjustment willingness. Detailed procedure is given in Algorithm 4.

Input: the values of , , , and .
Output: the adjustment values of , and .
Step 1. Obtain the evaluation values to be modified of em for the attribute fn of the alternative xp. Hence, the difference matrix P × N is defined as:
Step 2. If  = 0, it means that the evaluation values is same as the subgroup Gk’s opinion, and the em’s evaluation value for the attribute fn of the alternative xp does not need to be modified. Then, turn to the Step 5; otherwise, go to the next step.
Step 3. is denoted as the modified value. Considering both the subgroup Gk’s opinion and the DM ems allowed modification range, the modification calculation is given as follows.
(a) If  > 0, it means that the evaluation values is higher than the subgroup Gk’s opinion. The evaluation values therefore should be reduced appropriately. Then,
(b) < 0, it means that the evaluation values is lower than the subgroup Gk’s opinion. The evaluation values therefore should be increased to a certain extent. Thus,
Step 4. Repeat Step 3 until the values cannot be modified.
Step 5. End. Let is equal to , and calculate the value of by equation (1). Output and .

The CRP is an iterative process that should be terminated by a condition. For the existing consensus model, a consensus threshold and the maximum iterative number generally should be set subjectively in advance [2, 4, 9, 3942, 4446]. In this study, we refer to the concept of consensus improvement rate proposed by Zhong et al. [5], the consensus adjustment rate is defined to judge whether the adjustment process is terminated. The detailed procedure is performed as follows.

We suppose that in the t-th stage, the consensus adjustment cost ACt is defined as

Then, the total adjustment cost TACt before the t-th stage can be derived as

Definition 4. (see [5]).Suppose that consensus adjustment rate is denoted as CARt before the t-th stage, which considers both the adjustment cost and the consensus improvement rate. The equation can be given as follows:where GCLt represents the group consensus level after adjusting the t-th round. If CARt ≤ CARt + 1, it means the consensus adjustment rate is well, and then the CRP can be proceed; otherwise, the CRP should be terminated, and the (t + 1)-th adjustment should be restored. Thus, the t-th adjustment is the last adjustment. To compare the termination condition, i.e., consensus adjustment rate CARt, the consensus feedback process for at least one round.

5.3. The Framework of the Proposed Consensus Model

As the above aforementioned, the procedure of the proposed consensus model can be summarized as follows, and the framework can be shown in Figure 1 (Algorithm 5).

6. Case Study

6.1. Case Background

Earthquake is a natural phenomenon. According to statistics, the number of earthquakes in China from 2010 to 2020 was 6029, of which 48 were above magnitude 6.0. The earthquake will not only cause huge losses to the regional economy, but also cause huge casualties. Therefore, it is very critical to prepare before the earthquake to reduce casualties.

Tangshan, in Hebei Province of China, is an earthquake prone area. A county in this city plans to build several earthquake shelters. However, due to limited conditions, only one earthquake shelter can be built according to the existing resources. There are four feasible alternatives to choose from: (1) x1: near the station; (2) x2: near a school; (3) x3: near the residential area; (4) x4: near the hospital. The 20 experts participated in the building of the project, including government officials, construction builders, emergency management experts and many other fields. These experts need to evaluate the feasible alternatives from the following five factors: (1) f1: cost; (2) f2: capacity; (3) f3: construction difficulty; (4) f4: individual preference; (5) f5: time constraint. For example, when building a large capacity shelter, its manufacturing cost, construction difficulty and construction time will usually increase, so everyone has different attitude on these factors. Due to each alternative has its own advantages and disadvantages, it is necessary to select a satisfactory alternative through group decision-making. The information provided by experts is reported in Appendix B.

6.2. Decision-Making Process

The proposed consensus model is applied to increase the consensus level and to obtain the optimal alternative, the steps of which are shown as follows. Noting that  =  =  = γ = 0.2,  = 0.5, and a = 0.8 in this case:Step 1. The 20 DMs are divided into 5 subgroups by using the Algorithm 1 as shown in Table 1.Step 2. Calculate the subgroups’ original evaluation matrices through equation (1), that is,Then, calculate the DMs’ weights and the attributes’ weights in Gk(k = 1,2, …, 5) by using Algorithm 2. And compute the subgroups’ weights and the collective attributes’ weights through Algorithm 3. The calculation results are shown in Table 2.Noting that in this step the computation results of the harmonious degree of the subgroups are shown in Table 3.Step 3. Based on equation (22), the unit adjustment cost for each DM can be calculated and then obtained, and the results are reported in Table 4.Step 4. The CRP.First, the computation results of the current consensus level and other information and the consensus threshold are recorded in Tables 5 and 6, respectively.Apparently, GCL0 = GCL = 0.8302 < δ = 0.8590, the consensus feedback then should be executed.The opinion of the DM e8 in G5 is first modified through adjustment process, and then obtain the adjustment result, i.e. GCL1 = 0.8362 < δ = 0.8590. The consensus adjustment process therefore should be executed again. The DM e19 in G1 is identified and adjusted, and then obtain the adjustment result, i.e. GCL2 = 0.8410 < δ = 0.8590. In this time, CAR1 = 2.4717 < CAR2 = 2.6229, so the CRP should not be terminated. Subsequently, the evaluation information of the DM e20 in G5 need to be modified, then the adjustment result is GCL3 = 0.8446 < δ = 0.8590, and CAR2 = 2.6229 < CAR3 = 3.0673. The consensus feedback should be executed again. The opinion of the DM e4 in G4 is modified. The adjustment result is GCL4 = 0.8489 < δ = 0.8590, and CAR3 = 3.0673 > CAR4 = 2.7436. The consensus feedback process should be terminated and T = 3. In addition, the fourth iteration result should be restored, and the final group consensus level is GCLT = 0.8446. The adjustment process of the consensus level and the change process of CARt are shown in Tables 7 and 8, respectively.The expert adjusted evaluation matrices are obtained and reported in Appendix B. And the subgroups’ adjusted evaluation matrices can be derived by equation (1), that is,Step 5. Based on the above calculation results, the group’s evaluation information is obtained, that is,and the score value for each alternative can be derived by equation (3), that is s(x1) = 0.4739, s(x2) = 0.5413, s(x3) = 6383, and s(x4) = 0.4901. Furthermore, the ranking of the alternative is x3 x2 x4 x1, and the optimal alternative is x3.

Input: The expert original evaluation matrices Qm, the number of subgroups K, the initial subject weight vector wm, and the allowed modification values and .
Output: The expert adjusted evaluation matrices, the number of iterations of CRP T, the final group consensus level GCLT and the optimal alternative xp.
Stage 1. Expert clustering.
Step 1. The DMs participated the LSGDM problem are divided into K subgroups by using Algorithm 1. Then, the subgroups G1,…, GK are obtained.
Stage 2. The weight determination.
Step 2. Calculate the DMs’ weights and the attributes’ weights for each subgroup through Algorithm 2. Then, the final weights of DMs and attributes for each subgroup can be derived. For the subgroup G1, (em ∈ Gk) and (n = 1, …, N) are obtained. It should be noted that there is an interactive process with DMs to improve the participation of DMs and better reflect the attitude of DMs. The details are shown in Step 10 of Algorithm 2.
Step 3. Compute the weights of the subgroups and the collective attributes by utilizing the Algorithm 3.
Stage 3. Obtain the unit adjustment cost cm by equation (22).
Stage 4. The CRP.
Step 4. Calculate the value of δ through equation (12), and let t = 0,
Step 5. Calculate the current consensus level GCLt, and judge whether the consensus level is acceptable. If GCLt ≥ , it means that an acceptable consensus level has been reached and go to Step 11; otherwise, let t = t + 1, and go to the Step 6.
Step 6. Determine which the subgroup and DM should be adjusted in the t-th round. The details are shown in Section 5.2.1.
Step 7. Modify the DM’s evaluation information matrix and obtain the adjusted evaluation matrix through Algorithm 4.
Step 8. Compute the consensus adjustment cost ACt in the t-th round by using equation (12).
Step 9. Compute the total adjustment cost TACt and consensus adjustment rate CARt before the t-th round by equations (13) and (14), respectively.
Step 10. Determinate whether the CRP should be terminated. If CARt-1 ≤ CARt, it should go to the Step 5; otherwise, the CRP should be terminated, let T = t − 1, and go to the Step 11.
Step 11. Output the final evaluation matrices of the DMs, the number of iterations of CRP T, and the final group consensus level GCLT. Then, go to the Step 13.
Stage 5. Selection process.
Step 12. Calculate the score s(xp)for each alternative by equation (3) and obtain the ranking of the alternative.
Step 13. Output the optimal alternative xp. End.
6.3. Discussion

In this case, the consensus level is changed from 0.8302 to 0.8446. Although the consensus threshold δ = 0.8590 has not been reached, and the consensus level has also been greatly improved. Also, the original information has been retained as much as possible in the consensus feedback process. Therefore, the proposed consensus model considering the interactive weights and the experts’ adjustment willingness is verified to be effective.

7. Comparative Analysis

In this section, to demonstrate that the characteristics of the proposed consensus method is imperative, comparative analyses that without the interactive and without the experts’ adjustment willingness, respectively, are implemented in Section 7.1 and Section 7.2.

7.1. The Proposed Model without considering the Interactive Weights

Continuing the case that given in Section 5, the proposed consensus model is executed without considering the determination of the interactive weight. Therefore, from Step 10 to Step 12 of Algorithm 2 and from Step 2 to Step 4 of Algorithm 3 are missing. The analyses process is performed as follows:Step 1-A. The content of this part is the same as the Step 1 of Section 6.2.Step 2-A. The subgroups’ original evaluation matrices do not change. But the weight determination process and results are changed and shown in Table 9.Step 3-A. Due to the interactive process is missing, the unit adjustment cost is not affected by harmonious degree. Thus, the determination method of the unit adjustment cost is simplified, that is,The unit adjustment cost without considering the interactive weights can be derived and shown in Table 10.Step 4-A. The computation results of the current consensus level and other information and the consensus threshold are reported in Tables 11 and 12, respectively.Apparently, GCL’0 = GCL’ = 0.8976 < δ = 0.9068, the consensus feedback then should be executed. The adjustment process of the consensus level is shown in Table 13. It should be noted that in the identification process the DM who have a higher weight is adjusted if there are several DMs who have the same unit adjustment.Step 5-A. Based on the above calculation results, the group’s evaluation information is obtained, that is,and the score value for each alternative can be derived by equation (3), that is s(x1) = 0.4674, s(x2) = 0.5284, s(x3) = 0.6243, and s(x4) = 0.4808. Furthermore, the ranking of the alternative is x3 x2 x4 x1, and the optimal alternative is x3.

7.2. The Consensus Feedback without considering the Adjustment Willingness

Similar to Section 7.1, the proposed consensus model is executed without considering the adjustment willingness. Therefore, Step 8 and Step 9 of Algorithm 2 are missing, and Algorithm 4 is no longer available. The analyses process is shown as follows:Step 1-B. The content of this part is the same as the Step 1 of Section 6.2.Step 2-B. The subgroups’ original evaluation matrices do not change. But the weight determination process and results are changed and shown in Table 14.Step 3-B. Due to the adjustment willingness of the DM is missing, the unit adjustment cost is not affected by the allowed modification range. Thus, the determination method of the unit adjustment cost is simplified, that is,The unit adjustment cost without considering the adjustment willingness can be derived and shown in Table 15.Step 4-B. The computation results of the current consensus level and other information and the consensus threshold are reported in Tables 16 and 17, respectively.Apparently, GCL’0 = GCL’ = 0.8306 < δ = 0.8592, the consensus feedback then should be executed. In this subsection, a novel adjustment rule is used, and the details are carried out as follows.Suppose that the evaluation matrices of the DM and the subgroup to be adjusted are and , then the adjusted evaluation matrix of the DM is  = 1/2  + . The adjustment process of the consensus level is shown in Tables 18 and 19, respectively.In this consensus feedback process, the maximum adjustment round T is 2.Step 5-B. Based on the above calculation results, the group’s evaluation information is obtained, that is,and the score value for each alternative can be derived by equation (3), that is s(x1) = 0.4713, s(x2) = 0.5263, s(x3) = 0.6223, and s(x4) = 0.4843. Furthermore, the ranking of the alternative is x3 x2 x4 x1, and the optimal alternative is x3.

7.3. Discussion

Based on the above introduction, the differences between the proposed model and without considering the interactive weights, the proposed model and without the experts’ adjustment willingness are shown as follows.

7.3.1. The Subgroups’ Consensus Level

The proposed model, without considering the interactive weights, and without the experts’ adjustment willingness of the subgroups’ consensus levels are shown in Figure 2. Specifically, the ranking of the initial consensus level is CL3 > CL4 > CL2 > CL1 > CL5 in the models M-1 and M-3 while model M-2 is CL3 > CL2 > CL4 > CL1 > CL5. The consensus levels of the subgroups G3 and G5 of the M-1 and M-3 are 0.9600 and 0.7707, respectively, while the M-2 are 1.0000 and 0.8584.

For the model M-2, the stage showing the decision results to experts is omitted in the process of weight determination. Hence, the decision information is not fed back to DM in time, and the decision-maker’s attitude towards the decision information is not obtained. Although the consensus level of the M-2 is higher than the M-1 (i.e. the proposed model), the calculation of consensus level is inaccurate and partial. Therefore, the decision-making result cannot fully represent the wishes and attitudes of the DMs.

For the model M-3, the stage reflecting the DMs’ adjustment willingness information is removed. Thus, the adjustment attitude of DMs to their own evaluation information cannot be known, and the lack and distortion of the DMs’ evaluation information may lead. Although the consensus level for each subgroup of the M-3 is very similar with the M-1, the DMs’ adjustment willingness (i.e. the stubborn degree to their own opinion) is omitted, and the acquisition of consensus level is not scientific and reasonable. Therefore, it may lead to the unreasonable decision-making result due to the DMs’ opinion is distorted or ignored.

7.3.2. The Changes of the Group Consensus Level GCLt and the Adjustment Cost

The proposed model, without considering the interactive weights, and without the experts’ adjustment willingness of the group consensus level GCLt and the adjustment cost are record in Table 20.

For the model M-2, after 2 rounds of the consensus feedback process, the consensus threshold and an acceptable consensus level has been reached. Due to the lack of timely attitude of DMs towards decision-making information, the satisfaction degree for each DM is not obtained. Also, the consensus level of the subgroup does not involve the harmonious degree, namely the recognition of the subgroup is not presented. Therefore, the consensus level of the M-2 improves faster than the M-1 because it does not consider the harmonious degree of the subgroup. Meanwhile, the calculation method of the unit adjustment cost is partial, and the adjustment cost of the M-2 increases faster than the M-1 and then is inaccurate and unscientific. Therefore, the decision-making result of the M-1 is more reasonable than M-2.

For the model M-3, after 2 rounds of the consensus feedback process, the termination condition is satisfied and the final consensus level can be obtained. Due to the lack of the DMs’ adjustment willingness information, the distortion of the DMs’ opinion may lead. And, the consensus level of the M-3 increases slower than the M-1. For instance, after 2 rounds of the iteration process the group consensus level of the M-3 is 0.8365 while the M-1 is 0.8410. And t after one round the group consensus level of the M-1 is 0.8362. It is obvious that the result of the M-3 adjustment twice is similar to that of the M-1 adjustment once. Meanwhile, the computation process of the unit adjustment cost is also partial, and the adjustment cost of the M-3 is lower than the M-1 and then is inaccurate and unreasonable. From the perspective of the experts’ adjustment willingness, the decision-making result of the M-1 is more scientific than M-3.

In summary, it is apparent that both considering the interactive weights and the experts’ adjustment willingness for the CRP is not only very reasonable but imperative.

8. Sensitivity Analysis

In this section, sensitivity analyses that the unit adjustment cost and the harmonious degree in the CRP, respectively, are presented in Section 8.1 and Section 8.2.

8.1. The Effect of the Unit Adjustment Cost cm in Consensus Feedback Process

In Section 6, the unit adjustment cost cm for each DM is obtain based on equation (22) and the value of is 0.5. In this subsection, a discussion based on the different unit adjustment cost is introduced. To further prove the rationality of the proposed consensus model, a sensitivity analysis is conducted with different values of  = {0.1, 0.3, 0.5, 0.7, 0.9}. The unit adjustment cost for each DM, the group consensus levels, and the score of the alternatives with different are recorded in Figure 3, Tables 21, and 22, respectively.

According to Figure 3, it is apparent that the unit adjustment cost cm increases with respect to the value of for each DM. For the different value of , the unit adjustment cost between two decision makers may be different. For example, the unit adjustment costs of e14 and e15 are 0.5000 and 0.6775 when  = 0.5, respectively, while they are 0.8608 and 0.7827 when  = 0.9. In other words, c14 < c15 when  = 0.5 while c14 > c15 when  = 0.9. Therefore, we set a moderate value (i.e.  = 0.5) in the case study due to we cannot accurately obtain the importance of the experts’ adjustment willingness or the satisfaction degree. For Table 21, the termination round is the first stage when  = 0.1 and 0.3, while the termination round is the third stage when  = 0.5, 0.7, and 0.9. This is because with the increase of adjustment cost, there are more opportunities, however, if the adjustment cost is too high, it is unfavorable for the consensus feedback process. Therefore, setting  = 0.5 is reasonable in the case study.

Based on Table 22, we found that the scores for each alternative are very similar. Due to the different unit adjustment cost, the different value of may lead to different DM to be adjusted in the same stage. Therefore, it will be a small difference in the score of the alternative with the different . However, the ranking of the alternatives is the same with the different . Therefore, it means that setting  = 0.5 is reasonable in the case study, and the proposed consensus model considering both the interactive weights and the experts’ adjustment willingness is stability and rationality.

8.2. The Effect of Harmonious Degree in the Consensus Level Calculation

In Section 6, the consensus level for each subgroup is computed based on equation (14) and the value of a is 0.8. To further prove the rationality and stability of the proposed consensus model, a sensitivity analysis is introduced with different values of a = {0.6, 0.7, 0.8, 0.9, 1.0}. The consensus thresholds, the group consensus levels and the adjustment costs, and the score of the alternatives with different values of a are reported in Tables 2325, respectively.

For Tables 23 and 24, it is apparent that the consensus thresholds and the initial group consensus levels increases with respect to a. Although the increase of the importance of the subgroups’ harmonious degree will lead to the decrease of the subgroups’ and the collective consensus level, it is very critical for the decision-making results. The consensus level of the subgroup not only represents the consistency of the opinion within subgroup, but also represents the DMs’ attitude towards the decision-making results and the stability of the sub group’s opinion. Therefore, it is reasonable and imperative that the consensus level considers both the opinion’s consistency and the DMs’ attitude. Meanwhile, the final group consensus levels satisfies the equation CART ≤ CART+1 when a = 0.6, 0.7 and 0.8, while that meets the equation GCLT ≥ δ when a = 0.9 and 1.0. This is because with the increase of a, the differences between the consensus threshold δ and the initial consensus level GCL0 decreases, the consensus termination conditions are easy to meet than before. For example, the difference between the consensus threshold δ and the initial consensus level GCL0 is 0.0468 when a = 0.6 while 0.0097 when a = 1.0. Hence, with the increase of a, the difference between the current GCLt and δ is less and less. Also, the constraint condition CART ≤ CART+1 can obtain the GCLT faster. Therefore, it is reasonable that the importance of the harmonious degree is 0.2 (i.e. a = 0.8) in the case study, and the termination condition is necessary and rationality.

According to Table 25, we found that the scores of alternatives are the same when a is 0.6 and 1.0, 0.7 and 0.8, respectively. This is because the iteration round is the same. Interestingly, the ranking of alternatives is the same when a is different. It shows that the proposed consensus model is more stable and the decision-making result is more scientific.

9. Conclusion

In this study, we propose a consensus model of LSGDM considering the interactive weights’ determination and the experts’ adjustment willingness, and apply it to select the building of an earthquake shelter. The main contributions and innovations of this research are shown as follows:(1)We develop a novel method of weight determination, which considers the DMs’ attitude towards the decision-making results, thereby ensuring the effective participation of DMs. Moreover, to improve the rationality of LSGDM, the harmonious degree is conducted in the calculation of subgroups’ weight. It is of significance for the DMs more involved in the decision-making process and the decision-making result more reflect the willingness of DMs.(2)By considering the experts’ adjustment willingness, it is ensured that the evaluation information is less distorted or lost. Moreover, the unit adjustment cost is designed. Subsequently, to improve the efficiency of CRP, an identification rule combines the unit adjustment cost and the consensus level is presented to retain as much original information as possible in consensus feedback process, which can easily reach an acceptable level of consensus.(3)An objective calculation method of the consensus threshold is conducted, and then a termination condition that considers both the current consensus level and the consensus adjustment rate is designed to objectively terminate the CRP. It not only compares the current consensus level with consensus threshold but compares the consensus adjustment rate. As a result, this method can address the subjectivity and unreasonableness of the preset consensus threshold and the maximum number of iterations to a certain extent.

Despite several valuable findings obtained by our research, there remain some limitations that should be further dealt with in the future. In this study, in addition to the interactive weights of the proposed model, the professional knowledge background and decision-making experience and other individual attributes of DMs should be considered. Moreover, with the development of social networks, the relationship between DMs that become complex should be considered in LSGDM. Also, the behavioral factors (i.e. non-cooperation) [47] and psychological factor (i.e. self-confident) [48] of DMs should also be conducted, it will be a meaningful research for the LSGDM problems.

Appendix

A: The Proof of Theorem 1

Proof. First, the values of and are both positive and in the interval [0, 1], so we have 0 ≤  +  ≤ 2 and 0 ≤  ≤ . For the function y = cos(x), y is monotone decreasing when x is in the interval [0, ]. Thus, we can obtain that the value of is in the interval of [0, 1]. Then, the value of is positive and in the interval [0, 1]. The function z = sin() is monotone increasing when x is in the interval [0, 1], so the value of z is in the interval [0, 1]. Therefore, the value of cm is in the interval of [0, 1].

B: The Information Provided by Experts

The information provided by experts are reported in Tables 2628. In this paper, the value range of is from 0 to 1 (Tables 29 and 30).

Data Availability

The data used to support the findings of this paper are included within the article (Case Study section and Appendix).

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

The authors appreciate the financial support of the National Key Research and Development Program of China (Grant no. 2018YFB1402500) and the Natural Science Foundation of Heilongjiang Province (Grant no. JJ2021LH1530). The Reform and Develop High-Level Talent Projects in Local Universities Supported by the Central Government (2020GSP13)