Data-driven Fuzzy Multiple Criteria Decision Making and its Potential ApplicationsView this Special Issue
Application of Multiattribute Decision-Making for Evaluating Regional Innovation Capacity
The growing imbalance in regional innovation development has become an urgent issue in China’s strategy to build an innovative country. To enrich the regional innovation capacity evaluation system, scientifically assess regional innovation capacity, and explore available pathways to improve regional innovation capacity, this paper introduces a multiattribute decision-making method for evaluating regional innovation capacity. First, a random forest model and the DEMATEL-based analytic network process (DANP) method are applied to calculate the weights of the evaluation attributes. Second, the multiobjective optimization by the ratio analysis method based on the maximum and minimum (MOORA-min-max method) is used to calculate the evaluation attribute gap ratios and regional innovation capacity of each region. Finally, the limitations of regional innovation development are identified based on the evaluation attribute gap ratios and the critical influence strength roadmap (CISR) to explore the regional innovation capacity improvement pathways. The results show that “output capacity of R&D personnel in universities and research institutes” is the most fundamental evaluation attribute in the regional innovation capacity evaluation, while “output efficiency of R&D funds in universities and research institutes” is the most influential evaluation attribute. Research in Sichuan and Inner Mongolia reveals that regions need to identify critical constraints in four aspects: knowledge creation, knowledge acquisition, enterprise innovation, and innovation environment, to improve regional innovation capacity.
Persistent imbalance in the development of regional innovation capacity constitutes a substantial bottleneck constraining the effort to upgrade countries’ integrated innovation capacity. China’s regional innovation development is influenced by its history, economy, and geography and thus varies significantly [1–3]. To better promote the role of regional innovation in high-quality economic development, Chinese policymakers have placed innovation at the centre of overall national development. The 19th National Congress of the Communist Party of China proposed that “innovation is the first impetus for leading development” and issued a strategic plan to accelerate the construction of an innovation-oriented country.
Regional innovation capacity is an essential indicator of an innovation-oriented country, and many scholars have focused on regional innovation capacity in recent years [4, 5]. Scholars have thoroughly studied the conceptual definition of regional innovation capacity, including its differences, indicators, influencing factors, and formation mechanisms [6–10]. For example, Shan constructed a system for evaluating regional innovation capacity based on four aspects: input capacity, innovation environment, management capacity, and innovation output . Hamidi et al. examined the relationship between regional compactness and regional innovation capacity in the United States . Tang et al. studied the spatial effect of absorptive capacity on regional innovation capacity from the perspective of knowledge spillover theory . Many organizations have also published reports on the evaluation of innovation. The World Intellectual Property Organization and others cofounded the Global Innovation Index (GII), which ranks countries according to their innovation capacity and economic performance. Similarly, the National Research Group on S&T Development Strategy released the China Regional Innovation Capability report, which provides an objective, dynamic, and comprehensive assessment of China’s innovation capacity in each region.
With the advent of the Fourth Industrial Revolution, the ability to innovate has once again become a vital capacity for countries to compete for the right to global value distribution. For China, which displays significant differences in regional innovation development, improving regional innovation is an urgent issue. An accurate evaluation of regional innovation capacity is the basis for monitoring the current state of regional innovation development, identifying constraints on regional innovation development, and exploring pathways for improving regional innovation capacity. Accordingly, this paper selects evaluation indicators from four aspects, namely, knowledge creation, knowledge acquisition, enterprise innovation, and innovation environment and then applies the multiattribute decision-making method to evaluate the innovation capacity of all 31 provinces in China and to explore pathways for improving regional innovation capacity. The main contributions of this paper are as follows: (1) The random forest model and the decision-making trial and evaluation laboratory- (DEMATEL-) based analytic network process (DANP) method are introduced to innovation capacity evaluation research, thereby enriching the method for calculating objective weights; moreover, a multiattribute decision-making evaluation model of regional innovation capacity provides a new method to evaluate regional innovation objectively. (2) The multiobjective optimization by ratio analysis (MOORA) method and the critical influence strength roadmap (CISR) are introduced into the field of innovation research to provide a new approach to scientifically identify the factors limiting regional innovation capacity.
2. Literature Review
To date, scholars have not reached a consensus on how to evaluate regional innovation . In the existing literature, innovation capacity evaluation research is divided into two aspects: evaluation indexes and evaluation methods. In terms of research on evaluation indexes, common indicators for evaluating regional innovation capacity include innovation resource inputs, innovation outputs, and innovation environments [15–17]. Other scholars have proposed patents as an essential indicator of regional innovation capacity [18–21]. For example, Hamidi et al. used the number of patents as one indicator of regional innovation capacity . In addition, scholars have assessed the selection of indicators for evaluating regional innovation capacity in terms of system structure and green innovation. For example, Han et al. used the innovation participant framework  to classify innovation participants into eight categories, including government departments, research institutes, colleges and universities, enterprises, and technology intermediaries. They selected 63 innovation participant indexes to analyse the innovation capacity of 16 cities in Korea . Chen et al. established regional innovation capabilities based on knowledge management from 6 aspects: knowledge base, knowledge creation, knowledge dissemination, knowledge sharing, knowledge application, and innovation environment . Wang et al. constructed a system of regional innovation evaluation indicators from green innovation inputs, green innovation outputs, and green innovation environments .
The research on evaluation methods for innovation capacity is divided into the calculation of evaluation index weights and the comprehensive evaluation of innovation capacity. The subjective weighting method and objective weighting method are the most common approaches for calculating the weights of evaluation indexes . The former refers to weights determined subjectively through people’s experience; for example, Shan developed a hierarchical analysis model of regional innovation capacity and used the analytic hierarchy process to calculate the weight of each evaluation indicator . The latter refers to the analysis of the relationships between indicators based on objective data; for example, Yan et al. determined the indicator weights of regional technological innovation capacity by using the entropy method and empirically analysed the technological innovation capacity of 80 regions in Hubei Province, China . Scholars have also proposed to comprehensively determine the weights of index combinations. For example, Xu et al. used the cloud model method and the entropy method to determine the initial weight of an index. Then, the cloud model was combined with the DEMATEL method to determine the final comprehensive weight of the indicator . In terms of the comprehensive evaluation of innovation capacity, Sheng et al. used grey system theory and the Delphi method to establish a system of indicators for evaluating regional S&T innovation capacity. They then applied a new grey cluster model based on a mixed centre-point triangular whitenization weight function to evaluate the regional S&T innovation capacity of five cities in Jiangsu Province, China . Yang et al. proposed a method based on uncertain linguistic variables for evaluating enterprise innovation capacity and analysed the innovation capacity of five firms . Similarly, Zhen introduced the induced 2-tuple linguistic Choquet ordered harmonic average (I-2TCOHA) operator to aggregate the 2-tuple linguistic information corresponding to each alternative and rank the alternatives to evaluate the technological innovation capacity of a firm . Li et al. proposed a multidimensional grey fuzzy decision-making method with feedback based on the weight vector and weight matrix and applied the method to evaluate regional financial innovation capacity . Wang et al. used the fuzzy analytic hierarchy process to evaluate regional green innovation capacity .
In summary, the subjective evaluation method (represented by the analytic hierarchy process) determines evaluation indicator weights based on subjective ideas, while ignoring the information provided by data, resulting in the lack of an objective scientific basis for the resulting weights. In contrast, the objective evaluation method can effectively avoid subjective problems but suffers from certain deficiencies; for example, the entropy method ignores the lack of a horizontal comparison between indicators. The random forest model can calculate the influence strength among indicators, thereby avoiding the problem of subjectivity, and thus can make comparisons between evaluation indicators. Existing scholars have performed a considerable amount of work on methods to evaluate innovation capacity and have proposed a series of policy recommendations; however, these suggestions are generally universal. Due to the differences among individual entities, the pathways to improving innovation capacity also vary. The MOORA method and critical influence strength roadmap (CISR) are introduced to evaluate innovation capacity; the former identifies the maximum gap ratio among the evaluation indicators, while the latter is a diagram showing the influence pathways among the evaluation indicators. The combination of these two can effectively identify the factors that restrict the development of individual innovation capacity and enables improvements in innovation capacity with individual differences.
3. Establishing the Indicator System and Building the Model
This section briefly introduces the construction of the indicators and models for evaluating regional innovation capacity. First, the main participants of the regional innovation system are universities, research institutes, companies, government agencies, and intermediaries [7, 17, 32], which are responsible for different roles: universities and research institutions with excellent researchers and sufficient innovation resources assume the role of knowledge creation; enterprises, as core participants in regional innovation systems, take on a greater role in transferring innovation knowledge; and government and intermediaries are responsible for providing a suitable environment for regional innovation. Second, the evaluation model uses the “drop-column importance” concept proposed by Terence et al. to measure the influence strength between evaluation indicators [33, 34]. The calculated influence strength is used as the raw data for the DANP approach, replacing the expert scoring approach. The DANP method is used to obtain a diagram of the influence strength network and the weight of each evaluation indicator. The MOORA method is then combined with maximum or minimum values to develop the MOORA-max-min method, which can calculate the gap ratio between the current level of an evaluation indicator and its maximum or minimum value. The regional innovation capacity is then calculated according to the weights of the evaluation indicators and the gap ratios. The model construction process is shown in Figure 1.
3.1. Establishing the Indicator System
The concept of the regional innovation system was first proposed by the British scholar Cooke in 1992. A regional innovation system is a regional organizational system formed by the division of labour, interconnected enterprises, universities, research institutions, intermediary services, and local governments within a specific geographical area . Evaluation indicators should consider the generation and application of knowledge and the underlying environment of regional innovation [36–39]. After analysing the characteristics and linkages of China’s regional innovation system, this paper constructs an evaluation indicator system from four aspects: knowledge creation, knowledge acquisition, enterprise innovation, and innovation environment. (1) Knowledge creation: knowledge creation is a source of regional innovation and refers to the process involving universities and research institutes, research funding, and inputs from researchers, including the inputs and outputs of innovation, measured mainly as research inputs, patents, and thesis outputs. (2) Knowledge acquisition: knowledge acquisition refers to the flow and utilization of knowledge within a region, measured mainly by thesis cooperation, corporate financial support, and the use of foreign investment. (3) Enterprise innovation: the translation of innovation results into products by firms is an important part of regional innovation. Corporate innovation is measured by corporate research inputs, patent outputs, and new product development. (4) Innovation environment: a good innovation environment can promote regional innovation. The innovation environment is measured mainly in terms of regional development, quality of intermediary services, and sustainability capacity. The resulting system of indicators for the evaluation of regional innovation capacity is shown in Table 1.
To guarantee the reproducibility of this study, the data in this paper come from publicly published statistical yearbooks and government reports, mainly including the China Statistical Yearbook, China Statistical Yearbook of Science and Technology, China Statistical Yearbook of High-Tech Industry, China Industry Economy Statistical Yearbook, China Torch Statistical Yearbook, statistical and analytical reports on Chinese science and technology papers, and related data released by the Ministry of Science and Technology, State Intellectual Property Office, State Administration for Industry and Commerce, and the Technology Innovation Fund for Science and Technology-based Small- and Mid-Size Enterprises (SMEs).
3.2. Building the Multiattribute Decision-Making Evaluation Model of Regional Innovation Capacity
3.2.1. Using a Random Forest Model to Construct an Initial Influence Strength Matrix
Step 1: discretize the data for all evaluation attributes through a three-level interval discretization method comprising the top third (marked “H”), the middle third (marked “M”), and the bottom third (marked “L”) of the value range for each evaluation attribute . Step 2: divide the n evaluation attributes into n groups according to the individual evaluation attributes (called decision evaluation attributes ) and n − 1 evaluation attributes (called conditional evaluation attributes ). Step 3: use the random forest model to calculate the influence strength relationship among conditional evaluation attributes and decision evaluation attributes ; the calculated influence vector of conditional evaluation attributes on is , where . Steps 2 and 3 are repeated n times for each decision variable to obtain n combinations of vectors. Furthermore, these n combinations of vectors are combined into an n-dimensional matrix , where matrix D is called the direct influence strength matrix and represents the degree of influence of the evaluation attribute on the evaluation attribute when . Steps 2 to 4 are repeated times (, one of the parameters of the random forest model, is the number of n estimators obtained through the learning curve) to obtain direct influence strength matrices D.
3.2.2. Applying the DANP Method to Draw the Influence Strength Network Diagram and Calculate the Evaluation Attribute Weights
Step 1: calculate the initial influence strength matrix. The direct influence strength matrices D are measured by averaging them using the following equation to obtain the initial influence strength matrix A: Step 2: normalize the initial influence strength matrix. The initial influence strength matrix A is converted by means of the following equations to obtain the normalized initial influence strength matrix Y: where is the reciprocal of the maximum value of the sum of the rows or the columns of the initial influence strength matrix and is used to normalize the initial influence strength matrix. Step 3: solve the total influence strength matrix. Based on the initial influence strength matrix D and the Markov chain matrix, the total influence strength matrix T is calculated using the following equation: where E is the n-dimensional identity matrix. Step 4: draw the influence strength network diagram. The following equations are used to obtain and , and then the influence strength network diagram is plotted: where represents the matrix transposition and represents the total influence strength of the evaluation attribute on the other evaluation attributes, called the degree of influence of the evaluation attribute. represents the total influence strength of the other evaluation attributes on the evaluation attribute and is called the degree of influence of the evaluation attribute. The centrality degree reflects the importance of the evaluation attribute in the system. The causality degree , when , indicates that the evaluation attribute is the causality attribute and influences the other evaluation attributes; if , then the evaluation attribute is the result attribute and is influenced by the other evaluation attributes. Step 5: build the unweighted supermatrix W and the weighted supermatrix . The unweighted supermatrix W is calculated using the total influence strength matrix T and equation (7). In addition, the unweighted supermatrix W is the weighted supermatrix because all the evaluation attributes in this paper are of the same level: Step 6: calculate the weights for each evaluation attribute. Equation (8) is iterated until the results converge to a stable limit supermatrix to obtain the weights:
3.2.3. Calculating Regional Innovation Capacity Using the MOORA-Min-Max Method
Step 1: establish a decision matrix. The decision matrix , where m represents the number of regions, n represents the number of evaluation attributes, and represents the value of the evaluation attribute in province x consisting of actual data from the 31 provinces in China. Step 2: normalize the decision matrix. Given the quantitative variation among the indicators for the evaluation of regional innovation capacity, the decision matrix must be normalized. Equation (9) is rewritten as equation (10) based on the concept of range normalization: Step 3: determine the gap ratio for each province. The gap ratio equation (11) is rewritten as equation (12) based on the normative decision matrix, and equation (13) is used to calculate the regional innovation capacity for each province:where represents the weight of the evaluation attribute, represents the gap ratio of regional innovation capacity in province x, and represents the regional innovation capacity in province x.
4. Empirical Study
This paper applies the constructed multiattribute decision-making evaluation model of regional innovation capacity to assess the regional innovation capacity of the 31 provinces in China and to explore ways to improve regional innovation capacity.
4.1. Calculating the Direct Influence Strength between Evaluation Attributes Based on a Random Forest Model
After the three-level interval discretization of the original data, the evaluation attributes are divided into 24 groups according to the individual evaluation attributes (called decision attributes ) and 23 attributes (called conditional attributes ). Then, learning curves are used to train random forest models. For example, “regional technology manpower investment (C1)” is set as a decision attribute, C2 to C24 are set as the conditional attributes, and a learning curve is used for parameter tuning. These steps are repeated to determine that the value of k is 20. After k is determined, the direct influence strength between the evaluation attributes is calculated using a random forest model to obtain 20 direct influence strength matrices; then, the initial influence strength matrix A is obtained via equation (1), as shown in Table 2.
4.2. Drawing an Influence Strength Network Diagram and Calculating the Evaluation Attribute Weights Based on the DANP Method
The DANP method uses DEMATEL to calculate the total influence strength matrix of the evaluation attributes and then solves the evaluation attribute weights, and the method is used to draw an influence strength network diagram.
This paper calculates the accuracy of the random forest model and verifies the consistency of the 20 direct influence strength matrices before calculating the total influence strength matrix to ensure the reliability of the evaluation results. The former step tests the accuracy of the influence strength calculated by the random forest models, and the latter step tests the consensus among the 20 direct influence strength matrices. The random forest model typically uses the out-of-bag error (OOB error) rate to measure the model quality. As shown in Table 3, the average OOB error is between 0.01 and 0.26, with the evaluation attribute “cooperation between enterprises and universities and research institutes (C8)” having the worst quality and an average OOB error of 0.26 with an average accuracy of 74.20%. The evaluation attribute “regional technology manpower investment (C1)” has the highest quality with an average accuracy of 99%. A consistency test among the 20 direct influence strength matrices yields an average consistency gap ratio of 3.129% (less than 5%), as shown in Table 4, indicating that the 20 direct influence strength matrices have a high degree of consensus and that the results are reliable.
The initial influence strength matrix A is transformed by using equations (2) and (3) to obtain the normalized initial influence strength matrix Y. Then, matrix Y is calculated according to (4)fd4 to obtain the total influence strength matrix T (Table 5). Finally, the degree of influence , degree of being influenced , centrality degree , and causality degree for each evaluation attribute are calculated using equations (5) and (6) (Table 6). Table 6 shows that evaluation attributes C1, C2, C3, C6, C10, C11, C17, C19, C20, C21, C22, and C23 are the cause attributes and C4, C5, C7, C8, C9, C12, C13, C14, C15, C16, C18, and C24 are the result attributes. The influence strength network diagram is then drawn according to the calculated centrality and causality of each evaluation attribute (Figure 2).
The influence strength network diagram visualizes the importance and grouping of each evaluation attribute (cause or result attribute), but it remains challenging to demonstrate the complex associations among each attribute. This paper introduces the concept of the net influence strength to reflect the relation between each pair of attributes. The net influence strength is the relative magnitude of and in the total influence strength matrix T, . When , the evaluation attribute greatly influences the evaluation attribute, and influences ; when , influences . For example, and in the total influence strength matrix T, , C3 influences C1, and thus, in the net influence strength matrix . Additionally, given that influence strength exists between two or more evaluation attributes, is null when . Similar calculations are repeated to obtain the net influence strength matrix (Table 7). For the evaluation attribute, is grouped to obtain a net influence grouping for each evaluation attribute (Table 8). The number of net influence groupings indicates the net influence strength level of the evaluation attributes. When the number of net influence groupings is equal, further analysis of the net influence strength among these evaluation attributes is needed. For example, for the evaluation attributes C1, C2, and C10, the net influence grouping is 18; in Table 8, C2 influences C1, and C10 and C10 influence C1. The relations between the evaluation attributes are clarified through net influence groupings, and the CISR is finally drawn (Figure 3).
The CISR illustrates that the “output capacity of R&D personnel in universities and research institutes (C3)” is the most prominent evaluation attribute of net influence strength and can be considered the most fundamental evaluation attribute, while the “cooperation between enterprises and universities and research institutes (C8)” is at the end of the CISR with the smallest net influence strength.
The unweighted supermatrix W and the weighted supermatrix are obtained from equations (7) and (8), and multiplicative operations are performed on the weighted supermatrix until the resultant convergent stable limit supermatrix is obtained. Then, the weights for each evaluation attribute can be determined (Table 9).
4.3. Calculating Regional Innovation Capacity Based on the MOORA-Min-Max Method
This paper normalizes the data from the 31 provinces in China for 2017 using equations (10) and (12) and calculates the gap ratio, and the results are shown in Table 10. Finally, the regional innovation capacity and ranking of each province are calculated using equation (13), as shown in Table 11.
4.4. Exploring Regional Innovation Capacity Improvement Pathways Based on CISR
Table 11 illustrates the differences in regional innovation capacity among the 31 provinces in China. This paper explores regional innovation capacity improvement pathways to improve the imbalance in regional innovation capacity. Due to page limitations, the following presents an analysis of both the possible pathways for improving regional innovation capacity based on the CISR and the regional innovation capacity gap ratio using Sichuan and Inner Mongolia as examples.
As shown in Figure 4, Sichuan’s largest gap ratio is “enterprise new product development capability (C16),” which is influenced by “regional knowledge cooperation (C7),” “enterprise R&D personnel input capability (C12),” “enterprise R&D output capacity (C14),” “enterprise core technology level (C15),” and “regional talent training (C18)”: among these five evaluation attributes, C7 has the largest gap ratio. Therefore, regional knowledge cooperation could be enhanced to improve Sichuan’s regional innovation capacity. Inner Mongolia’s largest gap ratio is “output efficiency of R&D funds in universities and research institutes (C5)”, which is influenced by “Enterprise R&D funding investment capacity (C13),” “enterprise R&D output capacity (C14),” “enterprise core technology level (C15),” and “enterprise new product development capability (C16)”: C16 has the largest gap ratio among these five attributes. Thus, Inner Mongolia should improve enterprises’ new product development capacity.
4.5. Management Implications of the Multiattribute Decision-Making Evaluation Model
Regional innovation capacity provides a comprehensive description of regional innovation development. Improving regional innovation capacity does not involve only a single area of improvement but instead requires four main areas. (1) Knowledge creation: innovation resources are fundamental to knowledge creation. Regions need to increase investment in both human and material innovation resources to increase innovation knowledge output. In addition, each region should improve the output efficiency of R&D personnel and R&D funding through measures such as optimizing the allocation of innovative resources. (2) Knowledge acquisition: cooperation is a meaningful way to fill resource gaps and achieve complementary strengths. Regional enterprises should strengthen financial support for universities and research institutes and actively introduce advanced technologies both domestically and abroad. Regional R&D personnel should develop cooperation between science, technology, and innovation research to improve the output of scientific and technological papers and other results. Local governments should develop regionally appropriate policies to encourage foreign funding for regional innovation development. (3) Enterprise innovation: as essential participants in innovation transformation, enterprises must increase the output of their innovation results by increasing resource investment and conducting research on core technology. Additionally, enterprises should improve their new product development capabilities to achieve economic benefits. (4) Innovation environment: a good innovation environment is an important prerequisite for regional innovation development. Regions need to improve their talent training systems by increasing spending on education. Innovation intermediaries should fully play their role in supporting regional innovation by providing financial and facility resources for regional innovation. Finally, regional innovation should consider energy consumption, environmental pollution, and other issues to achieve sustainable innovation.
This paper builds a multiattribute decision-making evaluation model of regional innovation capacity. The model uses a random forest model to determine the influence strength between evaluation attributes and obtains the objective weights of the evaluation attributes using the DANP method. Finally, the MOORA-min-max method is employed to calculate the regional innovation capacity in each of China’s 31 provinces and to explore regional innovation capacity improvement pathways. The empirical results suggest the following: (1) “Output capacity of R&D personnel in universities and research institutes (C3)” is the most fundamental evaluation attribute; this may be due to China's growing emphasis on industry-university-research as innovation-driven development strategies are proposed. Colleges and research institutes are essential subjects of industry-university-research. Therefore, C3 is the most fundamental evaluation attribute. (2) “Output efficiency of R&D funds in universities and research institutes (C5)” is the evaluation attribute with the largest weight. Given the emphasis on resource efficiency in regional innovation, C5 carries the greatest weight in regional innovation capacity evaluations. (3) Regional innovation capacity is an integrated reflection of innovation development, and enhancing regional innovation capacity involves identifying and addressing critical constraints.
Despite our efforts, two limitations provide ideas for future research. First, the evaluation indicator system constructed by different methods and innovation evaluation indicators (e.g., innovation policy) that are more difficult to monitor can affect the evaluation results, and future research needs to explore a more efficient evaluation indicator system. Second, the multiattribute decision-making evaluation model for regional innovation capacity proposed in this paper is a data-driven approach, and the parameters of the evaluation model ignore the subjective preferences of decision-makers, which should be considered in the future.
The readers can access the data from publicly published statistical yearbooks and government reports via the following links: http://www.istic.ac.cn/tabid/640/default.aspx, http://www.most.gov.cn/, http://www.cnipa.gov.cn/tjxx/index.htm, http://www.saic.gov.cn/“>http://www.stats.gov.cn/tjsj/ndsj/, http://tongji.cnki.net/kns55/navi/HomePage.aspx?id=N2019030267&name=YBVCX&floor=1, http://data.cnki.net/area/Yearbook/Single/N2018050242?z=D18, http://tongji.cnki.net/kns55/navi/HomePage.aspx?id=N2012110073&name=YZGJN, http://data.cnki.net/trade/Yearbook/Single/N201901Z0258?z=018, https://bbs.pinggu.org/thread-7955189-1-1.html, https://www.istic.ac.cn/tabid/640/default.aspx, http://www.most.gov.cn/, http://www.cnipa.gov.cn/tjxx/index.htm, http://www.saic.gov.cn/, and http://www.innofund.gov.cn/.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
This work was supported by the National Natural Science Foundation of China (71774036), Natural Science Foundation of Heilongjiang Province (QC2018088), and the Special Foundation of Central Universities Basic Research Fee (3072020CFW0904 and 3072020CFW0907).
S. L. Zhao, L. Cacciolatti, S. H. Lee, and W. Song, “Regional collaborations and indigenous innovation capabilities in China: a multivariate method for the analysis of regional innovation systems,” Technological Forecasting and Social Change, vol. 94, pp. 202–220, 2015.View at: Publisher Site | Google Scholar
H. Wang, L. An, and X. Zhang, “Evaluation of regional innovation ability based on green and low-carbon perspective,” Bulgarian Chemical Communications, vol. 49, pp. 55–58, 2017.View at: Google Scholar
Q. Yan and R. Y. Hou, “Evaluation of regional scientific and technological innovation capability and empirical research,” Agro Food Industry Hi-Tech, vol. 28, no. 1, pp. 3179–3181, 2017.View at: Google Scholar
K. Q. Sheng and S. F. Liu, “On evaluation system for regional scientific and technical innovation ability and a positive study of 5 cities in south of Jiangsu,” Journal of Grey System, vol. 27, no. 4, pp. 92–103, 2015.View at: Google Scholar
T. Parr, K. Turgutlu, C. Csiszar, and J. Howard, “Beware default random forest importances,” March, vol. 26, 2018.View at: Google Scholar
H. Jiao, J. Zhou, T. Gao, and X. Liu, “The more interactions the better? The moderating effect of the interaction between local producers and users of knowledge on the relationship between R&D investment and regional innovation systems,” Technological Forecasting and Social Change, vol. 110, pp. 13–20, 2016.View at: Publisher Site | Google Scholar
Z. Yang, T. Ouyang, X. Fu, and X. Peng, “A decision-making algorithm for online shopping using deep-learning-based opinion pairs mining and q -rung orthopair fuzzy interaction Heronian mean operators,” International Journal of Intelligent Systems, vol. 35, no. 5, pp. 783–825, 2020.View at: Publisher Site | Google Scholar