Abstract
Considerable resources are needed when implementing the ERP project, so it is necessary to evaluate its performance. Firstly, the evaluation index system of implementation performance of the ERP project was built, and an Analytic Network Process (ANP) which can fully take the relationship between evaluation indexes into account was employed to determine the index weight. Secondly, an improved matter-element extension model, which can overcome the limitations and inadequacies of traditional matter-element extension model when performing the comprehensive evaluation, was proposed to evaluate the implementation performance of the ERP project. Finally, taking an enterprise's ERP project as an example, a comprehensive evaluation was done, and the empirical analysis result shows that this proposed hybrid evaluation model is feasible and practical.
1. Introduction
The Enterprise Resource Planning (ERP) system which is built on the information technology and systematic management thoughts can provide a decision-making management platform for enterprise’s management team and staff. The ERP system plays a significant role in improving the business processes and competitiveness of an enterprise. The implementation of ERP changes the organizational and business mode, which brings a huge impact on each enterprise’s department. Since considerable enterprise resources are needed when implementing ERP project, it is quite necessary to build a reasonable and effective comprehensive evaluation method to evaluate the performance of ERP project.
Many evaluation models have been used to evaluate project performance, such as analytic hierarchy process (AHP) [1], fuzzy analytic hierarchy process (FAHP) [2], data envelopment analytic hierarchy process (DEAHP) [3], balanced scorecard (BS) [4], and neural network analysis method [5]. In addition, many studies have also been conducted on evaluating ERP project performance. Chen and Lin [6] proposed a fuzzy linguistic performance indicator based on network flow model to assess the performance of ERP systems. Zhan et al. [7] presented an evaluation model based on the Triangle Whiten Function. Xu [8] used an AHP method to evaluate the performance of ERP, considering the feedback and dependence factors. Razmi et al. [9] proposed a fuzzy analytic hierarchy process model (FAHP) which combines fuzzy theory with analytic hierarchy process to evaluate the ERP project. Chang et al. [10] constructed a conceptual model to measure the performance and competitive advantages of ERP from a supply chain management perspective. Hanet al. [11] used ABCD monitoring table and SPA project evaluation method to assess the ERP project. The AHP method does not consider the relationship between different indexes of control level in the index system, which weakens the objectivity of the evaluation result. BP neural network evaluation method does not determine the index weight but it requires a large amount of training samples. Although data envelopment analysis method is relatively objective, it is not suitable for qualitative analysis. ABCD monitoring table and SPA project evaluation method can cover a wide range of indexes, but the quantification of indexes is very difficult. Therefore, a more practical and objective evaluation method needs to be proposed.
The matter-element extension analysis model transforms practical problem into a formal one using matter-element and extension theory and presents grade of things through calculating the correlation between the matter-element to be evaluated and each level. In addition, the matter-element extension analysis can be also used though there are few samples. Li and Zhang [12] assessed the performance of the employees through establishing a qualitative and quantitative performance evaluation method based on the matter-element extension theory, and results shows that this method is more objective. Zhou [13] established a performance evaluation model based on matter-element and correlation function. Qualitative and quantitative evaluations on the performance of conglomerate merger were done. However, if index value exceeds the controlled field, correlation function cannot be calculated. Therefore, the traditional matter-element extension model needs to be improved.
In order to evaluate the performance of enterprise’s ERP project, a hybrid evaluation model combining ANP and improved matter-element extension model was proposed. Firstly, the evaluation index system was built; secondly, an ANP was used to determine the weight of each index which fully took the relationship among various indicators into account; and then, an improved matter-element extension model which can overcome the limitations and inadequacies of traditional matter-element extension model was proposed. Finally, taking an enterprise’s ERP project as an example, the comprehensive evaluation was done, and empirical analysis results show that this hybrid evaluation model is feasible and practical.
2. Building the Performance Evaluation Index System of ERP Project
2.1. The Implementation Effect of ERP Project
ERP system requires a large number of enterprise’s resources and changes the organizational and business mode in enterprise. The implementation of ERP project brings multiple effects on enterprise.
ERP project involves many aspects of enterprise management, such as production management, financial management, sales management, purchasing management, and inventory management [14]. Therefore, the implementation of ERP project not only relies on IT departments, but also relies on the collaboration of other departments in enterprise. Managers must be familiar with management and technical business based on ERP. Tasks are completed by professional staff, so the overall quality of employees, production efficiency, and production capacity will increase correspondingly.
Based on the financial system, financial capital operation reaches a dynamic equilibrium. Meanwhile, turnover rate of the total funds and enterprise’s return on equity improve accordingly.
The implementation of the procurement and inventory system provides many inventory analysis methods for the supply department. This system can not only ensure the procurement of purchased parts timely, but also improve inventory levels, which can reduce the backlog of inventory funds and accelerate the efficiency, timeliness, and accuracy of the inventory turnover of delivery. The implementation of procurement and inventory system will raise overall level of operational management, market share, and new customer acquisition rate.
A good operation of ERP system makes data integral, accurate, consistent, and timely. Data sharing becomes accurate and timely accordingly. ERP project can also support business decision, improve forecasting production plan, and ensure a stable and efficient operation in an enterprise.
2.2. Building the Evaluation Index System
In order to evaluate the implementation performance of enterprise’s ERP projects accurately, it is necessary to establish a reasonable evaluation index system and grading standard. Based on former analysis about the effect of implementing ERP system, we use Delphi method to build a performance index system of ERP project [5, 14]. The finance management, operations management, and customer management are criterion layer indexes. In addition, 11 extended indexes are concluded in the subcriterion layer. The index system is shown in Figure 1.
3. The Establishment of the Hybrid Evaluation Model
3.1. Basic Theory of Extension Analysis
Matter-element extension model [13, 15, 16] is based on matter-element theory and extension set theory. We can determine the level of one thing through establishing classical field, controlled field, evaluation level, and correlation function. However, there are some limitations and deficiencies.(1)When any matter-element index value beyond its controlled field, the correlation function values are unavailable, so this model cannot perform evaluation.(2)The level of one thing is obtained by calculating correlation function in this model. From the perspective of algorithm, correlation degree can be regarded as an extension of membership degree in fuzzy mathematics, so the correlation degree principle is equivalent to the maximum membership principle [15]. In some case, however, the maximum membership principle cannot reflect the ambiguity of object’s boundary. It will lose information and lead to the deviation of results.
Aiming at the limitation of (1) point the classical domain and the matter-element to be evaluated should be normalized. Aiming at the limitation of (2) point the maximum membership degree criterion should be replaced by the correlation degree criterion.
3.2. The Establishment of the Improved Matter-Element Extension Model
The basic idea of matter-element evaluation method [13, 15] is as follows: first of all, the object is divided into levels and the range of each level is determined by database or experts; secondly, determine the weight of each index; Finally, calculate the closeness degree and determine the level of matter-element.
Matter element is the logic unit of matter-element extension model; it uses an ordered triple to describe things. represent the name, the characters, and the value of one thing, respectively. The basic steps of improved matter-element extension model are as follows.Determine the classical domain, controlled field, and matter element of the object to be evaluated.
Suppose the classical domain matter element is as follows: where represents the th grade; are different characteristics of ; and are the value ranges of about , respectively, namely, the classical field.
Suppose the controlled field matter element is as follows: where represents all grades of the object to be evaluated, and are the value ranges of about , namely, the controlled field of .
Suppose the matter element to be evaluated is as follows: where represents all grades of the object to be evaluated, and are actual data of about . Normalization [16].
When an actual value of index exceeds the controlled field, the correlation function cannot be calculated, namely, the denominator is zero. In this case, matter-element and extension model cannot be used to evaluate the performance of ERP project. Therefore the classical domain and matter-element evaluation should be normalized.
Normalize the classical domain as follows:
Normalize the matter-element evaluation as follows: Weight determination
The Weight of the evaluation index directly affects the quality and feasibility of a comprehensive evaluation. So the determination of index weight is very important to the result of enterprise performance comprehensive evaluation. Since the evaluation index system is complex and related, an Analytic Network Process is used to determine the weight of each index, which can fully take all characters of indexes into account. Establish and calculate the closeness function.
Zhang [17] used closeness degree criteria instead of maximum degree of membership criteria, and made a theoretical analysis. He put forward an asymmetric closeness formula as follows: where represents the value of closeness function; represents the distance; is the weight.
The value of closeness function about each index of the matter element to be evaluated with each level is calculated as follows: where represents the distance of matter element to be evaluated related to its corresponding normalized classical field; represents the weight of evaluation index; represents the number of evaluation index. Rating.
Suppose , the matter element to be evaluated belongs to the th level.
Suppose where represents the level variable eigenvalue of . The attributive degree of the evaluated matter-element tending to adjacent levels can be judged from .
3.3. Analytic Network Process Method
Analytic Network Process [18–20] was developed from analytic hierarchy process. This method fully considers the interdependence between elements, mutual influence between elements in the same level, and dominance relation from the lower level. All elements form a network structure of ANP. An ANP consists of two parts. The first part is the control layer, including goal and criterion. In this layer, each criterion is independent and controlled only by target element. The second part is the network layer; it is controlled by control layer, and the elements in the network layer influence each other (Figure 2).
3.3.1. Basic Operation Process
Suppose the elements in the control layer are and the elements in the network layer are . consisting of . Taking as a principle and as a subprinciple, we compare the influence of from other elements in the , form the comparison matrix , and calculate the weight matrix, respectively. In a similar way, we can obtain the influence matrix of each element influencing under each principle as follows:
Taking as standard, we compare the influence degree of each element to and we can get weighted matrix as follows:
The matrix is multiplied by the matrix ; then we get the weighted matrix: , , . If the limit of matrix is convergent and only is the limit relative ranking vector of each element to the element under the principle.
3.4. The Calculation Process of the ANP-Improved Matter-Element Extension Model
In summary, the steps of comprehensive evaluation based on the ANP and improved matter-element method are as follows.
Step 1. Determine the classical domain, controlled field, and matter element to be evaluated.
Step 2. Normalize the classical domain, controlled field, and matter element to be evaluated, when the measure data of index exceeds controlled field.
Step 3. Calculate the index weight based on the ANP.
Step 4. Calculate the value of closeness function of each grade.
The means that the matter element to be evaluated belongs to the th level.
4. Case Study
A manufacturing enterprise began to implement ERP project two years ago. In order to figure out the situation of this project, an evaluation of ERP project performance was proposed. The evaluation process is as follows.
4.1. Determine the Classical Domain, Controlled Field, Matter Element to Be Evaluated, and the Corresponding Normalization
(1) Establish the classical domain. The classical fields of quantitative indicators in evaluation index system were set according to related literatures and expert [7, 14]. The classical domain of each level is described as follows. , and represent high, good, medium, and bad performance, respectively,
(2) Establish the controlled field. The classical field of each index is equal to the sum of all classical field values.
(3) Establish the matter element evaluation. According to the measured data of enterprise performance evaluation index, the matter element to be evaluated can be built. and are as follows:
We know that the value of index has exceeded the classical domain, so the closeness function cannot be calculated in traditional matter-element extension model. We should normalize the classical domain and controlled field to improve the traditional model. The normalized classical domain and controlled are below:
4.2. Calculate the Index Weight
An ANP was used to determine the weight of performance evaluation index, which can fully take the relationship between various indicators into account. Because to the calculation is complicated, we employed “Super Decision” software to calculate the weight of each index. The steps are as follows. Establish ANP network.According to the index system diagram in Figure 1, the control level and network level are developed based on the dominance and feedback relationship. The dependencies among indexes in the control level are shown in Figure 3. The relationships among indexes in network level are depicted in Figure 4. An ANP model is developed on the network structure of elements. Structure the judgment matrix. Based on the clear relationship between elements, the judgment matrixes are concluded through comparing element sets and elements, respectively, by nine-scale method. For example, the pairwise comparisons of elements in clutter B1 are conducted and the judgment matrix is shown in Table 1, and the local weight can be concluded by “Super Decision” software. Check the consistency of judgment matrix. If CR ratio exceeds 0.1, we should change the judgment matrix until all CR ratios are less than 0.1. Calculate the weighted supermatrix and the limit matrix. Calculate the weighted supermatrix and the limit matrix of indicator elements by software. The results are shown in Figures 5 and 6. Calculate the weight of indexes according to the weighted supermatrix and limit matrix. The results are shown in Table 2.
4.3. Calculate the Value of Closeness Function
Calculate the distance of the evaluated matter element related to new classical domain, just as shown in Table 3.
The value of the closeness degree of each grade is below:
4.4. Determine the Performance Rating
Since , , it is shown that the performance level of ERP project in this enterprise belongs to “high.”
4.5. Sensitivity Analysis
Sensitivity analysis is performed according to the performance evaluation index system of ERP project. When the index value or weight of index changes, its level variable characteristic value also changes correspondingly. When the index value changes by 5%, 10%, −10%, −20%, −30%, −40%, −50%, respectively, the level variable characteristic value changes as shown in Figure 2. When the weight of index changes by ±10%, ±20%, ±30%, ±40%, or ±50%, the level variable characteristic value changes correspondingly, just as the Figure 3.
As we can see from Figure 7, the project performance indexes have large effect on the evaluation results, which means that the sensitivity of is very strong. For example, when the index changes, the range of the value varies from 1.2 to 1.8. Although ERP project performance level does not change, the level gradually tends to “good” level from “high” level. With the change of index , the value changes a little from 1.5 to 1.65. It indicates that the variation of index value will not change the level which the project performance tends to or belongs to.
In Figure 8, we can know that the change of and indexes affects the value obviously, but other indexes have less effect on the value. In a word, and are sensitive indexes in the ERP project performance evaluation.
5. Conclusions
ERP system has been introduced into enterprises for many years. It is necessary to evaluate the performance of the ERP project in enterprise. However the factors which affect the performance of ERP project are complex and related. So a hybrid evaluation method of ERP project performance which considers these peculiarities was proposed. In order to analyze the performance of ERP project, an index system of comprehensive benefit evaluation including finance, customer, and operation management is established in this paper. An ANP was proposed to determine the weight of each index which fully took the relationship between various indicators into account. Due to the limitations and inadequacies of traditional matter-element extension model when performing the comprehensive evaluation, an improved matter-element extension model was proposed. And then, taking one enterprise’s ERP project as an example, the comprehensive evaluation was done. The empirical analysis results show the performance of ERP project in our case belongs to “high” level and our proposed hybrid evaluation model is feasible and practical. Finally, a sensitivity analysis is performed to find sensitive index in the ERP project evaluation. The analysis results show that total asset turnover ratio and data transfer efficiency are sensitive indexes, so this enterprise should pay more attention to them.
Acknowledgments
This study is supported by the National Natural Science Foundation of China (Grant no. 70971038) and the Humanities and Social Science project of the Ministry of Education of China (Project no. 11YJA790217). The authors are grateful to the editor and anonymous reviewers for their suggestions on improving the quality of the paper.