Application of Set-Valued Statistical Methods for Excellent Performance Evaluation

Ji, Xuefang; Wang, Wei; He, Jun; Wang, Ziyi

doi:https://doi.org/10.1155/2022/7904743

Mobile Information Systems

On this page

Abstract Introduction Results and Analysis Conclusion Data Availability Conflicts of Interest References Copyright Related Articles

Special Issue

Edge Intelligence in Internet of Things using Machine Learning 2022

View this Special Issue

Research Article | Open Access

Volume 2022 | Article ID 7904743 | https://doi.org/10.1155/2022/7904743

Application of Set-Valued Statistical Methods for Excellent Performance Evaluation

Xuefang Ji,¹Wei Wang,¹Jun He,¹and Ziyi Wang²

Academic Editor: Abid Yahya

Received06 Apr 2022

Accepted23 May 2022

Published29 Jun 2022

Abstract

The performance of an organization depends on the model and methodology adopted by the organization in carrying out casual activities. The high-performance model, which is widely recognized internationally, is an effective method and tool for the comprehensive performance management of organizations. It is used to measure how much value each employee brings to a firm in terms of improved revenue, as well as overall employee return on investment when compared to industry norms. The overall goal of the performance review process is to improve a team’s or organization’s performance in order to boost customer satisfaction. However, there are many problems with the current performance evaluation methods. For instance, the currently adopted approaches are fully reliant on the assessment team’s human scoring and lack the processing and detection of scoring results, which significantly reduce the objectivity of the evaluation outcomes. In order to sort out this issue, this paper constructs an excellent performance evaluation assessment method based on set-valued statistics, which obtains the final comprehensive score value by performing set-valued statistical processing on the scores of each evaluator and detects the evaluation results by calculating the confidence level. Hereby, the case analysis is conducted and Excel modeling to assist quantitative evaluation work is adopted. The analysis results reveal that the proposed technique is better than the earlier approaches.

1. Introduction

The term “performance evaluation” refers to a well-structured and efficient way of assessing an employee’s work and outcomes based on his or her job responsibilities. It is used to figure out how much value each person brings to a firm in terms of improved revenue, as compared to industry standards and overall employee investment return. In other words, we can also say that it is a mechanism for recognizing and reviewing an employee’s effort over a period of time [1]. A successful performance measurement system (PMS) is a set of performance indicators that stipulate valuable information to help a company in managing, planning, regulating, and executing its operations [2]. In general, every employee should be rated yearly on the basis of his/her work anniversary, but it depends on his/her performance that either he/she is promoted or given an appropriate wage boost distribution. Employees are more self-aware of their performance measures as a result of the performance evaluation, which provides them with regular feedback.

A performance evaluation system or PMS has several important purposes for an organization [3, 4]. First and foremost, the basic purpose of the entire performance review process is to enhance the way a team or organization performs in order to increase customer satisfaction. To know about the work of an employee that he/she has done over a period of time and in a specific area of development, it is necessary to collect a periodic performance report. Regular performance evaluations can also aid in determining the extent of an employee’s professional progression and the amount of motivation with which he/she contributes to the organization’s success [5].

The performance excellence model, widely recognized internationally, is an effective method and tool for comprehensive performance management of an organization [6]. Under the situation of global economic integration, the implementation of a high-performance model has become one of the effective ways for countries to enhance the competitiveness of enterprises [7]. It is very necessary to study and discuss the methods of high-performance evaluation, to further improve the science of high-performance evaluation results.

Keeping the different challenges of existing models and methods in mind, this study develops an effective performance evaluation assessment technique based on set-valued statistics that calculates the final comprehensive score value from the scores of each evaluator and identifies the evaluation outcomes by calculating the confidence level. In this way, a case analysis is carried out and Excel modeling is used to aid quantitative evaluation work. The analysis results reveal that the proposed model is way better than the earlier models and methods.

The remaining structure of the paper is organized as follows: Section 2 discusses the analysis of excellent performance evaluation methods. It begins by providing a thorough explanation of high-performance assessment methodologies before assessing the high-performance system. An excellent performance evaluation method based on set-valued statistics is presented in Section 3. This section further deliberates the building of a quantitative evaluation model for high-performance based on set-valued statistics, case analysis of quantitative evaluation of excellent performance based on set-valued statistics, and building up a set-valued statistical model. Section 4 is about the results and analysis that gives the outcomes of the proposed model. Finally, the whole themes, methodology, and findings of the paper are recapped in Section 5.

2. Analysis of Excellent Performance Evaluation Methods

2.1. Overview of High-Performance Evaluation Methods

The evaluation index system specified in the “Excellent Performance Evaluation Criteria (2019-2020)” is composed of organizational overview, leadership, strategy, customers, measurement analysis and knowledge management, labor, operations, and results, all of which except overview have corresponding assignments [8, 9]. The assignments are shown in Table 1.

The specific scoring method of the excellent performance evaluation is to form an evaluation team composed of several evaluators, analyze the abovementioned secondary indicators according to the application materials and on-site inspection of the participating units, and give a compliance score one by one after forming a unified opinion. The compliance score is based on a 5% difference between 0 and 100 percent, with a total of 21 possible score values. According to the collegiality rules, the middle score is used when the difference between the review groups is less than or equal to 15%; when the difference between the review groups is 20% or 25%, the decision is discussed or the middle score is used; and when the difference between the review groups is equal to or greater than 30%, mandatory discussion and decision are required. At last, the final evaluation result of the evaluated unit is summed up by multiplying the compliance score of each secondary index by the index score.

2.2. Assessment of High-Performance Evaluation Methods

The current excellent performance evaluation method is a scoring model based on multilevel factor analysis [10, 11], and its basic ideas are worthy of recognition:(i)According to the evaluation of excellent performance, the evaluation team is required to give quantitative evaluation results, but since all evaluation indicators are qualitative indicators, it is a more reasonable method for the evaluator to quantify the scores of each indicator at this stage.(ii)On the basis of a detailed analysis of various evaluation factors, the model builds a multilevel and multifactor evaluation index system, which makes the scoring method of excellent performance evaluation more refined.(iii)The model assigns different scores to each evaluation index, which is in line with the multilevel and multiattribute characteristics of high-performance evaluation factors.

However, there are still some problems with the current excellent performance evaluation methods, some of which are listed as follows:(i)To undertake collaborative discussion and analyses, a group of evaluators is formed by this method. This kind of “face-to-face” evaluation cannot avoid the mutual influence and confrontation between the evaluators. The approach of obtaining the median number or the necessity of essential consistency established by college regulations substantially affects the scientificity of the assessment procedure, especially when the evaluators disagree on the ratings.(ii)This method completely relies on the human scoring of the evaluation team and lacks the processing and detection of the scoring results, which will greatly weaken the objectivity of the evaluation results.(iii)This method sets the index conformity score as 21 gradable values with a score difference of 5%, which is highly subjective and arbitrary.

3. Excellent Performance Evaluation Method Based on Set-Valued Statistics

Although excellent performance evaluation indicators are qualitative indicators, fuzzy mathematics can quantify qualitative indicators to a certain degree of accuracy. Therefore, fuzzy mathematical methods can be introduced into the evaluation work to quantify empirical judgments. This process is discussed in detail in the following sections.

3.1. Building a Quantitative Evaluation Model for High-Performance Based on Set-Valued Statistics

In the existing research results, many researchers proposed using the fuzzy comprehensive evaluation method to quantitatively analyze the qualitative indicators [12, 13]. The fuzzy comprehensive evaluation method requires the evaluator to quantify the evaluation opinion with a certain membership degree of a certain number or a fuzzy number when evaluating a certain index, which is unscientific in the author’s opinion [14, 15]. Firstly, it is almost impossible to expect the evaluator to make precise assessments on each evaluation index due to the complexity of objective elements and the evaluator’s limited knowledge and expertise. The ambiguity of the indicators was removed early. Secondly, the fuzzy comprehensive evaluation method cannot measure the consistency of all evaluators’ judgments on the same index [16].

Keeping the limitations of the earlier approaches in mind, we proposed the fuzzy mathematical method of set-valued statistics for the quantitative evaluation of high performance. Set-valued statistics is an extension of classical statistics and fuzzy statistics. Classical statistics get a definite point in the phase space in each experiment, while set-valued statistics get a fuzzy subset in each experiment. Therefore, inaccurate judgments can be handled by set-valued statistics, and a variety of different opinions can be conveniently collected, thereby reducing random errors in expert judgments. The different steps of the proposed method are as follows: Step 1: in the first step, the evaluator calculates the conformity score for the evaluation index. Under the set-valued statistical method, the evaluator does not give a certain membership degree of a definite number or a fuzzy number, but a fuzzy subset, which is represented by an interval estimation value, denoted as , where k is the rater k, . In the evaluation process, the index with a greater degree of certainty of the judgment gives a result of a score with a smaller interval while an index with a low degree of certainty of the judgment gives a result of a score with a larger interval. If there are n evaluators, a set-valued statistical sequence of index evaluation can be obtained as described in Step 2: the second step is to perform set-valued statistical processing on the index conformity score. The calculation formula of the index evaluation is represented in Step 3: the third step is to check the consistency of all evaluators’ judgments on the same index by calculating the confidence level. The confidence level (p) can be calculated using

In (3), the value of can be calculated using the following equation:

The closer p is to 1, the more concentrated the index’s assessment value is and the more consistent the judgments of all evaluators on the same index are. Besides this, the closer p is to 0, the more discrete the evaluation value of the index is and the lower the consistency of all evaluators’ judgments on the same index is.

3.2. Case Analysis of Quantitative Evaluation of Excellent Performance Based on Set-Value Statistics

In this section, we carried out a case analysis of the quantitative evaluation of excellent performance on the basis of the set-valued statistics. Suppose that a unit is being evaluated for excellent performance. The evaluation team consisted of 5 people, and the scores were processed according to the set-valued statistical model.

In the first step, five evaluators scored the conformity of the evaluation indicators. The scores of some indicators are listed in Table 2.

As it can be seen from the ratings, the evaluators can give grades according to their own professional judgment. The scoring result does not have to be a specific score ranging from 0 to 100 percent with a 5% difference, but it can be an interval-estimated value, which is consistent with the characteristics of qualitative indicators that quantify scoring ambiguity. In addition, the range of the rating interval of the evaluator can be large or small, and it is not required to be a multiple of 5%. For example, the difference between the scores of evaluator B for the above three indicators is 13% at the maximum and 4% at the minimum. Rater E’s rating interval differed by a maximum of 20%.

The second step is to perform set-valued statistical processing on the index conformity score. Taking the index evaluation of “product and process results” as an example, the calculation of the index conformity evaluation value is given in

The third step is to calculate the confidence. Taking the evaluation of the “product and process results” index as an example, its confidence level (p) is calculated as shown in

The fourth step is index scoring. Multiplying the index’s compliance score by the index score is the score of the evaluation index, and the sum is the final evaluation result of the evaluated unit. The scores of some indicators and the set-valued statistical processing results are shown in Table 3.

From the data in Table 3, it can be seen that except for the indicator of “customer results,” the evaluators’ judgment degree of evaluation of other indicators is higher than 0.98, indicating that the evaluators’ judgment opinions are relatively consistent. Since the evaluators have certain differences in the judgment of the “customer’s result” indicator, if the evaluation team believes that the confidence level of 0.9762 does not reach an acceptable level, the evaluators can be organized to reevaluate the indicator.

Evaluators C and E in the example above believe that the conformance of the “labor’s outcome” index could not be determined for whatever reason, and thus, they did not provide a score. If most estimators give ratings, then it does not affect the final evaluation of the indicator. This point further illustrates the scientificity and operability of the set-valued statistical method.

3.3. Set-Valued Statistical Model Building

The calculation formula of the set-valued statistical processing method is relatively complicated, and the workload of the calculation is also overloaded, which is not convenient for the development of the evaluation. Therefore, in this paper, we utilized Excel to establish a set-valued statistical model to assist the calculation, which can improve work quality and work efficiency.

First, we created two worksheets in the workbook and named them “Statistical Processing of Set Values” and “Calculation Results”; then, Excel’s VBA editor is used to create a macrofile (named “jizhi”). The overall process of statistical processing is represented by Algorithm 1.

	Sub jizhi ()
	Dim s
	Dim As String
	h = ActiveSheet.UsedRange.Rows.Count
	l = ActiveSheet.UsedRange.Columns.Count
	s1 = 0
	For i = 1 To h
	For j = 1 To l - 1
	s1 = ActiveSheet.Range(Cells(i, j + 1).Address).Value ^ 2 - ActiveSheet.Range(Cells(i, j).Address).Value ^ 2 + s1
	s2 = ActiveSheet.Range(Cells(i, j + 1).Address).Value - ActiveSheet.Range(Cells(i, j).Address).Value + s2
	Next j
	Next i
	s = (s1/s2)/2
	Sheets(“Calculation Results “).[$b$2].Value = s
	For m = 1 To h
	For n = 1 To l − 1
	= (ActiveSheet.Range(Cells(m, n + 1).Address).Value − s) ^ 3 − (ActiveSheet.Range (Cells (m, n).Address).Value − s) ^ 3 +
	Next n
	Next m


	Sheets(“result”).[$b$3].Value = p
	End Sub

4. Results and Analysis

This section discusses the results attained via the utilized method. In this section, we input the scores of each evaluator in the “Set-Valued Statistical Processing” worksheet and then execute the macro-operation to get the processing results in the “Calculation Results” worksheet. For example, a screenshot of the calculation process and the result of the “product and service result” indicator in the case are shown in Figures 1 and 2.

The scores of some indicators and the set-valued statistical processing results are shown in Figure 3. Table 2 (Section 3.2) has the real data for these indicators; however, the “omitted” column has been deleted because it was empty.

On the basis of the data in Table 2 and Figure 3, the confidence level and conformity index are calculated as shown in Figure 4. All the details about the calculation are given in Section 3.2 of the paper.

Before using Excel’s macro to write the set-valued statistics function, it is necessary to perform data entry and execution operations by indicators one by one and the operation steps are slightly cumbersome. To solve this problem, there is a need for a professional programming language in order to use for coding. Considering the stability and portability of the program, Microsoft’s language is used to write a set-valued statistical calculation program. First, the number of columns for the set-valued statistical calculation program is selected for processing the data, the data are entered once again, and then the operation is executed to get the result. However, professional programming requires a certain foundation, and the threshold is slightly higher. In contrast, as a commonly used office software with a high market share, it is easy to use Excel’s own macros to write set-valued statistics, which is also recommended in this paper.

5. Conclusion

In this paper, the set-valued statistics method is introduced into the performance evaluation work, which develops a quantitative evaluation method of performance excellence based on the set-valued statistics. Compared with the existing excellent performance evaluation methods, this method has a number of advantages as follows: firstly, this method allows the evaluators of the evaluation team to express their true opinions independently and does not require that the opinions of the evaluators must be completely consistent, as long as the confidence of the evaluation results reaches an acceptable level, which reflects the effectiveness of the scientific method. Secondly, this method performs set-value statistical processing on the scores of each evaluator and detects the evaluation results by calculating the confidence level, which ensures the objectivity and credibility of the evaluation results. Thirdly, this method does not require each evaluator to make accurate judgments on each evaluation index but only requires a fuzzy subset, which is in line with the characteristics of human qualitative judgment and reflects the rationality of the method. Finally, this study uses Excel to establish a set-valued statistical model to assist the calculation, which can improve the work quality and work efficiency and guarantee the operability of the method.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

References

S. H. Appelbaum, D. Nadeau, and M. Cyr, “Performance evaluation in a matrix organization: a case study (Part Two),” Industrial and Commercial Training, vol. 40, 2008.
View at: Google Scholar
S. Tangen, “Analysing the requirements of performance measurement systems,” Measuring Business Excellence, vol. 9, 2005.
View at: Google Scholar
M. G. Brown, Baldrige Award Winning Quality: How to Interpret the Baldrige Criteria for Performance Excellence, CRC Press, Boca Raton, FL, USA, 2017.
N. A. Lawrence and M. S. Hammoud, “Strategies to implement the baldrige criteria for performance excellence,” International Journal of Management Excellence, vol. 9, no. 1, p. 1040, 2017.
View at: Publisher Site | Google Scholar
B. Shen, L. Price, and H. Lu, “Energy audit practices in China: national and local experiences and issues,” Energy Policy, vol. 46, pp. 346–358, 2012.
View at: Publisher Site | Google Scholar
J. Yang, Y. Cui, F. Song, and T. Lei, “Infrared small target detection based on non-overlapping patch model via l0-l1 norm,” Electronics, vol. 9, p. 1426, 2020.
View at: Publisher Site | Google Scholar
J. V. Pascual Ivars and J. M. Comeche Martínez, “The effect of high performance work systems on small and medium size enterprises,” Journal of Business Research, vol. 68, no. 7, pp. 1463–1465, 2015.
View at: Publisher Site | Google Scholar
J. L. Vajda, Evaluation and Selection of Oil Shale Extraction Methods Using Multi-Attribute Utility Theory, The George Washington University, Washington, DC, USA, 2021.
J. R. Evans and W. M. Lindsay, Managing for Quality and Performance Excellence, Cengage Learning, Boston, MA, USA, 2013.
X. Gu and Q. Zhu, “Fuzzy multi-attribute decision-making method based on eigenvector of fuzzy attribute evaluation space,” Decision Support Systems, vol. 41, no. 2, pp. 400–410, 2006.
View at: Publisher Site | Google Scholar
A. Rikkiev and S. Makinen, “Success factors for technology integration convergence collaborations: empirical assessment,” in Proceedings of the PICMET’09-2009 Portland International Conference on Management of Engineering & Technology, pp. 374–382, Portland, OR, USA, August 2009.
View at: Google Scholar
G. Wang, C. Xiao, Z. Qi, F. Meng, and X. Liang, “Development tendency analysis for the water resource carrying capacity based on system dynamics model and the improved fuzzy comprehensive evaluation method in the Changchun city, China,” Ecological Indicators, vol. 122, Article ID 107232, 2021.
View at: Publisher Site | Google Scholar
X. Wang, S. Li, Z. Xu et al., “An interval fuzzy comprehensive assessment method for rock burst in underground caverns and its engineering application,” Bulletin of Engineering Geology and the Environment, vol. 78, no. 7, pp. 5161–5176, 2019.
View at: Publisher Site | Google Scholar
S. Yang, M. Guo, X. Liu, P. Wang, Q. Li, and H. Liu, “Highway performance evaluation index in semiarid climate region based on fuzzy mathematics,” Advances in Materials Science and Engineering, vol. 2019, Article ID 6708102, 7 pages, 2019.
View at: Publisher Site | Google Scholar
Q. Qu, K.-Y. Chen, Y.-M. Wei, Y. Liu, S.-B. Tsai, and W. Dong, “Using hybrid model to evaluate performance of innovation and technology professionals in marine logistics industry,” Mathematical Problems in Engineering, vol. 2015, Article ID 361275, 8 pages, 2015.
View at: Publisher Site | Google Scholar
Z. Zhaoning, M. Na, and Z. Peng, “Based on the fuzzy set-valued statistics and the fuzzy mathematics theory in air traffic control system safety appraisal application,” Physics Procedia, vol. 33, pp. 511–520, 2012.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2022 Xuefang Ji et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

123

Downloads

217

Citations