Scheduling Performance Evaluation of Logistics Service Supply Chain Based on the Dynamic Index Weight

Abstract

Scheduling is crucial to the operation of logistics service supply chain (LSSC), so scientific performance evaluation method is required to evaluate the scheduling performance. Different from general project performance evaluation, scheduling activities are usually continuous and multiperiod. Therefore, the weight of scheduling performance evaluation index is not unchanged, but dynamically varied. In this paper, the factors that influence the scheduling performance are analyzed in three levels which are strategic environment, operating process, and scheduling results. Based on these three levels, the scheduling performance evaluation index system of LSSC is established. In all, a new performance evaluation method proposed based on dynamic index weight will have three innovation points. Firstly, a multiphase dynamic interaction method is introduced to improve the quality of quantification. Secondly, due to the large quantity of second-level indexes and the requirements of dynamic weight adjustment, the maximum attribute deviation method is introduced to determine weight of second-level indexes, which can remove the uncertainty of subjective factors. Thirdly, an adjustment coefficient method based on set-valued statistics is introduced to determine the first-level indexes weight. In the end, an application example from a logistics company in China is given to illustrate the effectiveness of the proposed method.

1. Introduction

Service supply chain is a new trend in supply chain research [1], and logistics service supply chain (LSSC) is a type of service supply chain centered on the cooperation of logistics service capacity [2, 3]. The main structure of the LSSC is the mode where the functional logistics service providers (FLSPs) flow to the logistics service integrator (LSI) and then to the manufacturers or retailers [2, 3]. FLSPs consist of traditional functional logistics enterprises, such as transportation and storage enterprises, among others, whose service function is simple and standardized, and the business is limited within a certain area; they are integrated as the suppliers by the LSI when a domestic or international logistics service network is established. For instance, as a LSI, Baogong logistics company in China integrates over 500 warehousing companies and 1200 highway transport companies as their FLSPs to provide personalized logistics services for many world famous companies such as P&G and Unilever. Therefore, managing logistics service operation has become a core content of LSSC management.

Scheduling is an important part of service operations management, which is the activity of allocating and coordinating the enterprise’s resources, such as the workforces, machines, vehicles, and material, and stipulates the tasks to be executed by employees within a certain period [4, 5]. Scheduling has an important influence on enterprise performance by determining the order lead time, delivery flexibility, machine and material usage, and staff workload [4, 5]. Reference [6] pointed out that a process can be managed only when its performance can be measured. Therefore, studying the evaluation method of scheduling performance can not only compare the performance within each period, but also identify the problems during scheduling so as to improve the scheduling process. There have already been studies focusing on performance evaluation of LSSC. Reference [7] studied the effect of time scheduling on LSSC scheduling. But existing articles only consider the results evaluation [8], using KPIs, for instance, rather than consider the three phases: before scheduling, during scheduling, and after scheduling.

In terms of scheduling performance evaluation method, much scheduling research concentrates on improving existing methods for specific classes of scheduling situations using techniques from operations research or mathematical programming and, lately, also from artificial intelligence [4, 5, 9, 10]. The majority of these studies pay attention to the performance evaluation of scheduling in a single enterprise or sector. However, different from the scheduling performance evaluation of an individual enterprise, for LSSC, the following problems should be paid more attention to and should be solved.(1)LSSC is composed of many members (e.g., one LSI and many FLSPs), and the impact of scheduling activities from these members on the performance should be considered. So, how can we give full consideration about the different demands of these members when the LSSC performance indexes are designed?(2)Different from solid product, service contains the characteristics of customer influence, intangibility, and inseparability; the scheduling indexes of LSSC may include both quantitative index and qualitative index ones. How to consider the qualitative indexes and quantitative indexes as a whole?(3)Different from general project performance evaluation, scheduling activities are usually continuous and multiperiods. The activities of the previous period will have an impact on those of the next one. For example, along with the increasing of scheduling times, the understanding of decision-maker for each index weight may change [11], which leads to the adjustment of index weight. Therefore, index weight has the feature of dynamic change, so how to select a proper dynamic evaluation method is an important issue.

In this paper, the performance evaluation index system of LSSC scheduling is proposed, and then an evaluation method based on dynamic index weight is adopted. In this method, firstly a multiphase dynamic interaction method is introduced by considering the difficulty of converting qualitative data into quantitative data. As a result, the quantitative results are more objective. Second, as there are many second-level indexes and dynamic variation is required, the maximum attribute deviation method is introduced to determine weight of second-level indexes. Thirdly, as there are few first-level indexes, an adjustment coefficient method based on set-valued statistics is introduced to determine the weight index. An application example from a logistics company in China illustrates that the method proposed is more suitable for scheduling performance evaluation of LSSC.

This paper is organized as follows. In Section 2, the literature review on evaluation methods of scheduling performance is presented. Section 3 proposes the performance evaluation index system of LSSC scheduling. Section 4 gives the evaluation method of dynamic index weight. Section 5 provides the innovations and advantages of the model proposed. The application in a logistics company to prove the effectiveness of this method is provided in Section 6. The last two sections put forward the main conclusions and future insights.

2. The Literature Review

Research about scheduling performance evaluation on LSSC mainly includes two aspects: one is the design of evaluation index system; the other is the evaluation method. Previous studies of supply chain performance mostly focused on manufacturing supply chain, but the research on SSC is an emerging field now. Thus, this section will review the performance evaluation research not only on manufacturing supply chain but also on SSC. The research on performance evaluation methods about scheduling management and supply chain management (SCM) will be discussed as well.

2.1. Performance Evaluation System of Supply Chain

Since the 1990s, the research about manufacturing supply chain performance evaluation has begun and gained many achievements [12]. The authoritative research institute of supply chain, Supply Chain Council (SCC), in 1996 presented 13 indexes about performance evaluation in supply chain operations reference (SCOR). Reference [13] established an evaluation system based on resources, output, and flexibility. Reference [14] studied why and how the SCM influenced the organization’s time performance. Reference [15] put forth the performance evaluation system based on four main processes of supply chain, planning, purchasing, assembling, and delivering. On the basis of the SCOR model, [16] proposed the performance evaluation system from the perspective of customers, internal processes, and shareholders. Reference [17] built up the performance evaluation system from the view of financing, customers, operation procedure, organization learning, and growth by adopting the balanced scorecard.

With the development of SSC, more and more studies focus on the performance evaluation of SSC. Reference [18] made the definition of SSC from the perspective of professional services outsourcing and established a comprehensive performance evaluation system. Reference [19] explored the integration SCOR model with service business and developed a reference model of performance evaluation for service organizations. Reference [20] made a review on performance evaluation issues of SSC and constructed the performance evaluation system from the aspects of the strategic, tactical, and operational levels.

In the field of LSSC, based on the characteristics of LSSC, [21] designed the performance evaluation system on the procedure joint process of LSSC and used an ANP method. Reference [7] proposed a time scheduling model and explored the effect of time delay coefficient and cost coefficient on the performance of LSSC.

When considering the evaluation index, the qualitative indexes and quantitative indexes are different. Reference [13] stated that qualitative evaluations were vague and difficult to utilize in any meaningful way. Thus, quantitative performance measures were often preferred over such qualitative evaluations. However, the chosen numerical performance measure may not adequately describe the system's performance [22]. Therefore, many studies use both qualitative and quantitative indicators [23–25]. In these papers, the qualitative indexes are evaluated by questionnaire and they ignore the stability and the consistency of the respondents.

2.2. Scheduling Performance Evaluation Method

Moreover, the methods of scheduling performance evaluation are investigated by many scholars [26–28], but these studies are mostly emphasized by using the method of operation research or optimization algorithm to improve the performance [4, 5, 10] rather than performance evaluation. Reference [29] optimized the performance evaluation method of production scheduling within a workshop, but the method was not extended to supply chain environment. Reference [8] conducted an empirical study about performance evaluation indexes of scheduling with a three-part questionnaire.

Dynamic evaluation method used in this paper is also applied in many fields. References [30, 31] examined air quality model’s changes in pollutant concentrations with dynamic evaluation method. Furthermore, the performance evaluation methods of supply chain are enriching with the development of the evaluation theories. Basically, there are several methods including balanced scorecard [17, 32], data envelopment analysis [33, 34], analytic hierarchy process [35, 36], analytic network process [3], and fuzzy evaluation method [37]. Different approaches can be applied to different environments.

From the literature review, it can be found that performance evaluation research of supply chain has gained abundant achievements, but the research on scheduling performance evaluation of LSSC has not been reported before. Therefore, it is necessary to analyze the characteristics of the scheduling performance evaluation in depth and explore the evaluation index system and evaluation methods, which will provide scientific reference for improving the effectiveness and efficiency of LSSC scheduling.

3. The Scheduling Performance Evaluation Index System of LSSC

The goal of LSSC operation management is to provide the customers with the best logistics service through the cooperation of LSI and FLSPs. LSSC scheduling means that LSI manages the logistics service capacities of different FLSPs uniformly and makes various service capacities coordinate with each other so as to meet customers' logistics service demand. LSSC scheduling involves many aspects, including order allocation scheduling, time flexibility scheduling, and process scheduling. In order to complete the scheduling, suitable organization and coordination are required through the information communication, human resource allocation, relationship coordination, and other activities.

To evaluate the scheduling performance of LSSC, the evaluation index system should be given, because it provides significant value to users, which can tell the users how valuable the evaluated objects are [38, 39]. Some scholars have already proposed several performance evaluation systems of manufacturing supply chain, but these systems cannot be completely duplicated into service supply chain [40]. Therefore, the corresponding evaluation index system should be presented based on the characteristics of LSSC scheduling.

First of all, the evaluation index system of LSSC scheduling should take the hierarchy into consideration. Reference [41] pointed out that supply chain management issues covered a wide range of enterprise activities from the strategic through the tactical to the operational levels. Therefore, evaluation index system in this study will be divided into three levels, strategic environment level, operating process level, and scheduling results level, corresponding to strategic level, tactical level, and operational level, respectively.

Second, the index system should reflect the characteristics of supply chain and service. The essence of SCM is integration and coordination among members, while scheduling is the bridge of connecting upstream and downstream in SSC, whose performance evaluation should involve the coordination ability and cooperation effect [42–44]. Additionally, the service characteristics of SSC should be considered. References [45, 46] verified that the dimensions of performance evaluation for manufacturing supply chain are insufficient to evaluate the performance of SSC. Service is intangible and service delivery is quite different from product delivery, which should be considered in the index system design.

Third, the index system should take into account the different members’ demand. According to [15], the main reason why few enterprises succeed to maximize the overall supply chain profit through integration and coordination is that they fail to be integrated completely and do not share performance evaluation systems among them. Thus, measurements should be shared and manipulated by all supply chain members. There are three principal members in LSSC, LSI, FLSP, and customers. The LSI and FLSP focus on all the three levels, while the customers pay more attention to scheduling results level.

According to the consideration above, the scheduling performance evaluation index system of LSSC is proposed as in Table 1. It must be noted that for the indexes needed by LSI and FLSP, both LSI and FLSP should be joint during evaluation, and the final index value should be determined by LSI and FLSP jointly.

Table 1 

The scheduling performance evaluation index system of LSSC.

4. Scheduling Performance Evaluation Method Based on Dynamic Index Weight

In the section of introduction, there are problems in LSSC scheduling performance evaluation that should be solved. The first is how to quantify the qualitative indexes; the second is how to deal with the continuous and multiperiod characteristics of LSSC scheduling activities; and the third is how to implement the dynamic change of index weight. Thus, the scheduling performance evaluation method of LSSC based on the dynamic index weight is proposed. Section 4.1 proposes an improved method of multiphase dynamic interaction to quantify the qualitative indexes. Section 4.2 uses an improved attribute deviation maximization method to determine the weight of second-level indexes. In Section 4.3, we propose an adjustment coefficient method based on set-valued statistics to adjust the first-level indexes’ weight. Section 4.4 shows the main application process of this method.

The notations involved in the method are shown in Table 2.

Table 2 

Notations used in the method.

4.1. Quantify the Qualitative Indexes Using the Method of Multiphase Dynamic Interaction

This method is applicable to quantify the qualitative index by multiphase dynamic interactive expert scoring. The thought is described as in the following: let some experts rate the qualitative indexes with several rounds. Every round can be regarded as a revision for last round till it reaches a relatively stable and consistent level. Suppose that there are 𝑚 experts scoring for 𝑛 qualitative indexes, and the higher the score, the better the performance of the index for both cost-type indexes and benefit-type indexes. The index set evaluated is 𝐗={𝑥₁,𝑥₂,…,𝑥_𝑛} and the evaluator group set is 𝐒={𝑠₁,𝑠₂,…,𝑠_𝑚}. The interval of scoring is [0,10] and the score matrix is 𝑃=(𝑝_𝑖𝑗)_𝑛×𝑚.

4.1.1. Multiphase Dynamic Interaction and the Decision of Interaction Coefficient

The purpose of multiphase dynamic interaction is to give the evaluators a chance of revising their grading according to the group information of last round. 𝑃^𝑡(𝑡=1,2,…,𝑙) is assumed as the score matrix of the 𝑡th round. In the process of multiphase dynamic interaction, the variation of an evaluator’s score between two continuous rounds can reflect the influence by the group information of last round. 𝑎^𝑡_𝑗 can be used to indicate the similarity between the 𝑗th evaluator with other group members in the 𝑡th round, 𝑎^𝑡_𝑗=1𝑚−1^𝑚_{𝑘=1,𝑘≠𝑗}cos𝜃^𝑡_𝑗𝑘−cos𝜃^𝑡−1_𝑗𝑘, (1) where cos𝜃^𝑡_𝑗𝑘=(𝑝^𝑡_𝑗,𝑝^𝑡_𝑘)/|𝑝^𝑡_𝑗||𝑝^𝑡_𝑘|=∑^𝑛_𝑖=1𝑝^𝑡_𝑖𝑗𝑝^𝑡_𝑖𝑘/∑^𝑛_𝑖=1(𝑝^𝑡_𝑖𝑗)²∑^𝑛_𝑖=1(𝑝^𝑡_𝑖𝑘)².

During the course of different interaction phases, the influence of a group member is varied with score matrix. Thus, 𝑢^𝑡_𝑗 is used to indicate the interaction coefficient of the 𝑗th evaluator in the 𝑡th round as follows: 𝑢^𝑡_𝑗=1+𝑎^𝑡_𝑗∑^𝑚_𝑗=11+𝑎^𝑡_𝑗. (2)

The interaction coefficient vector of the 𝑡th round is 𝐮^𝐭=(𝑢^𝑡₁,𝑢^𝑡₂,…,𝑢^𝑡_𝑚).

4.1.2. The Aggregation of Scoring Information

The purpose of the aggregation is to obtain the comprehensive score of each index after the 𝑡th round scoring. In order to balance every evaluator’s opinion, the minimal deviation with the members’ scoring is used as the final score. By solving the following programming model, the final score of the 𝑖th index in the 𝑡th round can be reached as 𝑝_𝑖^𝑡∗(𝑖=1,2,…,𝑛;  𝑡=1,2,…,𝑙), where min𝜋𝑝_𝑖^𝑡∗=^𝑛𝑚_𝑖=1𝑗=1𝑢^𝑡_𝑗𝑝^𝑡∗_𝑖𝑗−𝑝^𝑡_𝑖𝑗²s.t.min_𝑗𝑝^𝑡_𝑖𝑗≤𝑝^𝑡∗_𝑖𝑗≤max_𝑗𝑝^𝑡_𝑖𝑗𝑖=1,2,…,𝑛,𝑗=1,2,…,𝑚. (3)

4.1.3. The Stability and Consistency of Group Information

After several rounds of interscoring, the opinions of group members are known to each other. So the group information tends to be relatively stable and consistent. The stability means the invariance of group information in consecutive rounds and the consistency represents the invariance between different members within the same round. According to the two indexes, the scoring termination condition can be determined.

The indexes stability for the  𝑡th round is calculated as follows: 𝜐^𝑡1=1−𝑚𝑛^𝑛𝑚_𝑖=1𝑘=1𝑝^𝑡_𝑖𝑘−𝑝^𝑡−1_𝑖𝑘². (4)

The stability vector is labeled as 𝝊=(𝜐¹,𝜐²,…,𝜐^𝑙).

The index consistency for the𝑡th round is calculated as follows: 𝜔^𝑡1=1−𝑚𝑛^𝑛𝑚_𝑖=1ℎ=1𝑝^𝑡_𝑖ℎ−𝑝_𝑖^−𝑡2, (5) where 𝑝_𝑖^−𝑡=(1/𝑚)∑^𝑚_𝑘=1𝑝^𝑡_𝑖𝑘 and the consistency vector is labeled as 𝝎=(𝜔¹,𝜔²,…,𝜔^𝑙).

In order to determine the termination conditions of interscoring, two thresholds for stability (𝜀) and consistency (𝜂) are given firstly. If there exists 1−𝜐^𝑡<𝜀, (6) we think that it passes the stability test. Meanwhile, according to the indexes consistency in the 𝑡th round, if there exists 1−𝜔^𝑡<𝜂 (7) we believe that it passes the consistency test. When both the stability and the consistency tests are satisfied, the multiphase dynamic interaction can be stopped. In practical application, the stability is more emphasized, so the threshold of consistency can be looser than that of stability.

4.1.4. The Aggregation of the Final Score Result

The comprehensive score of each index in different rounds 𝑝_𝑖^𝑡∗(𝑖=1,2,…,𝑛;  𝑡=1,2,…,𝑙) will have an impact on the final score result. For these impacts, the induced ordered weighted averaging (IOWA) [47] operators are adopted to aggregate the index score in different rounds.

Definition 1. Set ⟨𝑣₁,𝑓₁⟩,⟨𝑣₂,𝑓₂⟩,…,⟨𝑣_𝑙,𝑓_𝑙⟩ as 𝑙 two-dimensional arrays. Let ℎ_𝑤(⟨𝑣₁,𝑓₁⟩,⟨𝑣₂,𝑓₂⟩,…,⟨𝑣_𝑙,𝑓_𝑙⟩)=∑^𝑙_𝑖=1𝑤_𝑖𝑎_{𝑣−index(𝑖)}; then the function ℎ_𝑤 is regarded as 𝑙 dimensional IOWA operator, and 𝑣_𝑖 is induced component of 𝑓_𝑖, where 𝑣−𝑖𝑛𝑑𝑒𝑥(𝑖) stands for the subscript of the 𝑖th large number among 𝑣₁,𝑣₂,…,𝑣_𝑙 and 𝐰=(𝑤₁,𝑤₂,…,𝑤_𝑙) is weighted vector, which is satisfied with ∑^𝑙_𝑖=1𝑤_𝑖=1,  𝑤_𝑖≥0,  𝑖=1,2,…,𝑙.

Definition 1 indicates that the value of IOWA is obtained from the ordered weighted average of 𝑓_𝑖, which is ranked in order of size of 𝑣_𝑖. The values and location of 𝑤_𝑖 are not associated with those of 𝑓_𝑖, but associated with the location of the induced values.

In this paper, because the index stability is more important, it could be used as the induced components of IOWA operator. The final score of each qualitative index is 𝑃^∗=𝑝^∗_𝑖(𝑖=1,2,…,𝑛).

4.2. Determining the Weight of Second-Level Indexes

Using the method proposed in Section 4.1, we can convert the qualitative indexes into benefit-type quantitative ones. Now, decision matrix 𝐴=(𝑎_𝑖𝑗)_𝑐×𝑏 of whole index system within 𝑏 period is built up. However, as for the quantitative index 𝑎_𝑖𝑗, the two types of index, cost-type index and benefit-type index, need to be normalized. The method is described as in the following.

For cost-type index 𝑟_𝑖𝑗=min_𝑗𝑎_𝑖𝑗𝑎_𝑖𝑗,𝑗=1,2,…,𝑏. (8)

For benefit-type index 𝑟_𝑖𝑗=𝑎_𝑖𝑗max_𝑗𝑎_𝑖𝑗,𝑗=1,2,…,𝑏. (9)

It is important to note that because of the assumption that the higher the score, the better the performance of the index for benefit-type indexes, so all the qualitative indexes are changed into benefit-type index.

With (8) and (9), the normalized decision matrix 𝑅=(𝑟_𝑖𝑗)_𝑐×𝑏 of the whole index evaluation system within 𝑏 period can be obtained. Let Φ be the set of possible weight intervals for known indexes. The single-objective linear programming model can be established by using maximum deviation method based on deviation degree, as shown in max𝐷(𝑤)=^𝑏𝑐_𝑖=1^𝑏_𝑗=1𝑘=1𝑟_𝑖𝑗−𝑟_𝑘𝑗𝑤_𝑗s.t.𝑤_𝑗∈Φ. (10)

The weight of each index can be achieved by solving the linear programming model.

4.3. Method for Adjusting the Weight of First-Level Indexes

By considering that in practice the decision-makers may have different demands of scheduling performance management in different periods, the first-level indexes of the evaluation system should be adjusted. Since there are only three indexes in the first level, the range of the adjustment will not change frequently. Thus, an adjustment coefficient method based on set-valued statistics is used to adjust the weight. And set-valued statistics have been used in many articles [48–51].

4.3.1. Expert Evaluation

In accordance with the requirements of different performance management, 3 to 5 experts are invited to estimate the proper adjustment ranges of three first-level index weights based on the past data. Because the experts may not hold the specific adjustment range accurately and objectively, they can give their opinions with an interval value [𝑑^𝑘_𝑖1,𝑑^𝑘_𝑖2](𝑑^𝑘_𝑖1<𝑑^𝑘_𝑖2,−1<𝑑^𝑘_𝑖1<1,−1<𝑑^𝑘_𝑖2<1), where 𝑑^𝑘_𝑖1 and 𝑑^𝑘_𝑖2 should be consistent with positive number or negative number. The values indicate the percentages of increase or decrease in the weight of original index. The matrix of adjustment range is 𝐁: 𝑑𝐁=¹¹¹,𝑑¹₁₂𝑑²₁₁,𝑑²₁₂⋯𝑑^𝑘₁₁,𝑑^𝑘₁₂𝑑¹₂₁,𝑑¹₂₂𝑑²₂₁,𝑑²₂₂⋯𝑑^𝑘₂₁,𝑑^𝑘₂₂𝑑¹₃₁,𝑑¹₃₂𝑑²₃₁,𝑑²₃₂⋯𝑑^𝑘₃₁,𝑑^𝑘₃₂. (11)

4.3.2. Processing the Data with Set-Valued Statistics

Set-valued statistics is an extension of classical statistics and fuzzy statistics. In classical statistics, a certain point in the phase space can be obtained in each test, while in set-valued statistics a fuzzy subset can be achieved. Set-valued statistics can deal with uncertain judgment so as to concentrate various opinions conveniently and reduce the random error. In this study, the method is adopted to handle the interval numbers of weight adjustment range, and the detailed algorithm is described as follows.

Let 𝑑_𝑖 be the aggregation point of the 𝑖th index; 𝑑_𝑖=∑^𝑛_𝑘=1𝑑^𝑘_𝑖2²−𝑑^𝑘_𝑖1²2∑^𝑛_𝑘=1𝑑^𝑘_𝑖2−𝑑^𝑘_𝑖1. (12) In order to measure whether the evaluations of experts are uniform, set 𝑔_𝑖 as the degree of ramification: 𝑔_𝑖=∑^𝑛_𝑘=1𝑑^𝑘_𝑖2−𝑑_𝑖³−𝑑^𝑘_𝑖1−𝑑_𝑖³3∑^𝑛_𝑘=1𝑑^𝑘_𝑖2−𝑑^𝑘_𝑖1. (13) So the consistency of experts’ evaluation (𝑒_𝑖) is measured as 𝑒_𝑖=11+𝑔_𝑖. (14) The closer the value of 𝑒_𝑖 to 1 is, the higher the consistency is. A certain threshold is set to testify. If the consistency is too low, the new round evaluation is required.

4.3.3. Weight of Adjusted First-Level Indexes

After the consistency test reaches the acceptable range, we can use the results to adjust the weight and then normalize the adjusted weight with 𝑤_𝑖^𝑡=𝑤^𝑡_𝑖^′∑³_𝑖=1𝑤^𝑡_𝑖^′, (15) where 𝑤^𝑡_𝑖^′=(1+𝑑_𝑖)𝑤_𝑖^𝑡−1 and 𝑤^𝑡_𝑖 is the weight of 𝑖th first-level index for the 𝑡th period.

4.4. Application Procedure of Performance Evaluation Method Based on the Dynamic Index Weight

The evaluation method of scheduling performance presented here is adopted to evaluate the performance data of the (𝑏+1)th period. The application procedure is described as in the following.

Step 1. Determine the weight of first-level indexes. According to the requirements of dynamic adjustment, the weight can be achieved by the method proposed in Section 4.4.

Step 2. Process the qualitative data. After the data of previous 𝑏 periods are collected, separate them into qualitative and quantitative data. For each period’s qualitative data, quantify them with the multiphase dynamic interactive scoring method. Then the decision matrix 𝐴=(𝑎_𝑖𝑗)_𝑐×𝑏 with fixed value can be obtained.

Step 3. Normalize 𝐴=(𝑎_𝑖𝑗)_𝑐×𝑏 and obtain the normalized decision matrix 𝑅=(𝑟_𝑖𝑗)_𝑐×𝑏 in 𝑏 periods.

Step 4. Determine the weight of second-level indexes using the maximum deviation method based on deviation degree proposed in Section 4.3.

Step 5. Collect the data of the (𝑏+1)th period and quantify the qualitative data with the multiphase dynamic interactive scoring method proposed in Section 4.2 and then normalize all the data in this period.

Step 6. Obtain the final score with the weight of two-level indexes and the fixed value of data in the (𝑏+1)th period.

The flow diagram is shown in Figure 1.

Figure 1 

Flow diagram of performance evaluation method based on the dynamic index weight.

5. Innovations and Advantages of This New Performance Evaluation Method

5.1. The Main Innovations and Advantages of This New Method

In the implementation of the performance evaluation methods, the most important two aspects are the accuracy of the data and the accuracy of the weight. Compared with other papers, this paper has made much effort on these two aspects.

Firstly, the new evaluation method proposed has advantages in quantifying the qualitative index. In general, there are two types of traditional quantifying methods. One is a simple method which needs one-off scoring and removing the singular points according to some rule [52, 53]; the other is a complex method, such as analytic hierarchy process [35, 36], analytic network process [3], and fuzzy evaluation method [37]. These methods are all one-off scoring and solving with judging criteria [23–25]. The above two types of traditional methods may be faced with the problem that the selected experts have different understanding with the indexes measure and only have once evaluation opportunity, which will lead to inaccurate results. Therefore, this paper uses the multiperiod interaction method. The core idea of this method is through several rounds interacting evaluations to make the group’s opinions reach certain stability and consistency so as to reduce the subjectivity and improve the accuracy in expert scoring.

Secondly, in previous evaluation methods, the weight of indexes was given, single-period and constant [7, 8, 15–17] while in this paper we consider the problem of indexes weight dynamic adjustment in the multiperiod context. We draw the multiperiod interaction method presented by [54] and improve it. The method of [54] is suitable for direct evaluation and ranking for several objects, but not for the quantified assessment of a single object. In this paper, the method is improved properly and applied to realize the quantified assessment for LSSC scheduling which is single object.

Thirdly, because there are only three first-level indexes (scheduling environment, scheduling process, and scheduling results) and, in multiperiod dynamic adjustment, their weight will not change frequently in practice, we adopt an adjustment coefficient method based on set-valued statistics to adjust first-level indexes weight. Obviously, set-valued statistics is a group decision and has better effect than general one-off scoring in integrating the opinions of experts group [48–51].

5.2. Other Merits of This New Performance Evaluation Method

Compared with other performance evaluation methods, this method has some other merits of the following aspects.(1)The evaluation index system used in this new method overcomes the shortcoming of traditional methods that only focus on a single enterprise or sector. From the perspective of supply chain cooperation, the evaluation index system emphasizes the whole scheduling process, namely, before scheduling, during scheduling, and after scheduling.(2)The maximum attribute deviation method is introduced to determine the weight of the second-level indexes, which can remove the uncertainty of subjective factors. This improves the method proposed by [55].(3)An adjustment coefficient method based on set-valued statistics is introduced to determine the weight of the first-level indexes, which can make the adjustment range more precise and decrease the difference among experts more significantly.

6. Method Application: A Case from Tianjin Baoyun Logistics Company, China

6.1. Case Description

6.1.1. Company Introduction

In order to verify the usability of the method, the data are collected from Baoyun Logistics Company in Tianjin, China. The company is a professional third-part logistics enterprise and an excellent logistics service integrator listed as the top 100 logistics enterprises of China in three consecutive years from 2005 to 2007. Currently, the company has 28 branches across the country and builds up a good relationship of cooperation with 32 large-scale warehousing enterprises, 20 transportation enterprises, and more than 15 professional logistics enterprises. By integrating these functional logistics service providers, Baoyun has established wide business connection with over 20 multinational customers such as P&G, Siemens, and Delphi Corporation and offered customized logistics service in accordance with logistics demands.

During the cooperation with P&G, Baoyun Logistics Company provides integrated service, including railway transportation service, warehousing service, and road distribution and delivery service. Generally speaking, Baoyun evaluates the scheduling performance of these services provided to P&G monthly. When the evaluation is conducted, P&G, logistics service providers, and Baoyun jointly participated, and the indexes are evaluated individually to get the original data. If some indexes involve multiple parts, they need to be negotiated with each other to determine the final values.

In this case, the original data were collected from January 2012 to May 2012. Then the method proposed in the paper is used to evaluate the scheduling performance of June according to the data of the previous 5 months.

6.1.2. Selection of Evaluation Experts

In the scheduling performance evaluation of Baoyun, the biggest challenge is how to quantify the qualitative data. For this, Baoyun selects the managers who directly take charge of scheduling activities as evaluation experts. Usually, the three members which are P&G, Baoyun, and FLSP of LSSC assign their managers to participate in the performance evaluation. The experts group was composed as in Tables 3 and 4.

Table 3 

Experts group for evaluating the first-level indexes.

Table 4 

Experts group for evaluating the second-level qualitative indexes.

It is important to note that, according to the task of evaluation subject in Table 1, the first-level indexes are evaluated by all the three evaluation subjects, P&G, Baoyun, and FLSP. There are 5 experts in total shown in Table 3. While for the second-level indexes which are closely related to practical scheduling process, they are evaluated by Baoyun and FLSP shown in Table 4. When evaluating, all experts give their scores independently first and then start multistage interactive grading.

6.1.3. Challenges and Solutions in Evaluation

There are three aspects of challenges. The first is that the experts are from different companies and they may have different preferences. The second is that it is hard for these experts to get together doing evaluation work. The third is that sometimes the divergences among experts are too big to obtain a consistent result. So we need more grading rounds.

To overcome these challenges, P&G, Baoyun, and FLSP cooperate a lot. First, before the evaluating, the experts studied and discussed the indexes to make a consistent understanding. Second, Baoyun Company assigns a secretary to coordinate the time of these experts. She would inform the meeting time ten days before. Third, to reduce the evaluation round, every expert would explain his evaluation in the first round, so that other experts can understand his considerations. This can improve the evaluation efficiency and reach an agreement faster.

6.2. Application of Scheduling Performance Evaluation Method

6.2.1. Step 1: Determine the Weight of First-Level Indexes

According to the evaluation index system, there are three first-level indexes, scheduling environment, scheduling process, and scheduling result. The index of scheduling result is composed of two factors, which are customer satisfaction and service quality. In this paper, it is assumed that the two factors are equally important. The weights of the three indexes for the previous 5 months are 0.3, 0.3, and 0.4, respectively, and need to readjust in the sixth month. The matrix of weight adjustment range obtained from 5 experts is as follows and the consistency test threshold is 0.95: ()()()𝐁=.0.0,0.1)(0.1,0.2)(0.1,0.3)(0.2,0.4)(0.1,0.20.1,0.2)(0.3,0.4)(0.2,0.4)(0.0,0.1)(0.2,0.3−0.3,−0.2)(−0.2,0.0)(−0.2,−0.1)(−0.2,−0.1)(−0.3,−0.2 (16)

According to the matrix 𝐁, adjustment range 𝑑_𝑖 could be calculated by (12); then consistency index 𝑒_𝑖 could be gotten by (13) and (14). Original weight 𝑤_𝑖^𝑡−1 and adjusted weight 𝑤^𝑡_𝑖^′ could be obtained by (15). The weight of first-level indexes is calculated as in Table 5.

Table 5 

The results of adjusted weight of first-level indexes.

See from Table 5 that the consistency index values of the experts evaluation are all over 0.95, and the adjustment of index weight is acceptable, so the first-level weights of the 6th period are 0.34, 0.35, and 0.31, respectively.

6.2.2. Step 2: Quantify the Qualitative Indexes Using the Method of Multiphase Dynamic Interaction

According to the original data, by using the method of multiphase dynamic interaction, the qualitative indexes of previous five periods can be converted into benefit-type quantitative ones, and the decision matrix 𝐴=(𝑎_𝑖𝑗)_𝑐×𝑏 can be obtained as in Table 6.

Table 6 

Decision matrix.

6.2.3. Step 3: Normalize Decision Matrix

Customer complaints rate (𝐶₃₁₁), response time for demand (𝐶₃₁₄), wrong scheduling rate (𝐶₃₂₁), and scheduling cost (𝐶₃₂₂) are cost-type indexes and the rest are benefit-type indexes. After the decision matrix is normalized, the normalized decision matrix can be obtained as in Table 7.

Table 7 

Normalized decision matrix.

6.2.4. Step 4: Determine the weight of second-level indexes

The maximum deviation method based on deviation degree is applied to calculate the weight of scheduling environment, scheduling process, and scheduling result respectively.

The possible index weight of each index for scheduling environment is Φ¹={𝑤¹=(𝑤¹₁,𝑤¹₂,𝑤¹₃,𝑤¹₄)0.25≤𝑤¹₁≤0.27,0.21≤𝑤¹₂≤0.23,0.24≤𝑤¹₃≤0.28,0.20≤𝑤¹₄≤0.32}.

And that for scheduling process is Φ²={𝑤²=(𝑤²₁,𝑤²₂,𝑤²₃,𝑤²₄)0.22≤𝑤²₁≤0.25,0.24≤𝑤²₂≤0.31,0.18≤𝑤²₃≤0.22,0.28≤𝑤²₄≤0.32}.

And that for scheduling result is Φ³={𝑤³=(𝑤³₁,𝑤³₂,𝑤³₃,𝑤³₄,𝑤³₅,𝑤³₆,𝑤³₇,𝑤³₈)∣0.120≤𝑤³₁≤0.135,0.118≤𝑤³₂≤0.132,0.109≤𝑤³₃≤0.141,0.101≤𝑤³₄≤0.141,0.114≤𝑤³₅≤0.147,0.119≤𝑤³₆≤0.132,0.106≤𝑤³₇≤0.134,0.113≤𝑤³₈≤0.134}.

Three single-objective linear programming models are established as in (17). Software MATLAB 7.0 is adopted and the weight of each index is obtained as shown in Table 8.

Table 8 

Weight of second-level indexes.

Consider max𝐷(𝑤)=⁵⁴_𝑖=1⁵_𝑗=1𝑘=1𝑟_𝑖𝑗−𝑟_𝑘𝑗𝑤_𝑗s.t.𝑤_𝑗∈Φ¹max𝐷(𝑤)=⁵⁴_𝑖=1⁵_𝑗=1𝑘=1𝑟_𝑖𝑗−𝑟_𝑘𝑗𝑤_𝑗s.t.𝑤_𝑗∈Φ²max𝐷(𝑤)=⁵⁸_𝑖=1⁵_𝑗=1𝑘=1𝑟_𝑖𝑗−𝑟_𝑘𝑗𝑤_𝑗s.t.𝑤_𝑗∈Φ³. (17)

6.2.5. Step 5: Process the Data of the Sixth Month Based on Multiphase Dynamic Interaction

In this step, the application example of the method based on multiphase dynamic interaction is given. The original data of the sixth month is shown in Table 9.

Table 9 

Original data of the sixth month.

According to the original data, 5 experts are invited to score the 8 qualitative indexes. After discussion, 4 times dynamic interaction are conducted. So 𝑚=5, 𝑛=8, and 𝑙=4. The scores are distributed between 0 and 10, and the matrixes are shown as follows: 𝐏^𝟎=𝐶₁₁₁𝐶₁₁₂𝐶₁₁₃𝐶₁₁₄𝐶₂₁₁𝐶₂₁₂𝐶₂₁₃𝐶₂₁₄𝐏8436555832682448253763657528637737473662_𝟏=𝐶₁₁₁𝐶₁₁₂𝐶₁₁₃𝐶₁₁₄𝐶₂₁₁𝐶₂₁₂𝐶₂₁₃𝐶₂₁₄𝐏8636744355547637853645973694889837657622_𝟐=𝐶₁₁₁𝐶₁₁₂𝐶₁₁₃𝐶₁₁₄𝐶₂₁₁𝐶₂₁₂𝐶₂₁₃𝐶₂₁₄𝐏7563425546458638347257835454676468486624_𝟑=𝐶₁₁₁𝐶₁₁₂𝐶₁₁₃𝐶₁₁₄𝐶₂₁₁𝐶₂₁₂𝐶₂₁₃𝐶₂₁₄𝐏6664426435657537457356846253648558557636_𝟒=𝐶₁₁₁𝐶₁₁₂𝐶₁₁₃𝐶₁₁₄𝐶₂₁₁𝐶₂₁₂𝐶₂₁₃𝐶₂₁₄.6664436445657547557356746453648558557646 (18)

Calculate the interaction coefficient with (1) and (2) as in the following: 𝐮^𝟏=()𝐮0.194,0.214,0.194,0.196,0.203_𝟐=()𝐮0.192,0.201,0.219,0.190,0.198_𝟑=()𝐮0.202,0.196,0.197,0.202,0.202_𝟒=(0.202,0.199,0.198,0.201,0.199). (19)

Aggregate the score of each round with (3): 𝐏^𝟏∗=()𝐏6.009,4.205,4.964,5.840,5.567,7.051,6.623,4.426_𝟐∗=()𝐏5.025,4.432,5.259,4.471,5.679,5.175,5.582,5.228_𝟑∗=()𝐏5.191,3.988,5.192,5.208,5.788,3.999,6.213,5.387_𝟒∗=(5.199,4.395,5.399,5.408,5.594,4.403,6.210,5.595). (20)

Set the stability threshold as 0.1(𝜀=0.1) and the consistency threshold as 0.25(𝜂=0.25), and then calculate the stability index and the consistency index of each round as follows: ()().𝝊=0.56,0.641,0.813,0.921𝝎=0.685,0.7,0.744,0.771 (21)

The index stability and index consistency of the fourth round are 0.921 and 0.771, respectively. The test value of stability index is 0.079 less than 0.1, and the test value of consistency index is 0.229 less than 0.25. So it can be concluded that the results pass the tests. Finally, the final score of each index with IOWA is aggregated: 𝐏^∗=(5.243,4.261,5.265,5.258,5.667,4.701,6.127,5.342). (22)

The data of the sixth month are normalized by combining the data in previous 5 months, and the results are shown in Table 10.

Table 10 

Normalized value of data in the sixth month.

6.2.6. Step 6: Obtain the Final Score

Obtain the final score by the fixed data as shown in Table 8 and the index weights determined in Steps 1 and 4. The expressions are shown as in the following: ()+()()0.86×0.25+0.61×0.21+0.80×0.24+0.85×0.30×0.340.90×0.22+0.72×0.28+0.91×0.22+0.77×0.28×0.35+0.69×0.135+0.93×0.130+0.98×0.109+0.74×0.141+0.64×0.147+0.93×0.119+0.98×0.106+0.97×0.113×0.31=0.815. (23)

Therefore, the final scheduling performance of the sixth month (June 2012) is 0.815. According to the performance ratings standardization of Baoyun Logistics Company, 0.815 reaches the good level, but it also has great potential to increase the scheduling performance in the future.

7. Conclusions

Evaluating the LSSC scheduling performance is beneficial to make better decision for LSI. In this paper, the scheduling performance evaluation of LSSC is explored in depth and the evaluation index system is established. Meanwhile, an evaluation method based on dynamic index weight is proposed. According to the study, the following conclusions are reached.(1)The evaluation index system of LSSC scheduling performance should be designed from the perspective of strategic level, tactical level, and operational level. The system proposed in this context considers not only the different demands of different members in LSSC, but also the requirement of coordination and characteristics of service, which is scientific and can be used for reference to scheduling performance management.(2)Considering the multiperiod feature of scheduling activities, the difficulty in measuring the qualitative indexes, and the needs of index weight adjustment, we propose a method based on dynamic index weight. In this method, firstly, for the difficulty in measuring the qualitative indexes, an improved method of multiphase dynamic interaction is adopted to improve the accuracy. Secondly, as for the need of index weight adjustment, an improved maximum deviation method based on deviation degree is utilized to determine the weight of second-level indexes, which can remove the uncertainty of human decision. Thirdly, an adjustment coefficient method based on set-valued statistics is developed to adjust the weight of first-level indexes reasonably. Thus, this method is more suitable than the traditional evaluation ones in LSSC scheduling performance evaluation.(3)Actual data from Baoyun Logistics Company were collected to exemplify this new scheduling performance evaluation method. The application example illustrates that the method can evaluate the scheduling performance scientifically and provide good basis to LSI for improving the performance of LSSC scheduling.

In this paper, the problem of LSSC scheduling performance evaluation is investigated and a new evaluation method is proposed, but there are still some limitations. For instance, the evaluation index system reflects the factors of before, during, and after the schedule, while the more detailed factors, such as order allocation, process selection, and time flexibility scheduling, are not considered. As for the evaluation method proposed, the possible intervals of second-level indexes weights are obtained by experience, which contains some subjectivity. How to improve the objectivity of index weight intervals is one of future directions. Moreover, with the evaluation value obtained from the method proposed, how to use it to forecast the performance of next period and present improvement program is also the focus of future researches.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This research is supported by the National Natural Science Foundation of China (Grant no. 71372156), supported by Humanity and Social Science Youth Foundation of Ministry of Education of China (Grant no. 2013YJC630098), and sponsored by China State Scholarship Fund and Independent Innovation Foundation of Tianjin University. The suggestions of the reviewers are also gratefully acknowledged.