Abstract
Performance measurement and forecasting are crucial for effective management of innovative projects in emerging knowledge-based companies. This study proposes an integrated performance assessment and forecasting model based on a combination of earned schedule methodology and the learning curve theory under risk condition. The operational performance is measured in terms of time and cost at completion indicators. As a novelty, the learning effects and Kalman filter forecasting method are employed to accurately estimate the future performance of the company. Furthermore, in order to predict the cost performance accurately, a logistic growth model is utilized. The validity of this integrated performance measurement model is demonstrated based on a case study. The computational results confirmed that the developed performance measurement framework provides, on average, more accurate forecast in terms of mean and standard deviation of the forecasting error for the future performance as against the traditional deterministic performance measurement methods.
1. Introduction
Measuring the operational performance is essential for the knowledge-based companies. For such companies, learning has direct effect on the productivity of the firms [1]. Normally, knowledge-based companies employ traditional project management methodology to monitor the progress of the plan and the actual performance on a regular basis and take corrective actions in case of delay or deviation from the initial schedule. As a result, the performance measurement is essential to control the project effectively and avoid repetitive errors. This is an important and relevant problem to study, particularly for emerging knowledge-based companies where the learning effects influence the performance of the firm [2].
Among different quantitative approaches to the performance measurement, the earned value method (EVM) has a considerable potential to be used as a basis for performance measurement [3]. However, the traditional EVM was developed based on the assumption that the performance is a constant function of time. The problem with this assumption is that the knowledge sharing and team learning affect the project performance during the project execution. Thus, the actual performance changes over time. The variability of performance causes complications when forecasting the final cost and time particularly under risk conditions. A practical model for analysing the nonlinear effects of learning on the firm’s performance is the learning curves (L-curves). L-curves are quantitative model of performance variations during the work progress. L-curves have been used in a vast range of contexts among scholars. The L-curve hypothesises performance improvement as a function of practice, with the most intense improvements happening at the opening of the learning procedure [4]. L-curves have been used by previous researches on project management, and there have been few scientific papers to extend traditional framework of the EVM with nonlinear components; see, for example, [5–8]. However, the previous scientific efforts have not expansively addressed the combination effects of learning and risks on the accuracy of the performance predictions. Also, deterministic performance measurement models, for example, EVM, have several deficiencies for the performance measurement of knowledge-based companies. Therefore, in this paper, a probabilistic performance assessment model is proposed based on learning curve theory and earned value management approach.
2. Related Works
Performance management systems are a set of processes used by organizations for supporting the on-going management through planning, measurement, forecasting, and analysis of performance and for facilitating organizational learning and change [9]. Apart from theoretical frameworks, for example, balanced scorecard (BSC), quantitative models are critical to be used to measure the progress and forecast the future performances. Table 1 provides taxonomy of quantitative methodologies for performance measurement. For a comprehensive review of the different project duration forecasting methods using earned value approach, see Vandevoorde and Vanhoucke [10]. Recent research has recognized the strong relationship between learning and performance. For example, Ngwenyama et al. [11] proposed an effective planning approach for software development project that will maximize the firm productivity using learning curve as the theoretical background. The value of the technology was estimated through a modified learning curve function. It was concluded that the designed performance measurement model supports the decision-making process for a wide range of technology implementation projects.
Plaza and Rohlf [6] developed a mathematical model that utilizes L-curves in forecasting of the project completion time. A training strategy was proposed that minimized the project consulting costs within a theoretical background for empirical analysis of learning. Plaza [4] addressed the accurate forecasting problem of project duration by the impact of the learning curve for information system projects. The highlight of Plaza’s work is a decision support system (DSS) integrating learning curve calculation with EVM. The outcomes indicate that the designed DSS has significant practical application to the control of projects. Bondugula [12] proposed an optimal project control process using Kalman filter forecasting method (KFFM) for updating. The proposed model was used for forecasting the cost estimation at completion (CEAC) and the estimated duration at completion (EDAC) addressing the risks and uncertainties in the project progress. However, the effects of learning on the performance have been ignored. Wang et al. [13] proposed a novel performance-oriented risk management framework that aligns project risk management with business strategic goals. The proposed performance measurement model was used to improve success rates of innovative research and development projects. The integration of balanced scorecard (BSC) and quality function deployment (QFD) method is proposed to recognize major performance measures and to transform organizational performance measures into project performance measures. Kim and Reinschmidt [14] proposed a new forecasting method based on the Kalman filter and the earned schedule (ES) approaches. The proposed model was validated using two real projects through extracting actual data about the status, trend, and forthcoming project schedule performance and related risks. Consistent forecasting model enables the project executive to make better decisions for well-timed control actions. Azeem et al. [15] developed three models to estimate the duration at completion of projects. The first and second models were deterministic on the basis of earned value (EV) and earned schedule (ES) approaches. The third one was a stochastic forecasting model based on the integrated Kalman filter forecasting model (KFFM) and earned schedule approach. A case study was used to validate the proposed performance measurement models. The outcomes exhibited that the KFFM provides more accurate predictions as against the EV and ES forecasting models. Sadeghi et al. [16] proposed a project competency model that addresses three dimensions of knowledge, performance, and competency criteria. The attained outcomes of the multicriteria decision-making process proved the applicability of the suggested performance competency evaluation method in practice. Chou et al. [17] proposed a novel hybrid multiple-criteria decision-making procedure on the basis of earned value management to measure the project performance. Numerical test cases were used to prove the applicability of the proposed performance assessment procedure. Qin et al. [18] addressed the workforce planning model for assigning tasks to multiskilled workforce by considering nonlinear learning effects of knowledge and requirements of project quality. A piecewise linearization scheme to learning curve was suggested. Also, a mixed integer linear programming model was proposed and then it improved by taking into account the performance of the experienced personnel and the upper bound of the employees’ experiences build-up.
According to the reviewed articles, the research gaps are as follows: as mentioned by Azeem et al. [15], a limitation of the KFFM is that it is appropriate only to the forecast of expected duration at completion, not to the prediction of cost estimation at completion (CEAC), though Kalman filter method can be extended to estimate CEAC so that schedule and cost estimating can be integrated within an integrated procedure. Moreover, despite the fact that several qualitative and quantitative studies have been directed toward the project performance measurement, only a few have analysed the effects of the learning on performance under risk situations. Measuring the operational performance of knowledge-based projects is a bit more problematic and constitutes the key idea of this study. The practice-oriented objective of the present study is to design interactive user-friendly application software that assists knowledge-based practitioners in the favourite implementation of the performance measurement model. The present research to be presented has two main parts: one that concerns the project performance measurement and a second that focuses on forecasting performance indicators in terms of time and cost of the project subject to the errors and risks. The contributions of the present study are threefold. First, this study extends to the model by Plaza [4], who focuses only on forecasting time at completion, by extending the performance measurement domain to analyse both time and cost with regression models. Moreover, the earned schedule technique is explicitly used as a basis to assess the nonlinear effect of learning on performance. Finally, compared with studies that utilize classic framework of earned value method, the present analysis takes into account the risk factors and focuses on innovation projects.
3. Performance Assessment Methodology
3.1. Time Performance Assessment
In this section, we provide the assumptions and formulations used for time performance assessment. Earned value indexes have been extensively used to forecast the future performance, in terms of both time and cost. A standard formula for project duration estimation is defined aswhere is the estimated project’s final duration. AD and PDWR represent the actual duration and the planned duration of work remaining, respectively. PDWR is affected by the learning effects. The earned schedule (ES) methodology forecasts the planned duration of work remaining. ES is an extension of the earned value management [3]. It overcomes limitations of the classic EVM through calculating expected duration at completion (EDAC) by extending the EVM-based SPI. In this method, the plan progress is measured in time units. The experimental results confirmed that the ES method affords more accurate estimates than EVM-based SPI calculations [10]. The earned schedule methodology can be formally expressed as
Let and denote earned value and the planned value at time (time point), respectively. The actual duration and the planned duration (PD) are also given. It should be remarked that, at the end of a project, (budget at completion). The generic formula for estimating the earned schedule duration can be written as follows:
The performance factor (PF) depends on the project characteristics and risk conditions. It is assumed that the duration of remaining work follows the trend of schedule performance index (). Thus, EAC can be mathematically expressed as
As a result, a time-based schedule performance index at time can be defined by
The expected duration at time is the ratio of planned duration to . The proposed forecasting method uses the inverse of in order to account for the schedule effect on CEAC. This inverse proportion is denoted by completion factor (CF). The CF specifies EDAC brought forth to unity and it can be presented in
3.2. Cost Estimation Model
This section provides the proposed cost estimation methodology. A number of approaches are found in the literature of the EVM to estimate cost at completion (CEAC), for example, index-based and regression-based techniques. We further extended the previous performance measurement model by providing analysis of forecast errors and integration of the influence of learning on performance and consequently on the CEAC calculation. Generally, index-based methods assume that remaining budget is modified by a performance index [29]. Regression techniques and growth model have been recognized as alternatives to traditional index-based cost estimation methods. Growth models and regression curve-fitting techniques improve the accuracy of the CEAC, particularly as they can be integrated with the EVM data and the earned schedule (ES) approach, so that they can provide more accurate and consistent forecasts. Among the S-shaped growth models, we employ logistic growth (LM) function for curve fitting and consequently to forecast the project cost (Figure 1). As can be seen, LM is normally distributed with an inflection point at 50% of total growth. This growth model was widely implemented in practice because of its easiness and analytical tractability. The generic formula of LM is represented in (7). This function consists of a future value asymptote of the model that represents the final cost (α), an initial size of project cumulative cost (β), and a scale factor (γ) that relates to the cost growth rate (GR):
In order to implement the cost estimation model, first the values of α, β, and γ are obtained through the analysis of nonlinear regression models. Afterward, the LM model is used to compute CEAC. More precisely, the CEAC formula is modified with the purpose of analysing the effect of learning on schedule progress and cost performances.
All the values of predictor and response variables (time and cost) units are normalized to input into the model. The normalization of time points to unity regards that a project time is 100% complete (that is to say, PD = 1). Each time point () is associated with a cost point to run the nonlinear regression curve fitting. These resultant cost points are then calculated as follows. The actual values of cost from time to actual time (AT) are standardized to unity (i.e., the normalized BAC equals 1). Afterward, the normalized values of up-to-now AC and PV are joined to obtain the values of the cost variable.
According to the Gauss–Newton approximation algorithm, the initial values of the LM parameters are adjusted to 1 with the accuracy level of 95%. At that time, the values of the three parameters are obtained through the regression analysis. Then, CEAC is computed through a modified formula so that, instead of adjusting it with a performance indicator, the remaining expected cost is calculated by the regression analysis:
Finally, the LM is modified to account for the possible effect of work progress on CEAC. The main assumption of this modification is that the schedule efficiency is likely to decrease the final cost. The value of indicates that a project completes on time. It is substituted by the completion factor. The integrated cost-schedule approach considers the schedule impact as a contributing factor of cost values. Finally, the modified CEAC equation is provided in
3.3. EVM Extension by Learning Curves
EVM is established on the notion that both estimated and actual performance are constant over time; however, in many knowledge-based companies, performance generally follows a nonlinear L-curve (Figure 2). The L-curve signifies the rate of performance progress throughout the project life cycle. With the aim of better understanding of the method, we first provide the notation used to explain the performance measurement model integrated with learning curves (see the list below). Planned duration () is computed based on the assumption that performance remains fixed during the project life cycle. This estimated time is associated with the constant planned performance (). According to the EVM principles, the schedule performance index (SPI) can be determined as
The Notation of the Performance Measurement Model : planned duration of the project : forecasted project duration regarding constant performance : forecasted project duration with learning effects : the minimum (best possible) duration : the time when the performance was measured : planned performance : actual performance : asymptote to the learning curve : the progress function of time () : the coefficient of the learning curve : the adjusted schedule performance index by learning impact : the earned value of the work performed according to the nonlinear progress function.
The actual performance can be measured by substituting the above expressions for EV and PV into (10) as
Likewise, the adjusted duration () is calculated according to the actual performance and the given SPI:
A formal representation of the learning curve is given in
The schedule performance index impacted by the learning curve can be calculated as follows:
In order to quantify the relation between SPI and , a corrective factor called performance correction index (PCI) is defined. can be further defined as
At this point, the values of the learning curve asymptote () and best possible project duration () can be determined as
The minimum project duration is resulting based on the assumption that performance is at all the time. The relation between EV and can be expressed as
PCI is an unknown parameter and it can be determined by incorporating the performance function into the earned value equation. It should be remarked that the area under the L-curve function represents the amount of work actually completed up to time and it can then be calculated as
After the calculation of the above integration, the performance adjustment index can be obtained as
As calculated in (19), PCI is related to the learning curve coefficient and time when the performance was measured. This indicates that, after the calculation of the learning parameter , the estimation of the rest of the unknown parameters will be forthright. The reader is referred to Plaza and Turetken [7], for a detailed explanation of the procedure of determining the value of .
Among the predictive approaches, Kalman filter method has been recognized as an efficient way to estimate the performance indicators with minimum error. In what follows, we explain how to integrate performance evaluation, risk analysis, and learning effects. The proposed performance evaluation model accounts for the risks of time and cost overruns. It calculates the standard deviation of the completion time and the deviation of the actual cost and the planned cost of the project. In addition, the accuracy of the measurement during the performance appraisal process is very important. Thus, a risk assessment method is required to be integrated with the performance measurement system.
3.4. Risk-Oriented Performance Measurement Using Kalman Filter
The Kalman filter is an efficient recursive forecasting procedure utilized to estimate the future state of a dynamic system in the existence of noises [30]. The Kalman filter has extended its application domain to different areas and many prediction and control problems. The reader is referred to the work by Li et al. [31] for further improvement of basic Kalman filter method. However, despite the wide range of potential applications, the Kalman filter has not been extensively used in the context of performance management. In this study, we implement Kalman filter forecasting method in combination with risk assessment model and learning curve. Kalman filter forecasting model uses a baseline plan and accounts for the cumulative progress curve that represents the amount of work to be completed at a time point. The forecasting technique focuses on the estimation of the deviation between the planned performance and the actual performance throughout the execution of the project. To perform the forecasting calculation, it requires the actual performance data, as well as the information regarding the budget at completion, the baseline progress curve, the planned duration (PD), and the prior probability distribution of the project duration at time . The basic components of the Kalman filter algorithm are provided in the list below. In this framework, the state of a dynamic system is represented at time by two sets of variables: (state variables) and (error covariance). The error covariance signifies the uncertainty associated with the estimations of the state variables. The states and error covariance are adjusted at each time point through measurement model and the system model. Since the future performance is uncertain, the system model has a probabilistic nature. The process noise represents the uncertainty associated with the system model. In the context of operational performance forecasting, the process noise is interpreted as the performance deviations as a result of inherent uncertainty associated with the execution plan.
The Basic Notation of the KF Forecasting Model : time variance : state variable : error covariances : process noise covariance matrix : measurement error covariance matrix : transition matrix : observation matrix : Kalman gain matrix : vector of random process noise : new observation : vector of random measurement noise : measurement error variable : dynamic system model : measurement model : prediction process : Kalman gain : updating process.
The focus is on the cost overrun and the variance (TV), which represents the difference between the initial plan (planned duration) and actual performance. The time variance is calculated the same as the schedule variance (SV) or cost variance (CV) as previously described. In other words, at any time point such as , the amount of is the difference between actual time and earned schedule (ES) and it is calculated as
Kalman filter estimates the expected duration at completion (EDAC) using the time variance during different periods. The work progress is represented as a system with two state variables that evolve over time: the time variance (TV) and its ratio of change over a forecasting horizon:
The calculations of the state variable and new measured (real observation) are done through the following formula:
Two types of errors are included in the performance measurement model. The first is the measurement error and the second is process error during the prediction. The error variables indicate the accuracy of the measured variable. The covariance matrix of process error () shows the uncertainty in the process model. The measurement error covariance matrix () represents the accuracy of the measured actual performance. The measurement error covariance matrix of the random error vector measurement () is calculated as
Kalman filter method estimates the posterior distribution according to the calculated initial distribution of the random variable and a set of model parameters. The covariance of estimation error is determined by the system state error and the difference between the system variable and its estimation () as follows:
The prediction is performed using an initial estimate () of the state variable based on the estimates at previous time interval () and the transmission matrix is calculated as follows:
In a measurement model, using the new observation (), the accuracy of estimates in previous iterations () is calculated as
Kalman gain matrix () is determined to minimize the covariance matrix of posterior estimation error (). The formula for this calculation is as follows:
Process noise matrix () is a controller of the moderating risk effects and Kalman gain (). Choosing the improper covariance as a fundamental factor results in the lack of proper functioning of Kalman filter model
To accurately estimate the elements of noise matrix , the primary distribution of time and costs is used. If the scheduled duration is denoted by the PD, then optimistic (), probable (), and pessimistic () estimates of the time (or cost) are defined as , , and . The parameters of primary distribution of the time and cost (such as mean and variance) are obtained using the three-point estimate (using PERT) as follows:
The process noise () should be estimated in such way that, at the end of the forecast period, the error covariance is equal to the initial distribution of predicted variance. The values of these parameters for both time and cost are calculated separately. In practice, the error variable () can be estimated using a three-point estimation method for measurement of error. The measurement error covariance () is an important factor in the implementation of Kalman filter and is an indicator for accuracy of measuring actual performance. If represents the value equal to the maximum possible measurement error variance, then the variance of measurement error () is obtained as
As a result, can be obtained from the above equation and placed in (27) during the update process. This is the way that risk analysis is performed in the proposed performance evaluation model.
4. Integrated Performance Measurement Model
The flowchart of the proposed performance measurement model is illustrated in Figure 3. The suggested steps for the development of a risk-based model to assess the time and cost performance of knowledge-based companies under learning effects are as follows. In the first step, the decision maker decides the learning growth coefficient () as well as the calculation of the performance correction index () influenced by the effects of learning. With regard to the relationship between the cost and time, estimation of the future costs is essential to assess the performance of the company. Furthermore, due to the fact that the time and cost indicators of the classical EVM have been estimated independently, a relationship must be found between time and cost at completion. As a result, a cost growth function that determines the project final cost has to be fitted using nonlinear regression analysis. As previously described, the logistic growth model (LM) is used to estimate project final cost. Afterward, the initial performance evaluation of knowledge-based companies is performed in terms of time and cost indicators. In this step of modeling, to assess the current state of knowledge-based company, the information on the performance of a benchmark company (as a case study) will be collected. In order to validate the performance of the proposed risk-based assessment model, the company’s performance indicators are estimated. In this stage, the performance evaluation is based on the empirical data, using classical EVM. Then, the Kalman filter model is used to forecast the time performance indicator (). In this step, risk analysis is performed using the Kalman filter to estimate the time and cost indexes. The schedule performance index obtained from the Kalman filter method is denoted by . The estimation of the deviations is measured on the basis of the difference between the expected performances and the concepts related to the earned schedule (ES). As a result, schedule performance index for a knowledge-based is derived as follows:
The schedule performance index calculated by the Kalman filter is denoted by . Then, based on the calculated value of the performance correction factor (), the final value of the schedule performance index () is estimated as follows:
This new performance index is customized for the knowledge-based companies. According to the above index, the estimated time at completion () under both learning and risk conditions is estimated as follows:
It should be mentioned again that the amount of EDAC is based on the calculated TV. Correspondingly, according to the estimated cost function , the final cost at the time of completion CEAC is calculated at the time . As a result, the cost estimation at completion (CEAC) is computed as follows:
Using the above performance indicators, knowledge-based companies will be able to assess the possibility of achieving the ultimate performance.
5. Case Study
The key objectives of the case study are to conduct a preliminary test and to validate the practical benefits of the performance measurement model. The methodology is also to evaluate and compare risk response strategies. Strategic management development company (AMIN) is knowledge-based company in the field of integration of the education services using comprehensive implementation of information and communication technology. Many of the customers of the company include the students, teachers, and anyone who is somehow involved in the education process. The summary data collected from the project files and the basic parameters determined for the performance measurement analysis are provided in Table 2. The information of the project including the project activities, duration, predecessors, the associated cost, and the percentage of complete is summarized in Table 3. PoS represent the initial probability of success. The decision maker uses this graphic user interface to decide the input data.
5.1. Performance Forecasting Results
In this section, the summery results of the earned value methodology, earned schedule method, and the combined Kalman filter and learning curve model are discussed. The values of the performance indicators obtained using different forecasting methods are provided in Table 4. Earned value and actual cost curves versus the planned value are depicted in Figure 4. The earned value and the actual project data at the end of the 7th month are shown in this graph. At the current time period, the earned value and actual and planned value cost are 33500.0 $, 35166.7 $, and 63850.0 $, respectively.
Figure 5 shows the result of a sensitivity analysis of the differences between EVM/LC forecasts () and those obtained by the EVM under different levels of SPI. According to the obtained outcomes, if SPI < 1 (behind the schedule), EVM calculations propose that more assets should be allocated in order to complete it according to initial plan. Nevertheless, as the graphs for different values specify, there is quite a relatively high probability that the knowledge-based company could finish the project on time since all values are lower than . Even though this remark is valuable, it is based on the most optimistic forecasts of project completion times, and so it may be impractical. At this point, it would be useful to further expand the proposed performance assessment model by computing the estimates for the time performance metric.
5.2. EDAC Profiles Produced by the KFFM
In this section, the probabilistic analysis of the time performance index is discussed. The obtained results are categorized into three main parts (probabilistic performance reporting graphs) as follows. These graphs are effective tools for displaying, analysing, interpreting, and evaluating the probabilistic performance prediction results. The KF output provides different viewpoints on the performance indicators and its associated risk factors and can support the knowledge-based companies to make up-to-date decisions as to corrective actions. It should be noticed that, in contrast to the traditional discrete-event simulation approach, KF method does not necessitate thorough activity-level information. The model inputs are the basic performance indicators (EV, PV, and AC as used in the terminology of the earned value method) and initial estimations of the project duration and cost at completion.
5.2.1. Probability of Success Graph
The probability of success graph supports the decision maker to estimate the probability of finalizing the project within the predetermined duration, according to the probability distribution functions. The probability distribution diagrams are obtained based on accumulative distribution function. According to the simulation results, the planned (prior) and posterior cumulative probability distribution curves of success are illustrated in Figure 6. These graphs signify the probability of finishing the project at a given due date.
The prior distribution is associated with the planned duration based on the expected variance quantified through engineering judgment. On the other hand, posterior probability distribution for the EDAC is estimated during the project execution. More specifically, the prior and posterior probability distribution curves signify the probability of completing the project at a given period, based on the initial forecasts and the updated predicting results, respectively. It is assumed that the prior variance of project duration equals the posterior variance. In this study, the prior variance of the project duration is estimated using three-point PERT formula. At 50% probability, as presented in Figure 6, the estimated EDAC at the end of the 7th mouth is approximately 1006 days. This result indicates that the schedule performance is behind the initial plan by 193 days. According to the worst-case scenario given at the 99% probability level, the EDAC is 1712 days, and then the schedule performance at 1% risk level is 899 days behind the initial schedule.
5.2.2. Probabilistic EDAC Profile
In this subsection, the probabilistic EDAC profiles obtained from the KFFM computations are discussed. These profiles include four curves: EDAC, planned, lower bound (LB), and upper bound (UB) that represent the probabilistic forecasts for the project duration. The estimation bounds of the EDAC can be obtained directly from the Kalman filter calculations according to the error covariance matrix . The red dashed line (EDAC curve) is displayed in Figure 7. It represents the estimated duration at completion index computed using the mean parameter of the posterior distribution function. As more progress is achieved, the EDAC, UB, and LB curves approach their final results. The UB and LB curves are considered at a desired confidence level (95%). For schedule performance forecasting reporting, the KF affords an accurate EDAC as 7th month. As can be seen, the EDAC produced by CPM method is placed within the UP and LB curves generated by the KF. The probabilistic EDAC profile indicates timely warning about a potential risk of schedule performance downgrade. The results indicate that the KF can be efficiently used to arrange for well-timed warnings about potential performance loss.
5.2.3. Probability of Success Profile
The probability of success profile supports the decision-making process in regard to risk management actions. The probability of success profile is illustrated in Figure 8. This graph is related to the likelihood of satisfying the time performance of the project. This profile is used as warning mechanism at a particular level of risk. As can be seen, the probability of success has dropped from 50% at the project start to 11% at 6th month indicating that the project is under the risk of delay. After that period, the probability of success profile increased to 14% at the 7th month. In this period of time, the project status is ahead of schedule with 14% probability of completing on time.
5.3. Cost Performance Assessment
As mentioned previously, the accuracy of forecasting CEAC is improved by employing a regression-based nonlinear methodology that integrates a logistic growth model with earned schedule method. In this section, this assertion is tested and the obtained results are discussed. The results of regression-based cost forecasting and analysis are given in Table 5.
The logistic model is fitted as (36) using Minitab software based on nonlinear regression analysis with Gauss–Newton algorithm. In the software setting, the confidence level is considered as 95%:
Figure 9 presents the graph of the fitted cost progress curve. As can be seen, the curve fits the actual cost and planned value data of the project. The curve fits the cost estimation at completion (as response variable) with an input of time being complete (as predictor of the fitting model). The obtained result indicates that, at 7th month, the cumulative project cost is about 58% of the total BAC.
According to the calculated properties of (36), the inflection point occurs when the project time progress is 50% and the cost is about 35% of the total BAC. Table 6 presents results of cost estimation for project case study. After 7 months of execution, the project is 76% complete and consequently this is the period in which the CEAC is calculated. At this time point, the cumulative project cost is about 58% of the total BAC.
The final step of the cost estimation process requires integration of the value of the CF to consider the effects of the schedule progress into the project’s cost. The value of the CF for project case is 1.37. In (9), the value of the time is substituted by CF as expressed in (10). The forecasted CEAC during the different time periods is calculated as illustrated in Figure 10. At the end of the 7th, value of the EVM-based CEAC is to some extent more than the final cost estimation by the KF/LC. The final cost estimations have more accuracy than those obtained without considering the CF.
5.4. Validation
In this section, we provide the comparison of the proposed time and cost performance measurement against the index-based forecasting methods. The EDAC index using CPM method is calculated according to the actual data. The EDAC is calculated by EV and ES approaches using (37) and (39), respectively:
Figure 11 shows the EDAC profile generated by the deterministic models (EVM and ES) and one produced by the KFFM. The percentage of error (PE) between the EDAC forecasted by the benchmark approaches against is calculated aswhere is the estimated duration at completion generated by the combined KF and learning curve analysis and is the estimated duration at completion produced by the CPM. The average error percentage is considered as average of the summation of all error percentages as summarized in Table 7. It should be remarked that CPM estimate the time performance at the activity level. Thus, it would be expected that CPM be the most accurate forecasting model among other approaches.
The results of Table 7 indicate that the KF/LC is, on average, the best model because its EDAC profile had the lowest mean and standard deviation of error as against the EDAC profile generated by the CPM Profile, while EDAC profile produced by the EVM, ES, and pure KF models has a greater mean and standard deviation of forecasting error.
As shown in Figure 11, the black line represents the EDAC profile generated by CPM. As it can be observed, the EDAC profile calculated by the KF/LC model had better intimacy to EDAC profile produced by CPM as against the EVM and ES methods. KF/LC generates the best EDAC profile since it has the lowest deviation from the EDAC profile calculated by CPM. On the other hand, the EDAC profile of EVM and ES methods has much greater error compared with KF and KF/LC methods. As a result, based on such comparison, it should be concluded that the KF/LC provides more reliable time performance predictions against the EV and ES performance forecasting approaches.
6. Conclusion Remarks
Existing methods of project performance assessment, for example, earned value management, are deterministic and therefore may fail to characterize the inherent complexity and associated risks in forecasting the performance of the innovative projects. In this study, the earned value methodology was extended to address the effect of learning on the performance of the innovative project under risk condition. These effects have so far been ignored in most earned value management applications. In the present study, EVM approach was extended by Kalman filter and learning curve to forecast the DEAC and then regression curve-fitting approach for cost forecasting adopted the growth model to predict the final cost at completion during different time periods. So schedule and cost forecasting were combined within a reliable approach. The practical benefits of the proposed regression curve-fitting approach are that it relates the past existing data with forthcoming planned data, while the traditional EVM approach exclusively relies only on historical performance data. This relationship between past, current, and future performance of the company was attained by the implementation of the logistic growth model.
The accurateness of the EVM, ES, KF, and KF/LC forecasting methods was assessed extensively at different forecasting periods. The comparative result exhibited that the KF/LC model was, on average, the best forecasting model because it had the lowest average and standard deviation of the error as against the EVM, ES, and KF models. Consequently, it can be concluded that the KF/LC provides more reliable performance forecast than the other two deterministic EVM and ES approaches as well as pure KF method. Furthermore, the combined KF/LC performance measurement model developed in this study affords probabilistic prediction bounds of EDAC and generates lower errors than those achieved by EVM and ES estimating approaches.
The future research aims at extending the performance measurement model that accounts for different learning functions. Accordingly, the model characteristics can be improved by addressing more realistic situation, for example, the incorporation of the time buffers and cost contingency as well as the organizational learning. The combined risk assessment and performance forecasting methodology can be compared with other artificial intelligence based forecasting and risk approaches such as fuzzy risk analysis and artificial neural network (ANN). The prediction model can be enhanced with integration of Kalman filter method and the Bayesian estimation method. Any effort expended in improving the accurate utilization of resources assigned to knowledge-based projects would have thoughtful effects on the performance of organizations, which is principally important in current business environment where acquiring resources is becoming progressively more complex.
Competing Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.