Abstract
All projects take place within a context of uncertainty. That is especially noticeable in complex hydroelectric power generation projects, which are affected by factors such as the large number of multidisciplinary tasks to be performed in parallel, long execution times, or the risks inherent in various fields like geology, hydrology, and structural, electrical, and mechanical engineering, among others. Such factors often lead to cost overruns and delays in projects of this type. This paper analyzes the efficiency of the Earned Value Management technique and its Earned Schedule extension, as means of forecasting costs and deadlines when applied to complex hydroelectric power production projects. It is worth noting that this analysis was based on simulation models applied to real-life projects. The results showed that cost forecasting becomes very accurate over time, whereas duration forecasting is not reliably accurate.
1. Introduction
The production of hydroelectric power is based on the installation and operation of large generation plants, which are considered to be highly complex projects. Examples from the international arena demonstrate complexity with statistics revealing significant deviations in terms of costs and execution periods compared to scheduled costs and deadlines. Various studies, such as those by the University of Oxford [1], the Institute for Energy & the Environment [2], and Awojobi & Jenkins [3], analyzed samples of 58 to 235 hydroelectric projects worldwide. The results are surprising, with average values ranging from increases of 27% to 99% over planned costs and increases up to 44% in terms of execution periods.
A hydroelectric power production project is made up of many interrelated and interdependent subsystems. These subsystems entail several engineering disciplines, like electrical, electronic, civil, mechanical, industrial, and environmental engineering, among others. Given such a context, a number of different organizations will naturally be involved in these types of projects. Complex projects are considered to be dynamic systems that develop within an environment of great uncertainty and unpredictability [4]. They call for an exceptional level of management and in fact applying systems developed for ordinary projects is normally insufficient for the level of complexity involved in the execution of such complex projects. The effect of complexity on the project is that it hinders the identification of goals and objectives and it bears upon the duration, costs, and quality of the project, which in turn affects the project’s development and management [5–7].
Properly identifying and handling the complexity of a project in its early stages is a critical factor for success. Implementing follow-up and control tasks during the development of a project aims at monitoring performance as well as obtaining projections and measurements that may anticipate possible deviations to the schedule. Time and cost management is closely linked to the monitoring and control of any project and a fundamental support of the processes required to manage project completion within the deadlines. In the same way, they enable the project to be completed within the approved budget [8–10]. The main factors that affect the complexity of a project are its size, interdependence and interrelations, goals and objectives, and multidiscipline engineering and technology teams, among others [11].
Earned Value Management (EVM) is one of the most widely used techniques worldwide with which to assess a project’s performance during its execution. Its application in project management has extended to important institutions like the National Aeronautics and Space Administration (NASA). The basis of this technique was presented by the US Department of Defense (DoD) in the 1960s and was further developed and improved during the 1970s and early 1980s. In 1998, the American National Standards Institute (ANSI) and the Electronic Industries Alliance (EIA) published guidelines for EVM. In the year 2000, the Project Management Institute (PMI) added the terminology and basic formulas of EVM [12]. An important extension to EVM is the Earned Schedule (ES) technique, which analyzes timing performance and provides key indicators based on duration [13].
This paper analyzes the efficiency of EVM and its extension as tools for forecasting costs and duration in complex hydroelectric power production projects. The analysis was based on applying these techniques to simulation models of the schedules of actual power production projects.
2. Models and Techniques
To undertake the proposed study, four complex hydroelectric power production projects were considered. Stochastic models were built for all of them, in which the random variables were defined as cost and duration of the jobs that make up the respective execution schedule. The projects used are real-life cases and their characterization is addressed in later sections.
By using numerical methods, simulations were set up that enabled the probability of occurrence by project completion for cost and duration to be calculated. In each simulation, EVM techniques and its ES extension were applied and an efficiency analysis of these techniques as means of forecasting cost and duration was performed. The main stages in the methodology applied are described below.
2.1. Stochastic Model of the Schedule
The uncertainty surrounding any complex hydroelectric power production project renders its planning necessarily dynamic and variable over time. In such a context, a simulation model is a powerful tool to enable a virtual study of the project’s actual behavior. A good simulation model allows for properties of the project planning and schedules to be depicted, ascertained, and predicted [14, 15]. Furthermore, it is important to bear in mind that the way the schedule is developed has a significant influence on the overall management of any project [16].
The models were based on schedules made up of a distributed series of interrelated tasks, and for each of the tasks likely cost and duration ranges were assigned. Figure 1 shows a graphic illustration of this idea. The dependency between tasks and the allocation of cost and duration to them were defined using conventional techniques: the Program Evaluation and Review Technique (PERT) and the Critical Path Method (CPM). The CPM allowed for total duration of the project to be worked out once the individual duration of each task is known, whereas the PERT provided the means with which to build uncertainty into those durations [17, 18]. The Monte Carlo method was used to simulate the model, which entails a random numerical method that uses statistical sampling techniques to find approximate solutions to quantitative problems [19]. On each run of the models, ten thousand scenarios were simulated for each schedule.
To characterize cost and duration variability in each task, probability distributions were used, specifically the PERT distribution function. In this way, the minimum, most likely, and maximum values were estimated according to American Association of Cost Engineering (AACE) recommendations [20]. In this regard, the research took advantage of the lead author of this paper’s 17 years’ prior experience in hydroelectric power production projects in the Republic of Ecuador. Cost and duration randomness was estimated on the basis of possible variations in the quantities and technical details of each individual task. Exceptional events that might occur in projects of this type, like errors in prior studies and project designs, lack of financial resources during project execution, or natural phenomena like earthquakes, among others, were not taken into account.
The models were developed on Microsoft Excel using Palisade @Risk software [21]. This software enables the use of Monte Carlo simulation techniques and can generate distributions of possible outcomes of any cell or range of cells on the model spreadsheet [22]. The decision to use @Risk for this research was based on the expertise of the firm that develops the software, which has maintained it on the market since 1987, currently with over 150,000 users in more than 100 countries across the world and with translations into seven languages. Other software options with significant experience are Crystal Ball by Oracle and Risk Simulator by Real Options Valuation [23, 24].
2.2. EVM Technique and Earned Schedule
The Earned Value Management technique evaluates the performance of a project during its execution by monitoring the integrated management of its scope, schedule, and costs. Specifically, this technique compares baseline performance with actual performance in terms of duration and costs [25].
To do so, the technique takes a series of fundamental measurements as the basis. The Plan Value (PV) refers to the sum of the planned costs for each task in each time period, from day 1 or the commencement of project implementation up to completion. The Earned Value technique considers this cost planning as the baseline that will serve for future performance comparisons. At another level, Actual Cost (AC) refers to the sum of the costs actually incurred in each task and for each time period. Earned Value (EV) refers to the work actually performed, expressed as a cost. This measurement is calculated by multiplying the percentage of the actual physical progress of each task by its budgeted cost. Finally, Budget at Completion (BAC) is the total budget as estimated in the project plan.
In addition, the Earned Value technique calculates indicators that numerically represent the performance of the project in terms of cost and schedule over a given period of time. The main measurements are Cost Variance (CV), Cost Performance Index (CPI), Schedule Variance (SV), and Schedule Performance Index (SPI).
The Earned Value technique analyzes the trends in project performance in terms of cost and duration, i.e., based on the historical performance readings calculated for each period (CV, CPI, SV, and SPI); it makes forward-looking statements and compares them to the initial budget and schedule. The main measurements for forecasting are Estimate at Completion (EAC) and Estimate to Complete (ETC).
Figure 2 is a Cost/Time graph showing the main indicators and predictors the EVM technique calculates.
An important extension to EVM is the technique called Earned Schedule. This technique consists of calculating new values for SV and SP but based on the variable Time and not on Cost, which the Earned Value technique normally calculates. This is achieved by calculating projections on the Earned Value curve for the Plan Value compared to the abscissa of the Time variable [26, 27].
This technique is based on the variables Schedule at Completion (SAC), Earned Schedule (ES), and Actual Time (AT). In addition, the main indicators are Time Estimate at Completion (TEAC) and the Schedule Performance Index (SPI(t)). Figure 2 presents a Cost/Time graph showing the main indicators and predictors that the Earned Schedule technique calculates.
EVM finds application in very varied contexts. It is a technique traditionally applied in the construction industry [28–30], where it has been regularly and successfully applied. But it has also been applied in different highly specialized contexts with a large number of special features and needs. Examples of this can be the projects developed by NASA within the framework of aerospace engineering [12] or the construction of nuclear plants [31]. These examples show the ability of the technique to be adapted to different scenarios. In this sense, there are interesting studies in which EVM is applied to the analysis and improvement of productive processes [32–35].
To verify the efficiency of EVM and its ES extension as techniques for forecasting cost and duration, the indicators contained in these techniques were calculated and compared with results obtained from the simulations, as explained below.
For each of the ten thousand iterations of the model, the cost prediction indicator proposed by EVM and corresponding to EAC was calculated. Furthermore, the simulated final cost (FCS) of the project was calculated and compared with EAC for each iteration. The efficiency of the prediction indicator is confirmed when the difference between EAC and FCS is similar to zero. Accuracy tolerance range was 2% of the total budgeted cost of the project. These calculations and comparisons were made in periods of 5 months up until the scheduled period for project completion.
In addition to that, for each of the ten thousand iterations in the model, the prediction indicator for duration proposed by the ES technique and giving the TEAC was calculated. Furthermore, simulated final project time (FTS) was calculated and compared with TEAC for each iteration. The efficiency of the prediction indicator is confirmed when the difference between the TEAC and FTS is similar to zero. Accuracy tolerance range was 5% of overall scheduled project time. These calculations and comparisons were made in periods of 5 months up until the period scheduled for project completion.
2.3. Case Studies
Four actual hydroelectric power production projects were taken as case studies. The basic technical details are given in Table 1. Currently, the projects named Santiago and Cardenillo are at the final design stage and implementation is planned for the coming years. For their part, Mazar Dudas and Sopladora are fully built hydroelectric plants now in operation [36].
The four projects are located on the River Amazon in the southeast of the Republic of Ecuador. The Santiago and Cardenillo projects are located entirely within the province of Morona Santiago at an approximate altitude of 280 and 550 meters above sea level, respectively. The Sopladora project is located in the provinces of Azuay and Morona Santiago at an altitude of 940 meters, while the Mazar Dudas project is located in the province of Cañar at an approximate altitude of 2,300 meters above sea level. Figure 3 shows their exact locations.
The schedules of the projects under analysis served as the input data for the simulation models. Project schedules list the series of jobs or tasks, how they relate to each other, their costs, and their distribution over time. Table 2 shows the main features of project schedules, where the number of tasks, costs, and total duration are presented in detail. These schedules are the ones prepared and used by the construction companies building the Mazar Dudas and Sopladora plants. In the case of Santiago and Cardenillo, the schedules were created by engineering firms. Depending on each company’s criteria, schedules can be more or less detailed in terms of the total number of tasks or steps, which typically range between 120 and 1306. Based on their expertise, the designers and builders of these projects defined duration and total cost within a range of 27 to 75 months and US$56 to US$2,684 million. The forecast duration and costs illustrate that these are large scale projects.
The schedules enabled Accumulated Cost-Duration Curves (S-Curves) to be plotted that represent the baselines for comparing project progress in accordance with the EVM technique. The baselines for the projects in question are presented in Figure 4.
3. Results and Discussion
Probability ranges were obtained by simulating schedule implementation at the hydroelectric power production projects in our case study.
Over the full duration of the projects, the models calculated possible ranges of occurrence. Table 3 shows the probabilities calculated for durations equal to or less than the scheduled duration of the projects in this case study. In the same way, estimated duration for each project is shown, with 95% likelihood of occurrence.
The values presented in Table 3 correspond to the results from the relative frequency histograms of duration returned by the stochastic models. Figures 5(a) and 5(b) show the histograms from the Sopladora project model. The abscissa axis represents duration in months, while the top bar shows the calculated probability ranges.
(a)
(b)
As far as total cost of the projects is concerned, the models estimated possible ranges of occurrence. Table 4 shows the probabilities calculated for costs equal to or less than the planned costs for the projects used in our case study. Estimated costs for each project with a 95% probability of occurrence are also presented.
The values presented in Table 4 correspond to the results of the relative frequency histograms of cost from the stochastic models. Figures 6(a) and 6(b) present the histograms resulting from the Cardenillo project model. The abscissa axis represents cost in millions of US dollars, while the upper bar shows the estimated probability ranges.
(a)
(b)
By applying the EVM technique and its ES extension to the models, the probable accuracy of the techniques was determined in regard to the results obtained from the simulations. The closer this probability is to 100%, the greater the efficiency and accuracy of the prediction are.
In regard to the cost and duration forecast, the models produced relative frequency histograms in which the probability ranges were calibrated at 2% of the total project cost and 5% of the total scheduled duration of the project. Figure 7(a) presents an example using the cost histogram for the Santiago project for period 40 and with the probability range calibrated to US $-26.8 and $+26.8 million, representing the 2% range of the total budgeted cost. Meanwhile, Figure 7(b) shows another example, this time the duration histogram for the Mazar Dudas project for period 10 and with the probability range calibrated to +0.7 and -0.7 months, representing 5% of the total scheduled duration.
(a)
(b)
Details of the cost and duration results obtained for each of the projects in our case study are given below.
3.1. Santiago Project
As far as forecast cost upon completion of the Santiago project is concerned, probability readings were obtained that reflect increased efficiency or accuracy of EVM in the various periods of time under analysis. This situation is set out in Table 5. In month 5 of execution, the probability of the predicted cost meeting the cost of the simulation was 23.00%, while in month 45 of execution, that probability reached 99.50% and in the following months it remained at 100% until completion. These probability calculations were based on a range of 2% of total budgeted cost (± US$ 26.8 million).
As far as the forecast for project duration is concerned, as Table 5 shows, the probability rates obtained for the different periods of time in the assessment improved their accuracy over time, reaching 52.10% in month 68, which was the last month in the project schedule. Probability calculations were based on a range of 5% of scheduled duration (±1.7 months).
3.2. Cardenillo Project
Turning to the cost forecast upon completion of the Cardenillo project, probability rates were obtained that demonstrated an increase in EVM accuracy or efficiency over the different periods of time under analysis, as seen in Table 6. In month 5 of execution, the probability of the predicted cost matching the simulated cost was 17.80 %, whereas in month 50 of execution, probability had risen to 99.60% and in the following months remained at 100% until completion. Probability calculations were based on a range of 2% of total budgeted cost (±US$ 9.4 million).
Table 6 shows forecast duration, in which the probability rates reveal improved accuracy over time, reaching 59.00% in period 70, which is very close to the scheduled rate upon completion. Only in period 75, which represents scheduled completion, do we see a leap in probable accuracy up to 95.60%. The probability calculations were based on a range of 5% of scheduled duration (±1.9 months).
3.3. Mazar Dudas Project
Forecast cost upon completion of the Mazar Dudas project returns probability rates that indicate greater accuracy, as shown in Table 7. In month 5 of execution, the probability of the predicted cost matching the simulated cost was 71.00%, while in month 15 of execution, the probability was 99.80% and in the following months it stood at 100% until completion. The probability calculations were based on a range of 2% of total planned cost (±US$ 0.6 million).
As Table 7 demonstrates in regard to forecast duration of the project, the probabilities obtained improved their accuracy over time, to reach a value of 44.10% in period 27, which is the last period in the project schedule. The probability calculations were based on a range of 5% of scheduled duration (± 0.7 months).
3.4. Sopladora Project
From the results presented in Table 8, it appears that the forecast cost upon completion of the Sopladora project shows probability rates with increased accuracy over time. In month 5 of execution, a probability of 64.30% was obtained, whereas in month 30 of execution, probability had risen to 97.80%. In the following months, a probability rate of 100% was maintained until completion. The probability calculations were based on a range of 2% of total budgeted cost (±US$ 6.8 million).
As far as the forecast of duration for the Sopladora project goes, as Table 8 shows, the probabilities obtained improved their accuracy over time up to a value of 74.20% in period 47. The probability calculations were based on a range of 5% of scheduled duration (± 1.2 months).
Having seen the results obtained for the various projects, let us now discuss them as a whole. As far as cost forecasts using EVM are concerned, in Figure 8, one can see how the probability of accuracy in all four projects under study performs. From the execution start-up periods, the tendency is for accuracy to increase in all the projects; i.e., the efficiency of this cost forecasting is seen to improve over time. It is clear that a probable forecast accuracy of 100% is achieved in all projects after about 60% of execution time has elapsed and that rate is maintained until the project is concluded.
Turning now to duration forecasting using ES, Figure 9 shows how accuracy performed in the four projects under study. From the execution start-up periods, the probability of accuracy tended to increase in all projects. However, up until 90% of the duration had elapsed, the improvement in accuracy was minimal, with a maximum of 62% of probable accuracy in the Santiago project and even lower values in the others. After 90% of scheduled duration, the probability of accuracy is seen to improve, although its performance is not the same in all the projects. In fact, this improvement ranges between 44% and 96% at the end of the scheduled duration period.
4. Conclusions
From the results shown in Figure 8, it is concluded that using EVM as a tool for predicting the costs of a complex hydroelectric power production project is seen to produce forecasts whose efficiency improves as project implementation progresses to a point after about 60% of scheduled duration where the cost forecasts it provides match simulated costs with a 100% probability of occurrence.
Complex hydroelectric power production projects are cost intensive and call for capital outlays of many millions of dollars implemented over several years, as shown in Table 2. In this context, EVM is a powerful tool that enables very efficient cost forecasting after approximately 60% of execution time has elapsed, thus allowing sufficient time for decisions to be taken to correct any budget deviations identified.
The ES technique, as an extension of EVM applied to complex hydroelectric power production projects, returned duration forecasts that tend to improve their accuracy as project implementation progresses, as seen in Figure 9. However, these trends are individual and differ among the four projects in our study. Furthermore, until 90% of scheduled duration has elapsed, its forecasting efficiency is seen to be low, as the best duration forecast corresponding to the Santiago project showed an accuracy probability of 62% at 88% of scheduled duration. After 90% of scheduled duration, the tendency is for forecasting efficiency to improve substantially. Nevertheless, the trend varies between the projects, ranging from 44% to 96% accuracy at 100% of the scheduled duration. In light of such performance, it can be concluded that ES is not an accurate technique and does not offer sufficient reliability as a tool for forecasting the duration of complex hydroelectric power plant projects.
While ES indicators are presented in units of time, the S-Curves that the technique uses to project these indicators are based on costs, such as EV and PV. The baseline used by EVM and its ES extension prioritizes the importance of tasks exclusively on the basis of their cost, so that other fundamental variables are not taken into consideration when weighing up the importance of a task, such as duration or the significance of tasks that form part of a critical path. A complex hydroelectric power plant project is characterized by long-lasting tasks and jobs, so it is essential to consider their duration as a principal variable together with the costs.
This paper analyzes the efficiency of EVM and its ES extension as a tool for forecasting cost and duration in complex hydroelectric power production projects and has detected certain inaccuracy in its forecasting of duration. On the basis of this diagnosis and as subsequent work to be done as part of the lead author’s Ph.D. thesis, the intention is to develop a new technique as a further extension of EVM that includes duration and criticality of tasks on critical paths as the baselines, in addition to costs. This would entail building a duration forecasting tool of suitable efficiency to afford project managers enough time to make decisions when facing deviations to scheduled deadlines. This initial analysis of how traditional techniques perform when applied to real cases that the lead author has participated in should provide a solid cornerstone in the achievement of such a goal.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that no conflicts of interest exist in regard to the publication of this paper.
Acknowledgments
This paper is based on the ongoing activities that form part of the lead author’s Ph.D. thesis, which he is writing at the International School of Doctorate of the National University of Distance Education; therefore, the authors would like to express their gratitude for the support provided by this institution.