Abstract
The purpose of this study is to evaluate the efficiency indices for 60 Brazilian electricity distribution utilities. These scores are obtained by DEA (Data Envelopment Analysis) and Bayesian Stochastic Frontier Analysis models, two techniques that can reduce the information asymmetry and improve the regulator's skill to compare the performance of the utilities, a fundamental aspect in incentive regulation schemes. In addition, this paper also addresses the problem of identifying outliers and influential observations in deterministic nonparametric DEA models.
1. Introduction
In the Brazilian Electrical Sector (SEB, for short), the supply of energy tariffs is periodically revised within a period of 4 to 5 years, depending on the distributing utility contract. On the very year of the periodical revision, the tariffs are brought back to levels compatibles to its operational costs and to guarantee the adequate payback of the investments made by the utility, therefore, maintaining its Financial and Economical Equilibrium (EEF, for short). Over the period spanned between two revisions, the tariffs are annually readjusted by an index named IRT given by where stands for the quantity related to the utility nonmanageable costs (acquisition of energy and electrical sector taxes) at the date of the readjustment, stands for the utility annual revenue estimated with the existing tariff (free of the ICMS tax) at the previous reference date IGPM (market prices index), and stands for the quantity related to the utility manageable costs (labor, third part contracts, depreciations, adequate payback of invested assets, and working capital) on the previous reference date ().
As shown in (1.1), the nonmanageable costs (VPA) are entirely passed through to the final tariffs, while the amount related to the manageable costs (VPB) is updated using the IGPM index discounted by the factor. This factor applies only to the manageable costs and constitutes the way whereby the productivity gains of the utilities are shared with the final consumers due to the tariff reduction they introduce. The National Electrical Energy Agency (ANEEL) resolution 55/2004 defines the factor as the combination of the 3 components (, and ), according to the following expression
The component accounts for the effects of the application of the IPCA index (prices to consumer index) on the labor component of the VPB. The component is related to the consumer perceived quality of the utility service and the component accounts for the productivity expected gains of the utility due to the natural growth of its market. The latter is the most important and its definition is based on the discounted cash flow method of the forward looking type, in such a way to equal the present cash flow value of the utility during the period of the revision, added of its residual value, to the utility assets at the beginning of the revision period. In summary, where is the period, in years, between the two revisions, is the value of the utility assets on the date of the revision, is the utility assets value at the end of the revision period, stands for both; the income tax percentage and the compulsory social contribution of the utility applied to the utility liquid profit, is the average capital cost, is the utility operational revenue, represents the various taxes (PIS/PASEP, COFINS and P&D), is the operational and maintenance utility costs, is the amount corresponding to the investments realized, and is the depreciation, all of them are related to year
The quantities that form the cash flow in (1.3) are projected according to the criteria proposed by ANEEL, resolution 55/2004. As an example, the projected operational revenue is obtained as the product between the predicted marked and the average updated tariff, while the operational costs (operational plus maintenance, administration, and management costs) are projected based on the costs of the “Reference Utility”, all are related to the date of the tariff revision.
To avoid the complexity of the “Reference Utility” approach and in order to produce an objective way to obtain efficient operational costs, ANEEL envisages the possibility of using benchmarking techniques, among them, the efficient frontier method, as adopted by the same ANEEL to quantify the efficient operational costs of the Brazilian transmission lines utilities [1]. The frontier is the geometric locus of the optimal production. The straightforward comparison of the frontier with the position of the utilities allows the quantification of the amount of improvement each utility should work on in order to improve its performance with respect to the others.
The international review conducted by Jamasb and Pollitt [2] shows that the most important benchmarking approaches used in regulation of the electricity services provided by utilities are based upon Data Envelopment Analysis [3] and Stochastic Frontier Analysis [4]. As cited in Souza [5], the first method is founded on linear programming, while the second is characterized by econometric models.
Studying cases of the SEB, authors such as Resende [6], Vidal and Távora Junior [7], Pessanha et al. [8], and Sollero and Lins [9] have used different DEA models to evaluate the efficiency of the Brazilian distributing utilities. On the other hand, Zanini [10] and Arcoverde et al. [11] have also obtained efficient indices for the Brazilian distributing utilities using SFA models. Recently, Souza [5] has proposed to gauge the cost efficiency using Bayesian Markov Chain Monte Carlo (MCMC) algorithm.
DEA and SFA approaches have distinct assumptions on their inner concept and present pros and cons, depending on the specific application. Therefore, there is no such statement as “the best” overall frontier analysis method.
In order to measure the efficiency (rather than inefficiency), and to make some interesting interpretations of efficiency across comparable firms, it is recommended to investigate efficiency indices obtained by several methods on the same data set, as carried out in the present work, where DEA and Bayesian SFA (BSFA hereafter) models are used to evaluate the operational costs efficiency of 60 Brazilian distributing utilities.
The paper is organized as follows. In Section 2, the basic concepts of the DEA and BSFA formulations are discussed. In addition, the Returns to Scale (RTS) question, the problem of detecting outliers, influential observations and Gibbs Sampler (MCMC) method are presented. Section 3 comments on the results. Conclusions are given in Section 4.
2. Methodology and Mathematical Models
2.1. The Deterministic DEA Approach
Data Envelopment Analysis is a mathematical programming-based approach for assessing the comparative efficiency of the set of organisational units that perform similar tasks and for which inputs and outputs are available. It is meaningful to point out that in the DEA terminology, those entities are so-called Decision Making Units (DMUs).
The survey by Allen et al. [12] reports that DEA was proposed originally by Farrell [13] and developed, operationalised, and popularised by Charnes et al. [14]. Ever since, this technique has been applied in a wide range of empirical work, such as education, banking, health care, public services, military units, electrical energy utilities, and others institutions. Zhu [15] describes that one of the reasons for this argumentation could be that DEA has the ability to measure the relative “technical efficiency” in a multiple inputs and multiple outputs situation, without the usual information on market prices.
In the framework here (DEA methodology), consider the case where there are DMUs to be evaluated. Each has consumed varying amounts of different inputs to produce different outputs . A set of feasible combinations of input vectors and outputs vector composes the Production Possibility Set (PPS, for short), defined by
It is informative, here, to stress the study developed by Banker et al. [16]. In short, they postulated the following properties for the PPS, which are worthwhile(i)postulate 1. Convexity;(ii)postulate 2. Ineficiency Postulate;(iii)postulate 3. Ray Unboundedness;(iv)postulate 4. Minimum Extrapolation.
Subsequent to some algebraic manipulations under the above-mentioned four postulates, it is possible to show that the PPS is given by where is the input matrix, is the output matrix, and is a semipositive vector in .
If postulate 3 is removed from the properties of the PPS, it can be verified that where is the unit vector. A complete presentation of this demonstration, worth reading, can be found in Forni [17].
Such results lead directly to two seminal DEA models. The first invokes the assumption of the Constant Returns to Scale (CRS) and convex technology, Charnes et al. [14]. On the other hand, the second assumes the hypothesis of Variable Returns to Scale (VRS), Banker et al. [16].
In the following section, methods for measuring Return to Scale (RTS) of the technology are presented.
2.2. Returns to Scale
As pointed out in Simar and Wilson [18], it is very important to examine whether the underlying technology exhibits nonincreasing, constant, or nondecreasing RTS. Of course, large amount of literature has been developed on the problem of testing hypotheses regarding RTS. For example, Färe and Grosskopf [19] suggested an approach for determining local RTS in the estimated frontier which involves comparing different DEA efficiency estimates obtained under the alternative assumptions of constant, variable, or nonincreasing RTS, but did not provide a formal statistics test of returns to scale. On the other hand, Simar and Wilson [18], again, discussed various statistics and presented bootstrap estimation procedures.
In some situations, it could be interesting to solve the RTS question by estimating total elasticity . Following Coelli et al. [20], this estimate, certainly attractive from the point of view of simplicity, can be computed by using the partial elasticity estimates . However, it is easy to verify that this approach will fail in the very general setup of a multioutput and multiinput scenario.
In terms of the partial elasticity estimates again, is given by
From its definition, the total elasticity is expressed as follows:
Once the value of the total elasticity is measured, immediately it is possible to identify the returns to scale type. Following Coelli et al. [20], three possible cases are associated with (2.5) as follows:(i) Constant Returns to Scale (CRS);(ii) Nondecreasing Returns to Scale (NDRS);(iii) Nonincreasing Returns to Scale (NIRS).
In conformity with what is mentioned up to here, the next section focuses on how to find the feasible DEA model based on the resulting total elasticity.
2.3. DEA Models Regarding Returns to Scale
As mentioned above, it is possible to determine the DEA best-practice frontier type through . In this context, let the CRS and VRS DEA models defined in (2.6) and (2.7), respectively, be where is a row vector of weights to be computed, is a vector of inputs for , and is a vector of outputs for .
By inspection of (2.6) and (2.7), it is remarkable to notice that the VRS model (BCC model) differs from the CRS model (CCR model) only in the adjunction of the condition . Cooper et al. [3] point out that this condition, together with the condition , for all , imposes a convexity condition on allowable ways in which the DMUs may be combined.
Based on the appointed comments, it may be found in Zhu [15] that if we replaced with , then we would obtain Nondecreasing Returns to Scale (NDRS) model, alternatively, if we replaced with , then we would obtain Nonincreasing Returns to Scale (NIRS) model.
With regard to the interpretation of these models, it is straightforward that DEA minimizes the relative efficiency index of each , comparing simultaneously all DMUs, subject to the constraints (remember that these constraints are equivalent to (2.2) and (2.3)).
Given the data, it is necessary to carry out an optimization for each of the DMUs. Accordingly, a DMU is said to be fully efficient when and, in this case, it is located on the efficiency frontier (reference set).
At this point another question arises: DEA models, by construction, are very sensitive to extreme values and to outliers. Even though Davies and Gather [21] reasoned that the word outlier has never been given a precise definition, Simar [22] defined an outlier as an atypical observation or a data point outlying the cloud of data points. This way, it is noteworthy that the outlier identification problem is of primary importance and it has been investigated extensively in the literature.
Besides this, it is important to stress that outliers can be considered influential observations. As stated by Dusansky and Wilson [23], influential observations are those that result in a dramatic change in parameter estimates when they are removed from the data. For some interesting discussions about outliers and influential observations, see also Wilson [24, 25], Pastor et al. [26], and Forni [17].
Herein, it is used to help detecting potential outlier the Wilson [24] method. This technique generalizes the outlier measure proposed by Andrews and Pregibon [27] to the case of multiple outputs. Nevertheless, as is seen from Wilson [25], it becomes computationally infeasible as the number of observations and the dimension of the input-output space increases.
This discussion ends by assuming that these very rich results obtained will be extended in the BSFA context.
2.4. The Statistical Model
The stochastic frontier models (also known in literature as composed error models) were independently introduced by Meeusen and van den Broeck [28], Aigner et al. [29], and Battese and Corra [30] and have been used in numerous empirical applications. Some of the advantages of this approach are (a) identifying outliers in the sample; (b) considering nonmanageable factors on the efficiency measurement.
Unfortunately, this method may be very restrictive because it imposes a functional form for technology.
This article uses a stochastic frontier model in Bayesian point of view. This technique allows to realize inference from data using probabilistic models for both quantities observed as for those not observed. Another feature of the BSFA framework is to enable the expert to include his previous knowledge in the model studied. For these reasons, Bayesian models are considered more flexible and thus, in most cases, they are not treatable analytically. To circumvent this problem, it is necessary to use simulation methods. The most used are the Markov Chain Monte Carlo (MCMC) methods.
2.4.1. Bayesian Stochastic Cost Frontier
The econometric model with composed error for the estimation of the stochastic cost frontier can be mathematically expressed as follows:
Assuming that is linear on the logarithm, the following model is obtained after the application of a log transformation in (2.8):
The equation (2.9) is called in literature as Translog function. When the crossed products are null, there is a particular case called Cobb-Douglas function. With this information, the deterministic part of the frontier can be defined as follows:(i)—natural logarithm of the output of the th DMU ();(ii)—natural logarithm of the th input of the th DMU (including the intercept); (iii)—a vector of unknown parameters to be estimated.
In (2.9), the deviation between the observed production level and the determinist part of the frontier is given by the combination of two components: , an error that can only take nonnegative values and capture the effect of the technical inefficiency, and , a symmetric error that captures any nonmanageable random shock. The hypothesis of symmetry of the distribution of is supported by the fact that environmental favorable and unfavorable conditions are equally probable.
It is worthwhile to consider that is independent and identically distributed (i.i.d, in short) with symmetric distribution, usually a Gaussian distribution, and that it is independent of . Taking into account the component ( ), this is not evident and thus can be specified by several ways. For example, Meeusen and van den Broeck [28] used the Exponential distribution, Aigner et al. [29] recommended the Half-Normal distribution, Stevenson [31] proposed the Truncated Normal distribution, and finally Greene [32] suggested the Gamma distribution. More recently, Medrano and Migon [33] used the Lognormal distribution. The uncertainty related to the distribution of the random term as well as the frontier function suggests the use of Bayesian inference techniques, as presented in pioneer works of van den Broeck et al. [34] and Koop et al. [35].
To this end, the sampling distribution is initially formulated. For example, considering the random term , that is, the Normal distribution with mean 0 and variance and (: Gamma function.), that is, , the joint distribution of and , given and the vector of parameters () is given by
Integrating (2.10) with respect to , one arrives at the sampling distribution where and is the cumulative distribution function for a standard normal random variable.
To use the Bayesian approach, prior distributions are added to the parameters and, following the hierarchical modeling, posterior distributions are given. In principle, prior distribution of may be any. However, it is usually nonadvisable to incorporate much subjective information on them and, in this case, appropriate prior specifications for the parameters need to be included. Here, consider the following prior distributions: (: Truncated Normal distribution.)
According to Fernández et al. [36], it is essential that prior distribution is informative ( and ) in order to ensure the existence of posterior distribution in stochastic frontier model with cross-section sample.
Following, in some cases, it is reasonable to identify similar characteristics among the companies evaluated and then, for including these information in the model, this procedure can be performed specifying for each of DMUs, a vector consisting of () exogenous variables. For these cases, Osiewalski and Steel [37] proposed the following parameterization for the average efficiency: where is the unknown parameters and, by construction, . If are dummy variables and , the distributions of may differ for different . Thus, Koop et al. [38] called this specification as Varying Efficiency Distribution model (VED, in short). If , then and all terms related to inefficiencies are independent samples of the same distribution. Again, according to Osiewalski and Steel [37], this is a special case called Common Efficiency Distribution model (CED, in short).
Regarding to priori distribution of parameters of the efficiency distribution, Koop et al. [38] suggested using with for , and , where is the hyperparameter to be determined. According to van den Broeck et al. [4], in the CED model, can be interpreted as prior median efficiency. Proceeding this way, it could be ensured that the VED model is consistent with the CED model.
In agreement with the above, it is important to present posterior full conditional distributions of parameters involved in the model
The posterior full conditional distribution of () presents the following general form: where
For ( for ), and denotes without its th element.
With regard to inefficiencies, it can be shown that they are distributed as a Truncated Normal distribution
As the posterior full conditional distribution for is known, Gibbs sampler could be used to generate observations of the joint posterior density. These observations could be used to make inferences about the unknown quantities of interest. It is worth remembering that the technical efficiency of each DMU is determined making .
2.4.2. The Gibbs Sampler (MCMC) Algorithm
According to Gamerman [39], the Gibbs sampler was originally designed within the context of reconstruction of images and belongs to a large class of stochastic simulation schemes that use Markov chains. Although it is a special case of Metropolis-Hastings algorithm, it has two features, namely
all the points generated are accepted;
there is a need to know the full conditional distribution.
The full conditional distribution is the distribution of the th component of the vector of parameters , conditional on all other components.
Again referring to Gamerman [39], the Gibbs sampler is essentially a sampling iterative scheme of a Markov chain, whose transition kernel is formed by the full conditional distributions.
To describe this algorithm, suppose that the distribution of interest is , where . Each of the components can be a scalar, a vector, or a matrix. It should be emphasized that the distribution does not, necessarily, need to be an a posteriori distribution. The implementation of the algorithm is done according to the following steps [39]: (i)initialize the iteration counter of the chain and set initial values (ii)obtain a new value from through successive generation of values (iii)change counter to and return to step (ii) until convergence is reached.
Thus, each iteration is completed after movements along the coordinated axes of components of . After convergence, the resulting values form a sample of . Ehlers [40] emphasizes that even in problems involving large dimensions, univariate or block simulations are used which, in general, is a computational advantage. This has contributed significantly to the implementation of this methodology, especially in applied econometrics area with Bayesian emphasis.
3. Experimental Results and Interpretation
To evaluate the efficiency, the utilities have been characterized by the 4 indicators marked in Table 1. The products are the cost drivers of the operational costs. The amount of energy distributed (MWh) is a proxy of the total production, the number of consumer units (NC) is a proxy for the quantity of services provided, and the grid extension attribute (KM) reflects the spread out of consumers within the concession area, an important element of the operational costs.
By now, it is useful to start for identifying the outliers among the utilities. This analysis was performed using FEAR 1.11 (a software library that can be linked to the general-purpose statistical package R), (The FEAR package is available at: http://www.economics.clemson.edu/faculty/wilson/Software/FEAR), and it is illustrated in Figure 1. In line with the study provided by Wilson [24], the log ratio plot showed in Figure 1 suggests four groups of outliers (see peaks when 1, 4, 7, and 12).
Hence, the following utilities are regarded as outliers: CEEE, PIRATININGA, BANDEIRANTES, CELESC, CELG, CEMAT, CEMIG, COPEL, CPFL, ELETROPAULO, ENERSUL, and LIGHT. It can be observed that this technique has classified the utilities with the largest markets, with geographical concentration and a strong industrial share of participation, for instance, BANDEIRANTES, CEMIG, COPEL, CPFL, ELETROPAULO, ENERSUL, and LIGHT.
Results concerning the measurement of efficiency were obtained by NIRS DEA models because the total elasticity is less than 1 (report to Sections 2.2 and 2.3).
The scores calculated, using the DEA Excel Solver developed by Zhu [15], for each of the 60 DMUs, are exhibited in Table 2.
By analyzing the scores obtained by M1 in Table 2, it can be observed that nine companies are on the best-practice frontier. Note also that seven (PIRATININGA, BANDEIRANTES, CEMIG, COPEL, CPFL, ELETROPAULO, and ENERSUL) were labeled as outliers. As shown in Souza et al. [41], these observations influence efficiency measurement for other DMUs in the sample. Given this information, it is meaningful to emphasize that these seven utilities can be considered influential observations.
In addition, to be useful for regulatory policy purposes and in line with the literature (see, e.g., Førsund et al. [42], Pitt and Lee [43], Coelli and Battese [44]), it is interesting to realize an investigation of the sources of inefficiency.
Zhu [15, pages 258–259] suggests a procedure for identifying critical output measures through the following super-efficiency model, where the th output is given as the pre-emptive priority to change where .
Four possible cases are associated with (3.1): (i) , (ii) , (iii) , and (iv) model defined in (3.1) is infeasible. In sum, the critical output is identified as the output associated with for efficient DMUs and for inefficient DMUs.
In conformity with what has been already exposed, Table 3 indicates to each , which is the most critical output measure that contributes to its inefficiency.
With respect to CEMIG and COPEL, these utilities do not present critical output measures because no feasible solution is found by solving (3.1). In short, such analysis can offer a first and reliable tool for tracing bad outputs.
Now, from a econometric standpoint, it is important to attribute a specification for the cost frontier. To this end, a Cobb-Douglas functional form was adopted, which is defined by
By the way, initially the CED Bayesian model is carried out using the free software WinBUGS (Bayesian inference Using Gibbs Sampling for Windows) that can be downloaded at http://www.mrc-bsu.cam.ac.uk/bugs/Welcome.htm.
As mentioned in Section 2.4.1, it is useful that the expert incorporates information on companies to the model. Accordingly, by inspection of M1 in Table 2, it is possible to obtain the following: prior median efficiency, that is, .
In this context, a simple summary (see Table 2) can be generated showing posterior mean, median, and standard deviation with a 95% posterior credible interval. Concerning the results summarized in M2 of Table 2, these reveal that the most efficiency scores are considerably higher than for the M1. This is due to the fact that the Exponential distribution is a bit inflexible in that it is a single-parameter distribution and it has a mode at zero. As such, it is also convenient to develop an alternative specification of the stochastic frontier model (e.g., see two parameter Gamma distribution, Greene [32]), but that is beyond the scope of this paper.
It is now the case of examining how much the outlier DMUs affect the efficiency measured for remaining DMUs. Thus, two NIRS DEA models are applied to each one of the groups of observations consisting of outliers (12 utilities) and not outliers (48 utilities). Of course, this consideration must be attributed to the Bayesian model through dummy variable. As previously seen in Section 2.4.1, this characterization refers to the VED Bayesian model. Accordingly, all the results obtained are showed in Table 4.
The comparison of the two DEA methods that have been studied so far allows the following two remarks: (i)if the efficient reference set is changed, the spanned frontier changes and, consequently, the efficiency scores;(ii)M3 has performed better than M1, since this model appraises much more efficient DMUs.
Similarly, checking on both M2 and M4, it is remarkable that the parametric nature of BSFA is found to be substantially less sensitive to outliers due to stochastic errors to be considered in this analysis.
Once the models are assessed, it is instructive to compute the Pearson correlation coefficients as well as the Spearman rank-order correlation coefficients among them. These results, statistically significant at the 5% level, are plotted in Figure 2.
(a)
(b)
Before concluding this section, it is easy to see that the histogram density in Figure 3 shows that the DEA distributions are approximately symmetrically distributed while the BSFA are distributions positively skewed. Again by looking in Figure 3, it is noticeable that big changes have occurred in the order of the distributions (DEA and BSFA). This situation is due to the assessment of outliers (e.g., inclusion of dummy variable on the BSFA model). Another interesting question concerns with the relationship between DEA and BSFA that is considerably somewhat nonlinear.
A final observation that has not been made here is that in all two Bayesian models, the chain was run with a burn-in of 20.000 iterations with 50.000 retained draws and a thinning to every 7th draw. The estimated coefficients (see Table 5) are significant and the analysis of convergence of parameters was accomplished through serial autocorrelation graphs.
4. Conclusions
The measurement of efficiency obtained by the DEA and Bayesian SFA model should express the reduction in operational costs. In accordance with that has been already exposed, the potential reduction of the operational costs for the th utility, that is, the operational cost recognized by the regulator, is equal to .
Accordingly, it is interesting that the analyst investigates the presence of outliers and influential points because they can affect the DEA scores. In the current paper, this issue has been dealt with, besides identifying critical output measures for each utility. In addition, it can be ascertained that the conjoint analysis of DEA and Stochastic Frontier in the Bayesian approach is fundamental. Indeed, this is demonstrated through easy incorporation of prior ideas and formal treatment of parameter and model uncertainty. An important aspect of BSFA is the calculation of the credible interval for the points estimated of technical efficiency.
Finally, further studies include using cluster analysis to find groups of similarity among the Brazilian electricity distribution utilities, so that the definition of frontier efficiency respects the heterogeneity of electricity sector in Brazil. Also, it is convenient to apply nonradial DEA techniques, different functional forms of the cost function, as well as other distributions to capture the effect of the technical inefficiency.
As a result, it is possible to draw conclusions of major significance for regulatory policy purposes.
Acknowledgments
The authors are grateful to anonymous referees for their helpful comments and to Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) for its financial support.