- About this Journal ·
- Abstracting and Indexing ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents

Journal of Applied Mathematics

Volume 2013 (2013), Article ID 102163, 21 pages

http://dx.doi.org/10.1155/2013/102163

## A Distribution-Free Approach to Stochastic Efficiency Measurement with Inclusion of Expert Knowledge

^{1}TD Canada Trust, Toronto, ON, Canada^{2}School of Information Technology, York University, 4700 Keele Street, Toronto, ON, Canada M3J 1P3^{3}Centre for Management of Technology and Entrepreneurship, University of Toronto, 200 College Street, Toronto, ON, Canada M5S 3E5

Received 24 October 2012; Accepted 11 May 2013

Academic Editor: Suh-Yuh Yang

Copyright © 2013 Kerry Khoo-Fazari et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

This paper proposes a new efficiency benchmarking methodology that is capable of incorporating probability while still preserving the advantages of a distribution-free and nonparametric modeling technique. This new technique developed in this paper will be known as the DEA-Chebyshev model. The foundation of DEA-Chebyshev model is based on the model pioneered by Charnes, Cooper, and Rhodes in 1978 known as Data Envelopment Analysis (DEA). The combination of normal DEA with DEA-Chebyshev frontier (DCF) can successfully provide a good framework for evaluation based on quantitative data and qualitative intellectual management knowledge. The simulated dataset was tested on DEA-Chebyshev model. It has been statistically shown that this model is effective in predicting a new frontier, whereby DEA efficient units can be further differentiated and ranked. It is an improvement over other methods, as it is easily applied, practical, not computationally intensive, and easy to implement.

#### 1. Introduction

There has been a substantial amount of research conducted in the area of stochastic evaluation of efficiency, such as the stochastic frontier approach (SFA) [1, 2], stochastic data envelopment analysis (DEA) [3, 4], chance-constrained programming (CCP) efficiency evaluation [5–8], and statistical inference to deal with variations in data. The problems associated with these methodologies range from the requirement for specifications of some functional form or parameterization to the requirement of a substantial amount of (time series) data. Relying on past and present data alone to provide a good estimation of the efficient frontier may not be suitable today due to the rapid evolution of these “*nuisance*” parameters. Hence, the inclusion of management's expert opinion cannot be excluded in efficiency analyses.

This paper proposes to develop a new efficiency benchmarking methodology that is capable of incorporating probability while still preserving the advantages of a function-free and nonparametric modeling technique. This new technique developed in this paper will be known as the DEA-Chebyshev model. The objectives are to first distinguish amongst top performers and second to define a probable feasible target for the empirically efficient units (as they are found from the usual DEA models) with respect to the DEA-Chebyshev frontier (DCF). This can be achieved by incorporating management's expertise (qualitative component) along with the available data (quantitative component) to infer this new frontier. The foundation of DEA-Chebyshev model is based on the model pioneered by Charnes et al. in 1978 [10] known as DEA. It is deterministic approach, which requires no distributional assumptions or functional forms with predefined parameters. The main drawback to deterministic approaches is that they make no allowance for random variations in the data. The DEA methodology has been chosen as a foundation for this research because of the following advantages.(i)It is nonparametric and does not require *a priori* assumption regarding the distribution of data(ii)It has the ability to simultaneously handle multiple inputs and outputs without making prior judgments of their relative importance (i.e., function-free)(iii)It can provide a single measurement of performance based upon multiple inputs and outputs.

DEA ensures that the production units being evaluated will only be compared with others from the same “cultural” environment, provided, of course, that they operate under the same environmental conditions.

The rest of the paper is organized as follows. Section 2 gives a brief literature review. Section 3 describes some possible causes of data discrepancies that may or may not be observable and their effects on the variables. Section 4 discusses the assumptions and mathematical formulation of DEA-Chebyshev model. Section 5 provides the simulation and comparison with other efficiency evaluation techniques. Finally, our conclusions are presented in Section 6.

#### 2. Literature Review

This section provides the applicable literature on past and present researches relating to stochastic models and weight-restricted models designed for performance measurements. They show the relevance of well-known methodologies used for estimating efficiency scores and constructing the approximated frontier in order to account, as well as possible, for noise which can have diverse effects on efficiency evaluation of human performance-dependent entities.

##### 2.1. Stochastic Frontier Approach

Aigner et al. [1] and Meussen and Van Den Broeck [2] independently and simultaneously proposed a stochastic frontier model known as the Stochastic Frontier Approach (SFA) for performance evaluation. SFA uses econometric methods for estimating the efficient frontier. The problems associated with SFA are, that weights (or parameters) have to be predefined to determine its functional form and this requires parameterization. Second, a distributional form must be determined in order to estimate random errors. Third, inclusion of multiple outputs is not easy to incorporate into the model. Finally, samples have to be large enough to be able to infer the distributional form for random errors.

##### 2.2. Stochastic DEA

Stochastic DEA is a DEA method that attempts to account for and filter out noise by incorporating stochastic variations of inputs and outputs while still maintaining the advantages of DEA [4]. The method relies on the theory that there will always exist an optimal solution for industrial efficiency. The variability in outputs is dealt with using the risk-averse efficiency model by Land et al. [11] with a risk preference function. Kneip and Simar [3] proposed a nonparametric estimation of each decision-making unit (DMU)'s production function using panel data over time periods. This filters the noise from the outputs. The fitted values of the outputs along with the inputs are then evaluated using DEA. In this instance, efficiency is determined by the distance of the estimated frontier to the observed DMUs. The drawback of this method is that a reasonable estimate of efficiency can be obtained only when and (number of DMUs) are sufficiently large.

##### 2.3. Chance-Constrained DEA

Chance-constrained programming was first developed by Charnes and Cooper [5] and Kall [7] as an operational research approach for optimizing under uncertainty when some coefficients are random variables distributed according to some laws of probability. The CCP DEA models in the past generally assumed that variations observed in the outputs follow a normal distribution. Variations in inputs are assumed to be the cause of inefficiency [12], while random noise occurs in outputs. Since the distribution of inefficiency is uncertain (although, theoretically assumed to be half-normal or gamma), the chance-constraint formulation is not applied to input constraints (inputs are held deterministic, while outputs are stochastic). Olesen and Petersen [9] state that the hypothesis concerning the amount of noise in the data cannot be tested. Using panel data, variations in the data can be dichotomized into noise and inefficiency. Another variation of CCP DEA was introduced by Cooper et al. [6] utilizing the “*satisficing concepts*.” The concept is used to interpret managerial policies and rules in order to determine the optimizing and *satisficing* actions, which are distinguished from inefficiencies. Optimizing and *satisficing* can be regarded as mutually exclusive events. The former represents physical possibilities or endurance limits and the latter represents aspirational levels.

All these CCP formulations have considered normal distributions for the probability of staying within the constraints. This method is effective when qualitative data is not available. However, expert opinion from management cannot be discounted with regard to data dispersion from the expected or correct values. Unfortunately, the current CCP is strictly a quantitative analysis based on empirical data and whose variations are said to be of a predefined distributional form.

##### 2.4. Assurance Region and Cone-Ratio Models

In an “unrestricted” DEA model, the weights are assigned to each DMU such that it would appear as favourable as possible, which is an inherent characteristic of DEA. Hence, there is a concern when largely different weights may be assigned to the same inputs and outputs in the LP solutions for different DMUs. This motivated the development of weight-restricted models such as the “assurance region” (AR) [13, 14], the “cone-ratio” (CR) [15], and other variations of these models.

The motivation behind weight-restricted models is to redefine the DEA frontier so as to make it as *practical* as possible; that is, altering the inherent characteristic of DEA when assigning small/large weights to certain inputs or outputs is not realistic. On the contrary, the stochastic frontier models redefine the frontier in the presence of noise or data disparity. Stochastic approaches are designed to evaluate DMUs based on the understanding that constraints may, *realistically*, not always hold due to noise. Weight restrictions are also applicable in stochastic approaches.

Weight restriction models deal directly with the model’s inconsistencies in a practical sense using qualitative information, whereas stochastic models deal with data discrepancies and inconsistencies using quantitative approaches to infer to the degree of data disparity. Although the motivations of these two methods are similar, the underlying objectives for their developments are not the same. Both are valid extensions of the normal DEA model in attempting to correct the frontier.

The Assurance Region (AR) model was developed by Thompson et al. [13] to analyze six sites for the location of a physics lab. This approach imposes additional constraints in the DEA model with respect to the magnitude of the weights. The AR is defined to be the subset of , the weight space that denotes the vectors of multipliers consisting of and , such that any region outside the AR does not contain reasonable input and output multipliers. An additional constraint for the ratio of input weights [14] can be defined as
where denotes the number of inputs, and are the weights for the input * *and input 1, respectively, and and are the lower and upper bounds for the ratio of multiplier.

The cone-ratio (CR) method was developed by Charnes et al. [15] which allows for a closed convex cones for the virtual multipliers. It is a more general approach compared to that of the AR. In the AR model, there can only be two admissible nonnegative vectors, one for the lower bound and the other for the upper bound of the ratio of virtual weights. However, in the CR case, there can be admissible nonnegative vectors for input weights and admissible nonnegative vectors for output weights; that is, the feasible region for the weights is a polyhedral convex cone spanned by and admissible nonnegative direction vectors for inputs and outputs, respectively, where represent the vectors and are the weights applied to select the best nonnegative vector. Similarly, the AR method is equivalent to selecting only two admissible vectors under the CR method. The lower and upper bounds are denoted as vectors in the two-input case respectively.

#### 3. Data Variations

##### 3.1. Two Error Sources of Data Disparity Affecting Productivity Analysis

Before we begin to make modifications to incorporate probability into the basic DEA model, it is crucial that the types of errors are identified, which are sources of data disparity. These can be segregated into 2 categories; systematic and nonsystematic errors. Nonsystematic errors are typically defined to be statistical noise, which are random normal and independent and identically distributed (i.i.d.). They will eventually average to zero. Systematic errors are defined to be “*the degree to which the measured variable reflects the underlying phenomenon depend on its bias and variance relative to the true or more appropriate measure*” [16]. Systematic errors or measurement errors are deemed to have the most disparaging effects because they introduce bias into the model. These may be caused by the lack of information.

The design of the new DEA model is intended to take into account the possibility of data disparity that affect productivity analysis while preserving the advantages that DEA offers in order to estimate the true level of efficiency. Due to data disparity, normal DEA results may contain two components of the error term. The first refers to statistical noise which follows a normal distribution, while the second refers to the technical inefficiency which is said to follow a truncated normal or a half-normal distribution. This can be achieved by relaxing the LP constraints to allow for these variations which may provide a better approximation of the level of efficiency.

The following general linear programming model illustrates the mathematical form of systematic and nonsystematic errors as defined previously. Variation in the variable () of the objective function will result in different values for the optimized coefficient ()
If the variation in is stochastic, then ; , by the Central Limit Theorem; one can characterize how closely the vector * *is scattered around its mean by the distance function; . denotes the variance-covariance matrix for [4].

Four scenarios are illustrated later which describes sources of data disparity. The notations are as follows: : observed input for for , : observed output for for , : expected value of input for , : expected value of output for , : bias of , : estimate of , : bias of , : estimate of .The following are equations defining the relationship between the observed and true or expected values for both inputs and outputs in a productivity analysis such as SFA where measurement errors and/or random noise and inefficiencies are a concern in parametric estimations: for some input for unit for some output for unit (considered for cases in which there may be some bias in output levels) unrestricted in sign.

The following four scenarios illustrate the impact of different errors and were constructed using the notations given previously. These scenarios follow the definition by Tomlinson [16].

*Scenario I.* Consider the following:
With zero bias and variance, observed input value is the true value . This implies that the data is 100% accurate. The expected value is exactly the same as the observed value. In reality, it is rare to have data with such accuracy.

*Scenario II.* Consider the following:
Bias is nonzero with zero variance; hence, errors are systematic. is not an unbiased estimator of . In this case, systematic errors are a problem where inputs are concerned. When measurement errors exist, the expected value is a biased estimator of the observed value. This in turn causes biases in DEA results. Empirical methods, such as DEA, make no allowance for this error and evaluate DMUs based strictly on the observed values. However, expectations of the observed values can be determined qualitatively and be incorporated into the LP.

*Scenario III*. Consider the following:
Expected value of a constant is the constant itself. Variance of a constant is zero. Hence . Bias is zero but the variance is nonzero. Hence, variations are due to statistical noise. A DMU that appears efficient may in fact be utilizing an input-output production mix that is less than optimal. Its seeming efficiency is caused by a variation to its favour. Results obtained using empirical models are prone to inaccuracy of this nature. However, the expected value will converge over time to the true value in the absence of bias.

*Scenario IV*. Consider the following:
Bias and variances are nonzero. This implies that both systematic and nonsystematic errors exist in the data. The variance corresponds to some input . The variable, , is affected by some random amount and some bias . Hence, is not an unbiased estimator of . This scenario corresponds to the drawback of empirical frontiers.

The term “measurement error” does not simply imply that data had been misread or collected erroneously. According to Tomlinson [16], it may also not be constant over time. The inaccuracy of the data collected may be due to the lack of implicit information which may or may not be quantifiable but are deemed to have the most disparaging effects because they introduce bias into the model.

##### 3.2. Chance-Constraint Programming and DEA

Deterministic methods such as DEA are not designed to handle cases in which, due to uncertainty, constraints may be violated although infrequently. Various methods have been employed to transform the basic DEA approach to include stochastic components. Two of the more popular methods are chance-constraint programming (CCP) and stochastic DEA. An extensive literature survey has revealed that CCP DEA has always assumed a normal distribution. The objective of this research is to redefine the probabilities employed in CCP productivity analysis, which would accommodate problems emanating from various scenarios where errors are independent but convoluted without assuming any distributional form. The independent and convoluted properties of the error terms make it difficult to distinguish between them, and hence, a distribution-free approach will be employed.

The advantage of using CCP is that it maintains the nonparametric form of DEA. It allows modeling of multiple inputs and outputs with ease. There is no ambiguity in defining a distribution or the interpretation of the results as had been demonstrated in the Normal-Gamma parametric SFA model [17]. CCP typically states that constraints do not need to hold “almost surely” but instead hold with some probability level. Uncertainty is represented in terms of outcomes denoted by . The elements are used to describe scenarios or outcomes. All random variables jointly depend on these outcomes. These outcomes may be combined into subsets of called *events*. represents an event and represents the collection of events. Examples of events may include political situations, trade conditions, which would allow us to describe the random variables such as costs and interest rates. Each event is associated with a probability . The triplet is known as a *probability *space. This situation is often found in strategic models where the knowledge of all possible outcomes in the future is acquired through expert opinions. Hence, in a general form, CCP can be written as
where and index of the constraints that must hold jointly. The previous probabilistic constraint can be written in its expectational form (or deterministic equivalent) where is an indicator of :

The focus of this paper is on the further development of DEA coupled with CCP. The benefit of applying CCP to DEA is such that the multidimensional and nonparametric form of DEA is maintained. To drop the *a priori* assumption discussed in [9, 11, 18] regarding the distributional form to account for possible data disparity, a distribution-free method is introduced. In [11, 18], CCP DEA input-oriented model is formulated on the basis that discrepancies in outputs are due to statistical noise while those in inputs are caused by inefficiency:

The CCP formulation shown in (14) is designed to minimize the radial input contraction factor , subject to the constraints specified. CCP DEA models in the past generally assume that the normal distribution suffices. For example, the assumption that the variation shown previously is normal, the formulation (14) can be written in the following vector deterministic form (15): where and denote the vectors of inputs and outputs, respectively. Assuming that each DMU is independent of others, then the covariance equals zero. denotes the standard deviation for which is formulated as where subscript denotes the number of DMUs. If the DMU under evaluation is , then , hence, (16) can be written as If and , then the efficiency scores calculated in CCP will be the same as that of DEA. This does not imply that all DEA scores will coincide with the CCP ones (except for ’s score). In this case the standard deviation disappears.

The first constraint in (15) states that there is a slight chance (i.e. ) that outputs of the observed unit may exceed those of the best practice units with a very small probability. is determined based on the assumption that the observed values are representative of their mathematical expectation. The second constraint is strictly deterministic which states that the best performers cannot employ more than amount of inputs, and if they do, they cannot be efficient and will not be included in the reference set of best performers.

Using the same mathematical formulation shown in (14) and (15), and by incorporating a distribution-free approach, the DCF is established.

#### 4. DEA-Chebyshev Model

The advantages of using DEA-Chebyshev model as an efficiency evaluation tool are that it provides an approximation of performance given that random errors and inefficiencies do exist, and these deviations are considered, either through expert opinion or through data inference. Nevertheless, the results should always be subject to management scrutiny. This method also provides for ranking efficient DMUs.

##### 4.1. Chebyshev’s Theorem

In a simplified explanation, the Chebyshev theorem states that the fraction of the dataset lying within standard deviations of the mean is at least where .

DEA-Chebyshev model developed in this paper will not be restricted to any one distribution but instead will assume an unknown distribution. A distribution-free approach will be used to represent the stochastic nature of the data. This approach is applied to the basic DEA model using chance-constraint programming. This distribution-free method is known as the Chebyshev inequality. It states that or equivalently

Let a random variable have some probability distribution of which we only know the variance and the mean [19]. This inequality implies that the probability of the sample mean, , falling outside the interval is at most , where refers to the number of standard deviation away from the mean using the notation in [19]. The one-sided Chebyshev's inequality can be written as as shown in [20].

Other methods considered to define the probabilities for DEA-Chebyshev model were the distribution-free linear constraint set (or linear approximation), the unit sphere method, and the quantile method. These methods were tested to determine which of them would provide the best estimate of the *true boundary* mentioned in [21]. The *true boundary* (called set ) is defined to be a two-dimensional boundary which is generated using some parametric function defined as the chance-constrained set shown later:
where and the vector are random variables. Let the function be defined as , and and denote the expected value and the standard deviation of , respectively. In this example . Twenty-nine samples were generated.

The distribution-free approaches tested were the Chebyshev extended lemma (24), quantile method (21), linear approximation (23), and unit sphere (22). The deterministic equivalent of these methods can be written in the following mathematical forms according to the notation used by [21]. is known as the quantile of order of the standardized variate of . If random variable, , belongs to a class of *stable* distributions, then the quantile method can be applied successfully. All *stable *distributions share the common properties of being specified by the parameters and of the general functional form and when convoluted will again give us . Examples of *stable* distributions are Binomial, Poisson, Chi-squared, and Normal [NOLA99].
where is an element amongst the 29 simulated samples of ; ( in this example); and . Vector * *is defined as :

Allen et al. have proven in their paper [21] that the quantile method was the least conservative, while the Chebyshev was the most conservative. When a method of estimation provides relatively large confidence limits, the method is said to be “conservative.” The advantage of those two methods is that they both have the tendency to follow the shape of the *true (real) boundary* more closely than the other two methods, that is, unit sphere and linear approximation [21]. Given that Chebyshev provides the most conservative point of view and has the inclination to follow the shape of the true boundary with no regard to distributional forms, this method was chosen as the estimation for CCP DEA. Although the error-free frontier (EFF) is unknown, we can, at best, estimate its location or estimate its shape with respect to the DEA frontier. The EFF represents the frontier where measurement errors and random errors are not present, but it does not imply *absolute* efficiency. This means that there can be room for improvement even for the DMUs on the EFF. The theoretical frontier represents the *absolute* attainable production possibility set where there can no longer be any improvements in the absence of statistical noise and measurement errors. It is undefined due to the fact that human performance limits are still undefined at the present time.

Since, we do not want to place an *a priori* assumption regarding which stable distribution best describes the random variables in DEA, the Chebyshev theorem will be used. The deterministic equivalent of (20) by Chebyshev's extended lemma is shown as (24).

*Derivation of ** in (24).* We use the one-sided Chebyshev’s inequality and the notation used by [21]:
which states that the probability that will take on a value that is greater than standard deviations away from its mean, , is at most . in chance-constrained programming can be expressed in the general form: . Hence,
Note that from here onwards as we discuss the DCF model, for simplification and clarity we will denote .

A *“k-flexibility function”* is coined because is a value that may be defined by the user (where denotes the user's certainty of the estimate) or inferred from the industry data. The unique property of is its ability to define such that it mimics the normal distribution given that random noise is present or to include management concerns and expectations with regard to their perceived or expected performance levels. This can overcome the problem of what economists coin as “nuisance parameters.” These parameters can be problems of controlling difficult-to-observe or unquantifiable factors such as worker effort or worker quality. When firms can identify and exploit opportunities in their environment, organizational constraints may be violated [22]. Because DCF allows for management input, the flexibility function can approximate these constraint violations. The mathematical formulation, implications for management, and practical definition of will be explained later.

##### 4.2. Assumptions in DEA-Chebyshev Model

Two general assumptions have been made when constructing the model. First, nuisance parameters (including confounding variables) will affect efficiency scores causing them to differ from the true performance level if they are not accounted for in the productivity analysis. Second, variations in the observed variables can arise from both statistical noise and measurement errors and are convoluted.

In the simulation to follow, as an extension to the general assumptions mentioned previously, we will assume that variations in outputs are negligible and will average out to zero [11, 18]. The variations in inputs are assumed to arise from statistical noise and inefficiency (inefficient use of inputs). Both of these errors contribute to the possible technical inefficiencies in DEA-efficient units. These possible inefficiencies are not observed in DEA since it is an empirical extreme point method. Using the same characteristics defined in SFA, statistical noise and measurement errors are said to be normally distributed , and inefficiency is said to be half normally distributed . Thus, the relationship between the expected inputs, , versus the observed, , can be written as
where denotes the convoluted error terms of input * *for .

The assumption regarding the disparity between the observed and expected inputs is to illustrate the input-oriented DEA-Chebyshev model. In input-oriented models, the outputs are not adjusted for efficiency, but the inputs are based on the weights applied to those DMUs that are efficient. This assumption regarding errors can be reversed between inputs and outputs depending on expert opinions and the objective of the analysis (i.e., input versus output-oriented models).

As an extension of Land et al. [11] and Forrester and Anderson [18], DEA-Chebyshev model relaxes the distributional assumption. In doing so, convolution of errors can be accommodated without having to specify some distributional form for both components. This method of approximating radial contraction of inputs or expansion of outputs is generally less computationally intensive than the bootstrap method, as CCP can be directly incorporated into the LP and solved in a similar fashion as the standard DEA technique. The bootstrap method introduced by Simar and Wilson [23] is more complex in that it requires certain assumptions regarding the data generating process (DGP) of which the properties of the frontier and the estimators will depend upon. However, this method of bootstrapping is nonparametric since it does not require any parametric assumptions except those to establish consistency and the rate of convergence for the estimators.

Theoretically, the DEA, algorithm allows the evaluation of models containing strictly outputs with no inputs and vice versa. In doing so, it neglects the fact that inputs are crucial for the production of outputs. However, the properties of a production process are such that they must contain inputs in order to produce outputs. Let the *theoretically* attainable production possibility set characterize the absolute efficient frontier, which is unknown, be denoted as . Thus, given that the set is not presently bounded, the inclusion is always true where , , denote the attainable set in Error-Free Frontier (EFF), DEA, and the DEA-Chebyshev frontier, respectively. It is certain that a DMU cannot produce outputs without inputs although the relationship between them may not be clear. The following postulates regarding the relationship between the three frontiers are expressed as follows.

*Postulate 1.* The DEA frontier will converge to the EFF; according to the central limit theorem [24]. Appendices A, B, and C provide the details. However, both DEA and DCF will exhibit a very slow rate of convergence to the theoretical frontier as the number of dimensions increases or when the sample size is small. This is known as the curse of dimensionality [25].

*Postulate 2.* The production possibility set of DEA is contained in that of . The DEA and the corrected frontier may likely overlap the EFF depending on the degree of data variation observed and estimated.

##### 4.3. Mathematical Formulation

An input-oriented BCC model will be used to illustrate this work. Here, is defined as the radial input contraction factor and is defined as the column vector corresponding to the “best practice” units, which will form the projection unto the frontier for an inefficient unit Consider the following chance constraint sequence as defined by Allen et al. [21]: where is a real number such that for all and for all .

Since it is difficult to establish a specific form of distribution with empirical data due to the convolution of different types of errors, a distribution-free approach is taken. In this case, the Chebyshev one-sided inequality [21] will be applied to convert (29). A deterministic equivalent can be approximated to (30) for the input of : where and , with strict inequality on the left hand side. For example, if , then ; hence, is calculated as . Based on the assumption that DMUs are independent of each other, then , for all where denotes some constant and . The value for can be defined as where denotes the probability of staying within the tolerance region defined using the one-sided Chebyshev's inequality. As increases, and the standard deviation will also increase; hence, it becomes more likely that the EFF will be within the confidence limits.

The value of can be defined such that the will be equal to or less than 1.645 so that DCF can provide a less conservative estimate of the upper and lower limits of the frontier when compared to . The standard normal distribution value has been used in the previous CCP efficiency evaluation methodology in [11, 26]. The reasoning behind wanting a less conservative estimation is because data collected will more likely be accurate than inaccurate. When , then increases exponentially into infinity. For , note that ; can be defined such that DEA-Chebyshev model provides less conservative estimates. Taking a glance at the CCP DEA developed by Land et al. [11], the results obtained, when assuming a normal distribution, can be shown to be drastically different from that of the expected frontier depending on the level of data disparity.

The deterministic LP formulation for DEA-Chebyshev model can be written in the following mathematical form: Let be an estimate for which is defined as where is a value based on management's expectations or is inferred from a time series of data which has been transformed into a single value. The model shown in (32) can also be modified such that only discretionary inputs are considered for stochastic treatment [27].

The value of can be defined such that its values are restricted between 0.5 (the point of inflection) and 0.6 if no qualitative information regarding expectations is available, but we are *almost* certain that the data obtained is accurate. The value of is then approximated as . In this case, the results produced will be less conservative than that of the normal distribution at (i.e., ). For , a deterministic model will suffice since the DEA-Chebyshev model will provide the same results as that of the DEA.

##### 4.4. The “*-Flexibility Function” *: Unique Management Interpretation

It may not be sufficient to develop a model that technically sounds with appropriate theoretical proofs. We cannot discount the fact that management expertise can play an important role in defining the corrected frontier nor should we cause the user to reject the model. Hence, DEA-Chebyshev model is designed to incorporate management input, which can become a crucial factor in the modeling process. One of the major advantages of this model is its flexibility as compared to models that require a distributional form. It can provide crucial information to management based upon their expertise and experience in their own field of specialization thereby redefining the efficient frontier.

In DEA-Chebyshev model, has a unique management interpretation and implication. It can be defined as the management’s opinion of the expected degree of competence with regard to either input or output usage. In other words, it is the *estimated degree of deviation from the observed level of performance*. The smaller the value of is, the more certain that the data is accurate and that little improvements can be made *ceteris paribus* or that expectations have been approximately met. When , then , implying that management is certain that the data they have obtained is accurate (no need to account for deviation or random effects or inefficiency) or that present expectations have been met. If , then it implies that the data obtained is extremely erroneous or that expectations are not met.

The value for is an aggregate of two factors (or two events). First, the certainty of inaccuracy is denoted by , and second, the *translated *percentage of inaccuracy is denoted by . Let denote the true/false existence of errors. When , it implies that the data is inaccurate. If , then ; otherwise, . In other words, event implies ; when the data is 100% accurate, then there is no deviation. Therefore, can be defined:

*Proof. *, since , then .

Hence, for , can be approximated as
The constant, , represents the degree of (the expert's) uncertainty.

When deviation due to errors is negligible, then % *deviation from observed *0. Hence will be at most 0.5. implying that the data is error-free, thus % *deviation from observed *. In this case, and . Based on (31), the value for should be restricted to not be less than 1, and therefore, . Otherwise, the confidence limits become too small, which implies that . We do not want this to occur because DCF should only equal DEA when there is absolute certainty that the data is error-free. Hence, must be defined such that (34b) for and zero otherwise.

##### 4.5. Approximating the Error-Free Frontier: Development of the DCF

Unlike the straightforward method in which DEA scores are calculated, DEA-Chebyshev model efficiency scores are slightly more complicated to obtain. There are five stages to the determination of the best efficiency rating for a DMU.

*
Stage I.* Determining the DEA efficient units.

*
Stage II*. Establishing the upper and lower limits for efficiency scores using DEA-Chebyshev model where the value of is defined to reflect management concerns.

*
Stage III*. Establishing the corrected frontier from the upper and lower limits calculated in stage II for DEA efficient units. The upper and lower limits of efficiency scores established by DEA-Chebyshev model for each of the DEA-efficient units form the confidence bounds for the error-free efficiency scores. These limits determine the most likely location of the EFF. The following are characteristic of DEA-Chebyshev model efficiency scores. (1)An efficient DMU with a smaller standard deviation implies a smaller confidence region in which the EFF resides, hence, this particular DMU is considered to be more robustly efficient since it is closer to the EFF. (2)It can be conjectured that for DEA efficient DMUs, and will always be true (not so for the inefficient units).(3)When* * where is a very large constant, it may be an indication that the DMU is likely an outlier.(4)In general, the mean efficiency score in DEA-Chebyshev model is such that , unless the third characteristic previously mentioned is observed.

#### 5. Simulation

Five data sets, each containing 15 DMUs in a two-input one-output scenario, were generated in order to illustrate the approximation of the EFF using the DEA-Chebyshev model. This will demonstrate the proximity of the DCF to the EFF. A comparison is drawn between the results provided by the DCF, DEA, and the CCP input-oriented VRS models as compared against the EFF.

##### 5.1. Step I: Simulation: The Data Generating Process

The first data set shown in Table 1 is known as the control group. It contains two inputs and one output generated using a logarithmic production function of the following form:
where is some constant and and * *are arbitrary weights or coefficients assigned to inputs. Input 1 has been chosen arbitrarily and input 2 is a function of , where is some arbitrary constant; in this case . This is to ensure that the frontier generated by the control group contains only efficient units and is convex. The linear convex combination in EFF consists of discrete production possibility sets defined for every individual DMU. Output is then calculated using the equation shown in (35) from a discrete set of inputs where , , and have been arbitrarily defined and are fixed for the all groups (control and experimental). The control group is one that contains no measurement errors or statistical errors and no inefficient DMUs. It will be the construct of the EFF.

The experimental groups are generated from the control group with the error components. Their outputs are the same as the control groups and are held deterministic, while inputs are stochastic containing confounded measurement errors distributed as half-normal nonzero inefficiency and statistical noise In (36a), inputs are confounded with random errors and inefficiency: where . Variability in the inputs across simulations is produced by different arbitrarily chosen and for the inefficiency component which is distributed half normally; for each simulation. Table 2 shows the details.

##### 5.2. Step II: Establishing Efficiency Scores: DEA, DEA-Chebyshev Model, and CCP Efficiency Evaluation

The DEA results were calculated using ProDEA, while CCP and DEA-Chebyshev model results were calculated using MathCad. The CCP LP formulation follows that from [11, 18], the upper and lower bounds for the CCP frontier are

Table 3 shows the results of the efficiency analysis for the DEA and CCP models. The -conditions which CCP must satisfy will be the same for the DCF. The value, , for CCP is approximately the same as that for the DCF. Although DMU11 is DEA efficient, it is not CCP efficient given that is has violated one of the two -conditions. Note that shown in Tables 3, 4, 5, and 6.

In this simulation, because we do expect data collected to be reasonably reliable, a less conservative model would be a better choice. Conservative models tend to provide results with greater standard deviation and therefore produce an estimate with less accuracy. The four simulations were designed to test CCP, DEA, and DEA-Chebyshev model to determine the accuracy of the results obtained in comparison to the EFF. The results for DEA, CCP, and DCF for all four simulations using the values of can be found in Tables 3, 4, 5, 6, 8, 9, 10, and 11. The upper (38a) and lower (38b) bounds for the constraints in the DCF formulation are given asWhen increases, also increases and so will the spread between the upper and lower bounds of .

When the degree of deviation from observed performance levels is available, the results generated using DEA-Chebyshev model are generally a more precise approximation of the EFF compared to CCP, which assumes the normal distribution. From the simulations, it has been shown that the alpha values based on the deviation from the observed level of performance consistently produce the best approximations. The estimated degree of deviation due to inefficiency from the observed level of performance is formulated as follows: where denotes management or expert defined values of data deviation (if available) and “” denotes a constant correction factor. In other words, it is a reflection of the users’ confidence of their own expectations where “” will always be greater than or equal to “0.” (deviation) is defined to be the perceived excess of inputs to observed inputs. The numerical calculations using (39) are shown in Table 7.

The Tables 8-11 show efficiency scores determined under DEA-Chebyshev model, based on the -values shown in Table 7.

##### 5.3. Step III: Hypothesis Testing: Frontiers Compared

All the efficiency evaluation tools will be measured against the control group to determine which of these would provide the best approximation method. Both CCP and DEA-Chebyshev model efficiency scores are defined in the same manner. The upper and lower bounds of the frontier determine the region where the EFF may likely be and is approximated by the DCF efficiency score, .

Using the results obtained in Step II, the four simulated experimental groups are adjusted using their respectively efficiency scores. The virtual DMUs are the DMUs from the four experimental groups in which their inputs have been reduced according to their efficiency scores from Step II, according to the contraction factor, for DEA, for CCP, and for DCF.

In this step, in order to test the hypothesis, the 12 data sets of virtual DMUs are each aggregated with the control group, forming a sample size of 30 DMUs per simulation. “DMU#” denotes the control group (or “sample one”) and “V.DMU#” denotes the efficient virtual units derived from the experimental group (or “sample two”) using the efficiency scores generated by DEA, CCP, and DEA-Chebyshev model, respectively. There are 12 data sets in total: three for each of the simulations (three input contraction factors per DMU, from DEA, CCP (normal), and DEA-Chebyshev model). The inputs for the virtual DMUs calculated from each of these three methodologies for the same experimental group will be different. The sample size of 30 DMUs in each of the 12 sets is a result of combining the 15 error-free DMUs with the 15 virtual DMUs. These 30 DMUs are then evaluated using ProDEA (software). It is logical to use DEA for our final analysis to scrutinize the different methods since this is a deterministic method, which would work perfectly in an error-free situation. The DEA results for the 4 simulations are given in Table 12.

In order to determine if the frontiers created by these models are substantially different from that of the control group (or the error-free units), the rank-sum-test and statistical hypothesis test for mean differences were used.

The DEA-Chebyshev model is scrutinized using several statistical methods, which show that there is a strong relationship between the DCF and the EFF. All the statistical tools used to test the DCF against the EFF have produced consistent conclusions that the *corrected frontier* is a good approximation of the EFF. The statistical methods used to test the DCF versus the EFF are the Wilcoxon-Mann-Whitney test (or the rank-sum test) and the -test for the differences in mean values of shown in Table 13. The rank-sum test is used to determine if the virtual DMUs established by the DCF are from the same population as that of the DMUs in the control group; if they are, then the difference in efficiency scores of both groups will not be statistically significant. This does not imply that the EFF and the *corrected frontier* are exactly the same but rather that the latter is a good approximation of the former. Its results are better than that of the CCP performance evaluation method developed by Land et al. [11] and Forrester and Anderson [18].

##### 5.4. Step IV: Efficiency Scores: DEA versus DEA-Chebyshev Model and Ranking of DEA Efficient Units

There can be more than one way of ranking efficient units. In the simplest (or naïve) case, empirically efficient DMUs can be ranked according to the score calculated as an average of the upper and lower limits from the DEA-Chebyshev model.

###### 5.4.1. Naïve Ranking

Table 14 illustrates the ranking of all DMUs. The figures in bold denote the DEA-Chebyshev model efficiency scores for the DEA efficient units. All production units are ranked in descending order of efficiency according to the average of the upper and lower limits, . An anomaly in DMU14 of simulation 3 is caused by an extremely small value for Input 2. Because the LP formulation for DEA, DEA-Chebyshev model, and CCP (normal) applies the greatest weight to the input or output in order to make a DMU appear as favourable as possible, Input 2 in this case is weighted heavily. In DEA, the mathematical algorithm does not allow the efficiency score to exceed 1.00; thus, this problem is not detected. In DEA-Chebyshev model and CCP, because efficiency scores are not restricted to 1.00, this problem arises indicating a possible outlier. It would be advisable to remove this DMU from the analysis. In this simulation, because the errors are generated randomly, the error-value for this DMU lies in the tail end of the distribution, hence, creating an outlier.

This method of ranking is naïve because it ignores the standard deviation, which indicates the robustness of a DMU's efficiency score to the possible errors and the *unobserved* inefficiency. It also does not distinguish between possible outliers and legitimate units.

###### 5.4.2. Ranking by Robustness of DEA-Chebyshev Model Efficiency Scores

The ranking in the order of robustness of a DMU begins with the efficiency score defined as . Those with are ranked from the most robust to the least robust (from the smallest standard deviation to the largest). The standard deviation is determined using the upper and lower bounds of the efficiency scores. Then the rest of the empirically efficient units are ranked based on their respective (using their standard deviations will also provide the same ranking for these units). Once all the empirically efficient units have been ranked, the remainders are organized according to their stochastic efficiency scores from the most efficient to the least efficient. The ranking of these inefficient units is very similar to that of the empirical frontier.

Ranking from the most efficient down, those DMUs which have a DEA-Chebyshev model score of (input oriented case) can fall into either of two categories: hyper-efficient or efficient/mildly efficient depending on how robust they are (based on their standard deviation). DMUs that are not printed in bold are DMUs that are DEA-inefficient (See Table 15), and hence, they are ranked below those which have been deemed empirically efficient. DEA efficient DMUs that fail to satisfy the conditions for will be given efficiency scores of at most 1.00.

##### 5.5. Further Analysis

Additional analyses were conducted by taking the observed DMUs in each simulation and evaluating them against the EFF, DEA, CCP, and DEA-Chebyshev model results. If DCF is a good approximation of the EFF, then the efficiency scores for the observed DMUs should not be substantially different from the efficiency scores generated by the EFF. This also holds true for CCP.

###### 5.5.1. Observed DMUs Evaluated against the EFF, CCP, and DCF

The efficiency scores of the observed DMUs from the experimental groups determined by the EFF (to be denoted as “exp.grp+EFF”) will provide a benchmark for evaluating the DEA frontier (“exp.grp+DEA”), CCP (normal) frontier (“exp.grp+CCP”), and the corrected frontier (“exp.grp+DCF”). A comparison is drawn between the efficiency scores of the experimental groups generated by the four frontiers.

The hypothesis is that the mean of the efficiency scores for the 15 observed units in the “exp.grp+EFF” group and the “exp.grp+DCF” group should be approximately the same (i.e., the difference is not statistically significant). From Table 16, the null hypothesis can be seen from the rank-sum test and the *t*-test at , and the difference is not statistically significant in simulations 3 and 4, hence, the corrected frontier is a good approximation of the EFF. Although the hypothesis test for simulations 1 and 2 indicates some level of significance, the results generated by the DCF model are still superior to those of the CCP and the DEA.

Table 16 shows the statistical tests used to compare the DEA, CCP, and DCF against the EFF. The *Pearson Correlation* analysis, (regression line) which ranges from −1 to 1 inclusively, reflects the extent of the linear relationship between two sets of data. The values, the rank-sum test, and the Pearson correlation observed for all the four simulations indicate that in general the DCF outperforms DEA and CCP (which assumed the normal distribution).

Outliers have a tendency to exhibit large standard deviations, which is translated to large confidence limits. Consequently, the reason for establishing DCF and CCP scores is to reduce the likelihood of a virtual unit from becoming an outlier. Also, the results generated by the stochastic models (as opposed to deterministic ones) such as the DCF and CCP can be greatly affected because the efficiency scores are generally not restricted to 1.00. In reality, outliers are not always easily detected. If the data set contains some outliers, the stochastic models may not perform well. DMU14 in Simulation 3 is an example of this problem. It can be solved by either removing the outliers or by imposing weight restrictions. However, weight restrictions are not within the scope of this paper.

#### 6. Conclusions

Traditional methods of performance analysis are no longer sufficient in a fast paced constantly evolving environment. Observing past data alone is not adequate for future projections. The DEA-Chebyshev model is designed to bridge the difference between conventional performance measurements and new techniques to incorporate relevance into such measures. This algorithm not only provides a multidimensional evaluation technique, but it has successfully incorporated a new element into an existing deterministic technique (DEA). This is known as the *k-flexibility function* which was originally derived from the one-sided Chebyshev's inequality. This in turn allows management to include expert opinion as a single value, such as a 20% net growth by next year end from the current year. The single value is dichotomized into unmet (or over target) present level of growths (or declines). Because management expertise is included, the expected growth (or decline) is not unreasonable and will inherently include factors which do not need to be explicitly expressed in the model such as environmental, economic, and social changes. Since these changes are becoming increasingly rapid, performance measures can no longer ignore qualitative inputs. In a highly competitive environment, future projections and attainable targets are key performance indicators. Intellectual capital and knowledge are today’s two most important assets.

The combination of normal DEA with DCF can successfully provide a good framework for evaluation based on quantitative data and qualitative intellectual knowledge of management. When no errors are expected, then standard DEA models will suffice. DCF is designed such that in the absence of errors, the model will revert to a DEA model. This occurs when the *k-flexibility function* equals zero. DEA provides a deterministic frontier which DEA-Chebyshev model works on to define the estimate of the EFF.

The simulated dataset was tested on DEA-Chebyshev model. It has been statistically proven that this model is an effective tool with excellent accuracy to detect or predict the EFF frontier as a new efficiency benchmarking technique. It is an improvement over other methods, easily applied, practical, not computationally intensive, and easy to implement. The results have been promising thus far. The future work includes using a real data application to illustrate the usefulness of DEA-Chebyshev model.

#### Appendices

#### A.

Note that semi-positive is defined to be the nonpositive characteristics of all data where at least one component in every input and output sector is positive; mathematically, , * *and , . That is to say that for every DMU, there must be at least one positive value in both input and output. The following properties were noted from Cooper et al. [26, 28].Let be the production possibility set (PPS of physically attainable points :
Each pair of input and output is regarded as semi-positive orthant point in dimensional space in Inefficiency: For any semi-positive PPS where and/or , it is also true that they belong to the set of attainable points .Convexity: if , and , such that , then Ray unboundedness: if a PPS , then for any scalar (refer to CRS).Any semi-positive linear combination of PPS in also belongs to .

Therefore, satisfying –, represents a vector of inputs and a vector of outputs for one DMU; represents a matrix of inputs and outputs, respectively, for all DMUs.

Definition of is as follows:

#### B.

As theoreticians have shown and used in the past the characteristic of errors in efficiency analysis, the simulations generated will incorporate those elements. These characteristics are (1)statistical noise: , i.i.d., unrestricted;(2)inefficiency: , i.i.d. half normal distribution where .Data variability is caused by statistical noise, measurement errors, and inefficiency. There errors can arise from either exogenous or endogenous variables such as poor management, economic growth, and environmental and sociological contributions.

#### C.

The corrected frontier is defined such that the production possibility space will always be greater than that of the DEA spaces (see ). That is to say that there will always be room for improvement in efficiency which companies are always aspiring for. The following are the properties of the corrected frontier. if and/or , then DCF will be different from DEA. If and , then the corrected frontier is also the DEA frontier. DMUs on the DEA frontier are a subset of those on the DCF frontier; some DEA efficient units will appear inefficient in DCF, in which case, the frontier is shifted away from the PPS (i.e., expansion of the PPS)., where the radial contraction of inputs (or the radial expansion of outputs) can be improved for DEA efficient units.

Although we do not have formal proof of the convergence of the corrected frontier to the EFF, due to the convergence of the DEA estimator to the EFF, → when , shown in [24], we can conjecture about the convergence of the DCF that it also converges to the EFF as sample sizes increase.

#### References

- D. Aigner, C. A. K. Lovell, and P. Schmidt, “Formulation and estimation of stochastic frontier production function models,”
*Journal of Econometrics*, vol. 6, no. 1, pp. 21–37, 1977. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - W. Meussen and J. Van Den Broeck, “Efficiency estimation from cobb-douglas production functions with composed error,”
*International Economic Review*, vol. 18, pp. 435–444, 1977. - A. Kneip and L. Simar, “A general framework for frontier estimation with panel data,”
*Journal of Productivity Analysis*, vol. 7, no. 2-3, pp. 187–212, 1996. View at Scopus - J. K. Sengupta, “Data envelopment analysis for efficiency measurement in the stochastic case,”
*Computers and Operations Research*, vol. 14, no. 2, pp. 117–129, 1987. View at Scopus - A. Charnes and W. W. Cooper, “Deterministic equivalents for optimizing and satisficing under chance constraints,”
*Operations Research*, vol. 11, pp. 18–39, 1963. View at Publisher · View at Google Scholar · View at MathSciNet - W. W. Cooper, Z. Huang, and S. X. Li, “Satisficing DEA models under chance constraints,”
*Annals of Operations Research*, vol. 66, pp. 279–295, 1996. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - P. Kall,
*Stochastic Linear Programming*, Springer, 1976. - O. B. Olesen and N. C. Petersen, “Chance Constrained Efficiency Evaluation,”
*Management Science*, vol. 41, no. 3, pp. 442–457, 1995. - O. B. Olesen and N. C. Petersen, “Chance constrained efficiency evaluation,”
*Management Science*, vol. 41, no. 3, pp. 442–457, 1995. - A. Charnes, W. W. Cooper, and E. Rhodes, “Measuring the efficiency of decision making units,”
*European Journal of Operational Research*, vol. 2, no. 6, pp. 429–444, 1978. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - Land, C. Kenneth, Lovell, C. A. Knox, and S. Thore, “Chance-constrained data envelopment analysis,”
*Managerial and Decision Economics*, vol. 14, pp. 541–554, 1993. - W. W. Cooper, L. M. Seiford , and K. Tone,
*Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software*, Springer, New York, NY, USA, 2nd edition, 2006. - R. G. Thompson, F. D. Singleton Jr., R. H. Thrall, and B. A. Smith, “Comparative site evaluations for locating a high-energy physics lab in Texas,”
*Interfaces*, vol. 16, pp. 35–49, 1986. - R. G. Thompson, L. N. Langemeier, C. T. Lee, E. Lee, and R. M. Thrall, “The role of multiplier bounds in efficiency analysis with application to Kansas farming,”
*Journal of Econometrics*, vol. 46, no. 1-2, pp. 93–108, 1990. View at Scopus - A. Charnes, W. W. Cooper, Q. L. Wei, and Z. M. Huang, “Cone ratio data envelopment analysis and multi-objective programming,”
*International Journal of Systems Science*, vol. 20, no. 7, pp. 1099–1118, 1989. View at Publisher · View at Google Scholar · View at MathSciNet - G. A. Tomlinson,
*Confounder Measurement Error and Exposure Odds Ratio Estimation [M.S. thesis]*, Department of Community Health, University of Toronto, 1993. - C. Ritter and L. Simar, “Pitfalls of normal-gamma stochastic frontier models,”
*Journal of Productivity Analysis*, vol. 8, no. 2, pp. 167–182, 1997. View at Scopus - J. Forrester and T. R. Anderson, “A new technique for estimating confidence intervals on DEA efficiency estimates,” in
*Proceedings of the International Conference on Technology and Innovation Management (PICMET '99)*, Seattle, Wash, USA, 1999. - R. V. Hogg and A. T. Craig,
*Introduction to Mathematical Statistics*, The Macmillan, New York, NY, USA, 3rd edition, 1970. View at MathSciNet - J. R. Birge and F. Louveaux,
*Introduction to Stochastic Programming*, Springer, New York, NY, USA, 1997. View at MathSciNet - F. M. Allen, R. N. Braswell, and P. V. Rao, “Distribution-free approximations for chance constraints,”
*Operations Research*, vol. 22, no. 3, pp. 610–621, 1974. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - I. M. Cockburn, R. M. Henderson, and S. Stern, “Untangling the origins of competitive advantage,”
*Strategic Management Journal*, vol. 21, no. 10-11, pp. 1123–1145, 2000. View at Scopus - L. Simar and P. W. Wilson, “Statistical inference in nonparametric frontier models: the state of the art,”
*Journal of Productivity Analysis*, vol. 13, no. 1, pp. 49–78, 2000. View at Scopus - A. Kneip, B. U. Park, and L. Simar, “A note on the convergence of nonparametric DEA estimators for production efficiency scores,”
*Econometric Theory*, vol. 14, no. 6, pp. 783–793, 1998. View at Publisher · View at Google Scholar · View at MathSciNet - L. Simar and P. W. Wilson, “Sensitivity analysis of efficiency scores: how to bootstrap in nonparametric frontier models,”
*Management Science*, vol. 44, no. 1, pp. 49–61, 1998. View at Scopus - W. W. Cooper, Z. Huang, V. Lelas, S. X. Li, and O. B. Olesen, “Chance constrained programming formulations for stochastic characterizations of efficiency and dominance in DEA,”
*Journal of Productivity Analysis*, vol. 9, no. 1, pp. 53–79, 1998. View at Scopus - A. Charnes, W. W. Cooper, A. Y. Lewin, and L. M. Seiford,
*Data Envelopment Analysis, Theory Methodology and Applications*, Kluwer Academic Publishers, 1994. - R. D. Banker, A. Charnes, and W. W. Cooper, “Some models for estimating technical and scale efficiency in data envelopment analysis,”
*Management Science*, vol. 30, no. 9, pp. 1078–1092, 1984. View at Scopus