- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Advances in Operations Research
Volume 2011 (2011), Article ID 259439, 18 pages
Using DEA Factor Efficiency Scores to Eliminate Subjectivity in Goal Programming
Department of Technology and Society, College of Engineering and Applied Sciences, Stony Brook University, Stony Brook, NY 11794-3760, USA
Received 30 April 2011; Accepted 26 July 2011
Academic Editor: Carlos Romero
Copyright © 2011 Herbert F. Lewis. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Many real-world problems require decision makers to consider multiple criteria when performing an analysis. One popular method used to analyze multicriteria decision problems is goal programming. When applying goal programming, it is often difficult if not impossible to determine the target values and unit penalty weights with any level of confidence. Thus, in many situations, managers and decision makers may be forced to specify these parameters subjectively. In this paper, we present a model framework designed to eliminate the arbitrary assignment of target values and unit penalty weights when applying goal programming to solve multicriteria decision problems. In particular, when neither of these parameters is available, we show how to integrate factor efficiency scores determined from data envelopment analysis into the model. We discuss an application of the methodology to ambulatory surgery centers and demonstrate the model framework via a product mix example.
Many real-world problems require decision makers to consider multiple (often conflicting) criteria when performing an analysis. For example,(i)financial managers consider both return on investment and risk when building a portfolio;(ii)advertising executives balance budget and audience reached when developing an advertising campaign;(iii)retail managers are concerned with providing acceptable customer service while ensuring that sales associates are not underutilized.
One popular method used to analyze multicriteria decision problems is goal programming [1, 2]. When applying goal programming, the analyst must specify a target value for each criterion (also referred to as a goal) and an associated unit penalty weight incurred when the target value cannot be achieved. It is often difficult if not impossible to determine the target values [3–6] and unit penalty weights [7–10] with any level of confidence. For instance, these issues arise in group decision making situations when members of the group may disagree on the relative importance of the criteria and the desired level of achievement for each of the goals. Thus, in many situations, managers and decision makers may be forced to specify these parameters subjectively.
Some work has been done to resolve these issues. Yu and Zeheny  and Steuer [12, 13] propose performing a multiparametric sensitivity analysis to reduce the arbitrariness in determining target values. A second approach involves setting infeasible high target values  which can lead to a nondominated solution to the goal program. Yet another approach to setting target values adapts the maximum likelihood criterion of Sengupta et al.  and Werczberger . A similar technique was proposed by Rakes and Franz . Narasimhan  suggests using “fuzzy” goals to help resolve the problem of assigning inappropriate target values. In addition, Dyer and Sarin  use multiattribute value functions to differentiate between deterministic alternatives, thus avoiding the need to specify target values. Kirkwood and Sarin  show how to rank multiattribute alternatives when partial information about the attribute weights is known. Gass  presents a process for determining unit penalty weights based on Saaty’s analytic hierarchy process . Finally, Butler et al.  provide a simulation-based method for performing sensitivity analysis on the weights assigned to attributes in multicriteria decision models.
In this paper, we present a model framework designed to eliminate the arbitrary assignment of target values and unit penalty weights when applying goal programming to solve multicriteria decision problems. If the analyst can specify both target values and unit penalty weights, we recommend the standard goal programming model. In the case where the analyst cannot specify the target values but can determine unit penalty weights, we present a model that minimizes the sum of the penalties incurred for all goals. When the analyst cannot specify the unit penalty weights but can determine the target values, we use a model that calculates the ratio of the achieved value to the target value for each goal and maximizes the minimum of these ratios.
Finally, when neither target values nor unit penalty weights are available, we present a two-step process that integrates factor efficiency scores  or factor inverse efficiency scores determined from data envelopment analysis (DEA)  into the model. A factor efficiency (inverse efficiency) score specifies the percentage to which a particular input (output) in the DEA may be decreased (increased) in order to make the decision-making unit (DMU) under consideration efficient. First, we solve a DEA model in which the DMUs represent the decision variables in the goal program. The inputs to each DMU are the “resources” required to yield one unit of the related decision variable, and the outputs are the unit contributions to each of the goals. Next, we solve a model that incorporates the factor efficiency scores obtained from the DEA into the goal program. In particular, we use the factor efficiency scores directly as coefficients of the decision variables in a set of constraints designed to produce solutions that treat the goals in a “balanced” manner. To our knowledge, this two-step process is new and is the major contribution of this paper. The motivation for developing the methodology is to guide an ambulatory surgery center in identifying its optimal procedure mix . We discuss this application in detail in a later section.
Links between multicriteria decision making and data envelopment analysis are present in the literature. Athanassopoulos  develops an interface between goal programming and DEA in multilevel planning. This method is used in the allocation of central grants to local authorities in Greece. Stewart  compares the concepts of efficiency in DEA with Pareto optimality in multiple criteria decision making. Athanassopoulos and Podinovski  use DEA-like models to evaluate dominance and potential optimality of decision alternatives in multiple criteria decision analysis with imprecise information. Sarrico et al.  use DEA as a performance measurement tool (within a decision support system context) for university selection in the U.K. Arakawa et al.  combine DEA and genetic algorithms in multiobjective optimization. Joro et al.  show that the DEA formulation is structurally similar to the multiple objective linear programming model. Post and Spronk  combine DEA and interactive multiple goal programming into a procedure for performance benchmarking. Takeda and Satoh  use DEA to “rank” alternatives in discrete multicriteria decision problems.
In the next section, we present the model framework. Next, we discuss the ambulatory surgery center application. Then, we demonstrate the model framework via a simple product mix example. Finally, we present our results and conclusions. We also include an appendix describing the breadth of problems to which the model framework may be applied.
2. The Model Framework
In this section, we present several mathematical programming models that reflect the availability of subjective values such as target values and unit penalty weights. Table 1 presents a framework for determining the appropriate model.
2.1. Model 1: Known Target Values and Known Unit Penalty Weights
When it is possible for the decision maker to specify target values and unit penalty weights for the goals, we can apply a standard goal programming model. The model is as follows.
where are the decision variables, and are the deviation variables for the goals, and are the unit penalty weights associated with the deviation variables, are the unit contributions of the decision variables to the goals, and are the target values for the goals. The system constraints are the typical “resource” constraints found commonly in mathematical programming problems, and the variable assumptions specify the type of variable (integer or continuous) and whether the variable is nonnegative. We note that this model assumes that all goals are at the same priority level. If this is not the case, the summation in the objective function will be split among the various priority levels. In addition, we note that we may have or for some since unit penalty weights may not be applicable to deviations both above and below the target value.
2.2. Model 2: Unknown Target Values but Known Unit Penalty Weights
When unit penalties are known but it is difficult or impossible for the decision maker to specify target values, we can solve the following formulation that minimizes the sum of the penalties incurred for all goals.
where are the unit penalty weights associated with the goals. In certain situations, are unit rewards rather than unit penalties. In such cases, the objective function becomes . If there are both unit rewards and unit penalties, we can maximize the total sum of the rewards minus the total sum of the penalties.
2.3. Model 3: Known Target Values but Unknown Unit Penalty Weights
When target values are known but it is difficult or impossible for the decision maker to specify unit penalty weights, we can solve the following formulation that calculates the ratio of the achieved value to the target value for each goal and maximizes the minimum of these ratios for goals with lower targets or equivalently minimizes the maximum of these ratios for goals with upper targets.
where is a variable representing the smallest ratio of the achieved level of each goal to its lower target and is a variable representing the largest ratio of the achieved level of each goal to its upper target. Not all goals will have both lower and upper targets. If the target value for goal is a lower bound include and if it is an upper bound include . We note that this model is nonlinear. When all target values are lower bounds, we can remove from the formulation, making it linear. Similarly, when all target values are upper bounds, we can remove from the formulation, making it linear.
2.4. Model 4: Unknown Target Values and Unknown Unit Penalty Weights
When it is difficult or impossible for the decision maker to specify target values and unit penalty weights, we can incorporate factor efficiency scores determined by solving a DEA model into the goal programming model.
Step 1 (The DEA Model). Decision makers seek solutions that satisfy their goals. In order to achieve the desired levels of these goals, it is often important to allocate limited resources efficiently. This is the fundamental idea behind incorporating DEA factor efficiency scores into goal programming models. A few examples follow.(i)In a product mix decision, products that require relatively low amounts of raw materials and labor but generate relatively high revenues and relatively low amounts of toxic waste are “efficient.” (ii)In a media selection decision, media that have relatively low advertising cost yet reach relatively large audiences in each of several desired demographic groups are “efficient.”The following two models represent an input-oriented and an output-oriented DEA model for DMU , respectively.
Input-Oriented DEA Model:
Output-Oriented DEA Model:
where is the weight placed on DMU by DMU , is the efficiency of DMU , is the inverse efficiency of DMU , is the level of input consumed by DMU , and is the level of output produced by DMU .
Each DMU in the DEA model represents a decision variable in the goal program. The inputs to each DMU are the “resources” required to yield one unit of the related decision variable, and the outputs are the unit contributions to each of the goals. Thus, in the DEA models presented above, is the level of “resource” required to yield one unit of decision variable and is the unit contribution of decision variable , to goal . The parameters appear as coefficients in the system constraints of the goal programming model.
In the product mix decision, the inputs to the DEA model are the raw material and labor requirements of the product and the outputs are the associated revenue and toxic waste generated. Similarly, in the media selection decision, the input to the DEA model is the unit advertising cost of the medium and the outputs are the associated audiences reached in each demographic group.
Step 2 (The Multicriteria Model). We incorporate the factor efficiency scores obtained from the DEA into the goal programming model. In particular, the factor efficiency scores appear as coefficients of the decision variables in a set of constraints designed to produce solutions that treat the goals in a “balanced” manner as follows.
where is a variable representing the smallest total factor efficiency achieved over all the goals, and is the factor efficiency for goal at DMU . If we use factor inverse efficiency scores instead of factor efficiency scores, the objective function becomes , and the constraint becomes , where represents the largest total factor inverse efficiency achieved over all the goals and is the factor inverse efficiency for goal at DMU .
3. Application to Ambulatory Surgery Centers
In Lewis et al. , the authors state, “Ambulatory surgery centers (ASCs) provide a low-cost alternative to traditional inpatient care. In addition, with health care reform imminent, it is likely that many currently uninsured people will soon acquire health care coverage, significantly increasing the demand for health services. ASCs are among the providers that can expect to see a substantial amount of this new pent-up demand and, therefore, ASCs are likely to continue their current growth into the foreseeable future. Those ASCs that plan accordingly by optimizing procedure mix and volume will benefit most from the increased demand.”
They apply the two-step process (Model 4) to guide an ASC in identifying its optimal procedure mix. The criteria are to (1) maximize reimbursement while (2) minimizing the total number of complications. The authors state, “We apply the two-step process, rather than a standard goal programming model, since we believe it will be extremely difficult, if not impossible, for ASC managers to specify the target values and unit penalty weights for the criteria (reimbursement and the total number of complications).”
The first step applies DEA to calculate the efficiency of each procedure based on the resources required to perform the procedure and the criteria upon which the procedure is evaluated. Specifically, the resources (DMU inputs) are (i)the average minutes for the procedure,(ii)the average nursing minutes for the procedure,(iii)the average anesthesiologist minutes for the procedure,(iv)the average technician minutes for the procedure, and(v)the average value of supplies and materials (in dollars) used for the procedure.
The criteria (DMU outputs) are (i)the average reimbursement received for the procedure and (ii)the complication rate for the procedure.
The authors further state, “The key idea in the process is that the ASC will want to perform a procedure more often if it has high factor efficiencies and less often if it has low factor efficiencies. We embody this notion in the second step by incorporating the output factor efficiencies into a bottleneck program that optimizes the mix of procedures while satisfying the ASC’s resource and operational constraints.” Specifically, the operational constraints ensure that (i)resource capacities are not exceeded, (ii)procedure demands are not exceeded,(iii)the total number of procedures performed does not decrease,(iv)the total reimbursement does not decrease, and(v)the total number of complications does not increase.
Using the methodology, the authors develop a series of scenarios to facilitate what-if analysis and demonstrate how the methodology can lead to higher reimbursement and lower complications. In the conclusion, the authors state, “The model suggested in this paper generates the most effective mixture of procedures taking into account current resources and performance. The optimal procedure mix suggests a strategic direction for administrators to evaluate and implement. Not having a strategic methodology leaves decision makers to make critical changes based on prior experiences and retrospective data without the benefit of a logical and systematic framework.”
4. Product Mix Example
In this section, we present our decision-making framework by applying all four models to a simple product mix example. We stress that the following is strictly an example and that the data are for demonstration purposes only. Consider a firm that produces eight different products. Each product requires a certain amount of time on a cutting machine, a certain amount of time on a finishing machine and a certain amount of labor to operate the machines. For each product, the unit profit and the waste per unit produced by the cutting process are known. This information is shown in Table 2 along with the total amount of time available on each machine and the total labor available. The firm also limits the production of each product to no more than 20 units.
Management has specified two criteria it wishes to consider when determining the number of units of each product to produce. Both are at the same priority level. The first criterion involves the total profit the firm will gain from production. The second involves the total waste produced by the cutting process.
4.1. Product Mix Model 1
For this model, we must specify a target value for each criterion and a unit penalty weight for each criterion to be charged when the target value is not achieved. As a base-case scenario, we choose a target value for total profit of $1000, a target value for total waste from the cutting process of 200 units, and a unit penalty weight for each criterion of 0.5. Thus, we solve the following model.
Product Mix Model 1:
where , and represent the quantity of each product produced.
4.2. Product Mix Model 2
For this model, we cannot specify target values but we can specify a unit penalty (reward) weight for each criterion. From our base-case scenario, we select a unit reward of 0.5 for the profit criterion and a unit penalty weight of 0.5 for the waste criterion. Thus, we solve the following model.
Product Mix Model 2:
4.3. Product Mix Model 3
For this model, we are unable to specify unit penalty weights but we can specify a target value for each criterion. From our base-case scenario, we choose a target value for total profit of $1000 and a target value for total waste from the cutting process of 200 units. Thus, we solve the following model.
Product Mix Model 3:
4.4. Product Mix Model 4
When we cannot specify either target values or unit penalty weights, we first solve a DEA model for each product using the products as the DMUs. As shown in Figure 1, the inputs to a particular DMU are the unit cutting, unit finishing, and unit labor requirements, and the outputs are the unit profit and unit waste produced. The DEA model for product is as follows.
DEA Model for Product in the Product Mix Example:
where and represent the efficiency and inverse efficiency scores for product , respectively, and are weights placed on the DMUs (products) by the DMU (product) under consideration. We note that larger values of the output waste make the production process less efficient. We call such a quantity a reverse output and incorporate it into our model using the methodology in Lewis and Sexton . We note the nonlinear constraint, which is necessary to ensure that the inverse efficiency score is indeed the multiplicative inverse of the efficiency score.
We present the results of the DEA in Table 3. We observe that products , , and are efficient, indicating that these products require relatively little input to produce relatively high output.
Next, we use the factor efficiencies obtained from the DEA to build Model 4. In particular, we use the factor efficiencies for each product as coefficients in the constraints for the criteria. The model is as follows.
Product Mix Model 4:
4.5. Product Mix Example Results
Table 4 shows the resources used, the levels of each product produced, the total profit achieved, and total waste incurred when solving each model in our base-case scenario. In all the models, products , , and are produced to capacity or nearcapacity (in the case of Model 3 and product ). These products use relatively low levels of input to produce relatively high levels of output as indicated by the efficiency scores of 1 from the DEA. In addition, all models produce some of product , which also has a relatively high efficiency score. All models produce 0 units of the more “inefficient” products , , , and . The profits and total wastes from the cutting process for all models are similar, ranging between $796 and $801, and between 240 and 250, respectively.
In Table 5, we present the results of the standard goal programming model when varying the unit penalty weights from those in the base-case scenario. We generate ten new scenarios by letting the unit penalty weight for each goal vary from 0 to 1 in increments of 0.1 so that the sum of the two unit penalty weights equals 1. The target values remain as in the base-case scenario at $1000 for the profit and 200 units for the total waste from the cutting process. It is clear that these solutions are dependent on the chosen unit penalty weights. In particular, the profit ranges from $220 to $810, and the total waste from the cutting process ranges from 160 to 300.
A similar situation occurs when we vary the target values. Table 6 presents the results of the standard goal programming model when varying the target values from those in the base-case scenario. We build eight new scenarios. First, we fix the total waste target value at 200 (as in the base-case) and build four scenarios having profit target values of $0, $250, $500, and $750. Then, we fix the profit target value at $1000 (as in the base-case) and build four scenarios having total waste target values of 225, 250, 275, and 300. We leave the unit penalty weights at 0.5 for both goals as in the base-case scenario. It is clear that these solutions are dependent on the chosen target values. In particular, the profit ranges from $40 to $810, and the total waste from the cutting process ranges from 120 to 300.
5. Discussion and Conclusion
In this paper, we present a model framework designed to eliminate the arbitrary assignment of target values and unit penalty weights when applying goal programming to solve multicriteria decision problems. Although other authors have considered this issue, each has either addressed the problem of subjective target values or subjective unit penalty weights, but not both. Our model framework also applies to situations in which the decision maker cannot specify target values or unit penalty weights. In addition, we show how to incorporate DEA factor efficiency scores into the model when the decision maker cannot specify both.
The model framework is applicable to a wide variety of problems encountered in marketing, finance, production management, logistics, and transportation management. We discuss the motivation for developing the methodology, namely, to guide an ambulatory surgery center in identifying its optimal procedure mix. To demonstrate the model framework, we consider a simple product mix example. From this example, it is clear that the selection of target values and unit penalty weights affects the solution to the goal program. Thus, when the decision maker cannot specify these parameters with any degree of certainty, arbitrarily making a selection can greatly influence the decision. However, using the methodology presented in this paper eliminates the need to select these parameters arbitrarily, and as shown, the solutions from applying Models 2, 3, and 4 to the product mix example appear reasonable.
A. Breadth of Applications
The model framework can be applied to a wide range of problems. We describe applications from marketing, finance, production management, logistics, and transportation. The applications we present in this section are exemplary and by no means comprehensive. For each application, we show how to build the DMU for the DEA model when applying Model 4. There may be other inputs and outputs that should be included when building the DMU depending on the specific problem. A limitation on the model framework is also discussed and an application where this limitation arises is presented.
A.1. Media Selection Problems
Media selection decisions involve determining the number of advertisements to be placed in each of several types of media (newspaper, television, radio, etc.) in order to reach desired levels of audiences in several demographic groups. An advertising budget limits the number of advertisements and thus the total audience that may be reached. When applying Model 4, each DMU in the DEA model represents an advertising medium. The input to each DMU is the unit advertising cost of the medium, and the outputs are the associated audiences reached in each demographic group.
A.2. Marketing Research Problems
In marketing research decisions, the analyst must first specify the demographic groups to survey and the types of surveys (phone, on-line, etc.) to conduct. Then, the analyst must determine the number of individuals to survey using each survey type from each demographic group. The unit cost of conducting a survey and the accuracy of the survey results are dependent on the survey type and the demographic group involved. When applying Model 4, each DMU in the DEA model represents a combination of a survey type and a demographic group. The input to each DMU is represented by a scalar of unit value, and the outputs are the associated unit cost and accuracy of the survey.
A.3. Portfolio Selection Problems
Portfolio selection decisions involve selecting which investment opportunities to invest in and determining how much to invest in each opportunity selected. Typically, investors consider both return and risk when making their decisions. When applying Model 4, each DMU in the DEA model represents an investment opportunity. The input to each DMU is the cost of the investment and the outputs are the associated return and risk.
A.4. Product Mix Problems
Product mix decisions involve determining the amount of each product a firm should produce. The products typically compete for the same limited resources. Objectives involve achieving desired levels of total profit and limiting total waste produced. When applying Model 4, each DMU in the DEA model represents a product. The inputs to each DMU are the raw material and labor requirements of each product, and the outputs are the associated profit and waste produced.
A.5. Make-or-Buy Problems
In make-or-buy decisions, a firm must decide the amount of a given product to produce in-house and the amount to purchase from a contractor. Such problems arise when the firm has limited production capacity available. The firm considers the costs associated with manufacturing and purchasing a product and the reliability of products produced in-house and by the contractor. When applying Model 4, each DMU in the DEA model represents whether a product is made by the firm or purchased from the contractor. The input to each DMU is represented by a scalar of unit value, and the outputs are the unit costs associated with the manufacture or purchase of the product and the reliability of in-house or contractor production.
A.6. Location Planning Problems
Location planning decisions involve determining where to locate one or more facilities. A particular facility may be evaluated on operating cost related to the location and distance to suppliers and customers. When applying Model 4, each DMU in the DEA model represents a potential facility location. The input to each DMU is represented by a scalar of unit value, and the outputs are the associated operating costs at the location and the distance to suppliers and customers.
A.7. Knapsack Problems
In knapsack problems, we are given a knapsack of known size and a set of items. Each item has a specific size and is evaluated across several criteria. We seek to select a subset of the items that fit in the knapsack and that together achieve desired levels of the various criteria. When applying Model 4, each DMU in the DEA model represents an item. The input to each DMU is the size of the item, and the outputs are the contributions of the item to the criteria.
A.8. Assignment Problems
In assignment problems, a set of agents must be assigned in a one-to-one fashion to a set of tasks. Each assignment of an agent to a task is evaluated according to the time it will take the agent to complete the task and the profit gained when the agent completes the task. We seek a pairing of agents to tasks that achieves desired levels of time and profit. When applying Model 4, each DMU in the DEA model represents the pairing of an agent to a task. The input to each DMU is represented by a scalar of unit value, and the outputs are the associated time and profit.
A.9. Transportation Problems
In transportation problems, units of a product must be shipped from the plants where they are produced to customers where they are sold. Each plant has a production capacity, and each customer has a demand. Each unit shipped from a specific plant to a specific customer incurs a shipping cost. In addition, the time it takes to ship from a plant to a customer may be important especially if the products are perishable. We seek a shipment plan specifying the number of units of the product to ship from each plant to each customer that considers both total shipment cost and average shipment time. When applying Model 4, each DMU in the DEA model represents the shipment from a plant to a customer. The input to each DMU is represented by a scalar of unit value, and the outputs are the associated unit shipping cost and transportation time.
A.10. A Limitation on the Model Framework
We have assumed that the decision variables represent “complete” entities that are directly related to the criteria. Examples are the individual media, investment opportunities, and products in the applications just presented. How do we build the DMU for the DEA model in Model 4 when this is not the case? For example, in the traveling salesman problem, the decision variables typically represent edges that make up the tour. If the specified criteria are measured relative to the tour produced (such as its total length or the number of vehicles it requires), we are unable to construct an appropriate DMU. We leave this as an area for future research.
The author wishes to thank the faculty of the College of Business at Stony Brook University for their thoughtful suggestions. Their input greatly enhanced the quality of this paper.
- A. Charnes, W. W. Cooper, and R. Ferguson, “Optimal estimation of executive compensation by linear programming,” Management Science, vol. 1, pp. 138–151, 1955.
- A. Charnes and W. W. Cooper, Management Models and Industrial Applications of Linear Programming, John Wiley & Sons, New York, NY, USA, 1961.
- J. L. Cohon and D. H. Marks, “A review and evaluation of multi-objective programming techniques,” Water Resources Research, vol. 11, no. 2, pp. 208–220, 1975.
- J. L. Cohon, Multiobjective Programming and Planning, Academic Press, New York, NY, USA, 1978.
- M. Zeleny, “The pros and cons of goal programming,” Computers and Operations Research, vol. 8, no. 4, pp. 357–359, 1981.
- H. Min and J. Storbeck, “On the origin and persistence of misconceptions in goal programming,” Journal of the Operational Research Society, vol. 42, no. 4, pp. 301–312, 1991.
- W. B. Widhelm, “Extensions of goal programming models,” Omega, vol. 9, no. 2, pp. 212–214, 1981.
- C. Romero, “Multi-objective and goal programming approaches as a distance function model,” Journal of the Operational Research Society, vol. 36, no. 3, pp. 249–251, 1985.
- C. Romero, C. Sutcliffe, J. Board, and P. Cheshire, “Naïve weighting in non-preemptive goal programming (Viewpoint and reply),” Journal of the Operational Research Society, vol. 35, pp. 647–649, 1985.
- S. I. Gass, “A process for determining priorities and weights for large-scale linear goal programmes,” Journal of the Operational Research Society, vol. 37, no. 8, pp. 779–785, 1986.
- P. L. Yu and M. Zeleny, “Linear multiparametric programming by multicriteria simplex method,” Management Science, vol. 23, no. 2, pp. 159–170, 1976.
- R. E. Steuer, “Multiple objective linear programming with internal criterion weights,” Management Science, vol. 23, no. 3, pp. 305–316, 1976.
- R. E. Steuer, “Goal programming sensitivity analysis using interval penalty weights,” Mathematical Programming, vol. 17, no. 1, pp. 16–31, 1979.
- S. S. Sengupta, M. L. Podrebarac, and T. D. H. Fernando, “Probabilities of optima in multi-objective linear programs,” in Multiple Criteria Decision Making, J. L. Cochran and M. Zeleny, Eds., pp. 217–235, University of South Carolina Press, Columbia, SC, USA, 1973.
- E. Werczberger, “Multi-objective linear programming with partial ranking of objectives,” Socio-Economic Planning Sciences, vol. 15, no. 6, pp. 331–339, 1981.
- T. R. Rakes and L. S. Franz, “Decision support models for project planning,” Omega, vol. 13, no. 1, pp. 73–74, 1985.
- R. Narasimhan, “Goal programming in a fuzzy environment,” Decision Sciences, vol. 11, pp. 325–336, 1980.
- J. S. Dyer and R. K. Sarin, “Measurable multiattribute value functions,” Operations Research, vol. 27, no. 4, pp. 810–822, 1979.
- C. W. Kirkwood and R. K. Sarin, “Ranking with partial information: a method and an application,” Operations Research, vol. 33, no. 1, pp. 38–48, 1985.
- T. L. Saaty, The Analytic Hierarchy Process, McGraw-Hill, New York, NY, USA, 1981.
- J. Butler, J. Jia, and J. Dyer, “Simulation techniques for the sensitivity analysis of multi-criteria decision models,” European Journal of Operational Research, vol. 103, no. 3, pp. 531–546, 1997.
- R. J. Kopp, “The measurement of productive efficiency: a reconsideration,” The Quarterly Journal of Economics, vol. 96, no. 3, pp. 477–503, 1981.
- A. Charnes, W. W. Cooper, and E. Rhodes, “Measuring the efficiency of decision making units,” European Journal of Operational Research, vol. 2, no. 6, pp. 429–444, 1978.
- H. F. Lewis, T. R. Sexton, and M. A. Dolan, “An efficiency-based multicriteria strategic planning model for ambulatory surgery centers,” Journal of Medical Systems. In press.
- A. D. Athanassopoulos, “Goal programming & data envelopment analysis (GoDEA) for target-based multi-level planning: allocating central grants to the Greek local authorities,” European Journal of Operational Research, vol. 87, no. 3, pp. 535–550, 1995.
- T. J. Stewart, “Relationships between data envelopment analysis and multicriteria decision analysis,” Journal of the Operational Research Society, vol. 47, no. 5, pp. 654–665, 1996.
- A. D. Athanassopoulos and V. V. Podinovski, “Dominance and potential optimality in multiple criteria decision analysis with imprecise information,” Journal of the Operational Research Society, vol. 48, no. 2, pp. 142–150, 1997.
- C. S. Sarrico, S. M. Hogan, R. G. Dyson, and A. D. Athanassopoulos, “Data envelopment analysis and university selection,” Journal of the Operational Research Society, vol. 48, no. 12, pp. 1163–1177, 1997.
- M. Arakawa, H. Nakayama, I. Hagiwara, and H. Yamakawa, “Multiobjective optimization using adaptive range genetic algorithms with data envelopment analysis,” in Proceedings of the 7th Symposium on Multidisciplinary Analysis and Optimization (AIAA '98), vol. 3, pp. 2074–2082, 1998.
- T. Joro, P. Korhonen, and J. Wallenius, “Structural comparison of data envelopment analysis and multiple objective linear programming,” Management Science, vol. 44, no. 7, pp. 962–970, 1998.
- T. Post and J. Spronk, “Performance benchmarking using interactive data envelopment analysis,” European Journal of Operational Research, vol. 115, no. 3, pp. 472–487, 1999.
- E. Takeda and J. Satoh, “Data envelopment analysis approach to multicriteria decision problems with incomplete information,” Computers and Mathematics with Applications, vol. 39, no. 9, pp. 81–90, 2000.
- H. F. Lewis and T. R. Sexton, “Data envelopment analysis with reverse inputs and outputs,” Journal of Productivity Analysis, vol. 21, no. 2, pp. 113–132, 2004.