Research Article | Open Access
Modifying Olympics Medal Table via a Stochastic Multicriteria Acceptability Analysis
This paper addresses the issue of developing a widely accepted Olympics ranking scheme based upon the Olympic Game medal table published by the International Olympic Committee, since the existing lexicographic ranking and sum ranking systems are both criticized as biases. More specifically, the lexicographic ranking system is deemed as overvaluing gold medals, while the sum ranking system fails to reveal the real value of gold medals and fails to discriminate National Olympic Committees that won equal number of medals. To start, we employ a sophisticated mathematical method based upon the incenter of a convex cone to aggregate the lexicographic ranking system. Then, we consider the fact that the preferences between the lexicographic and the sum ranking systems may not be consistent across National Olympic Committees and develop a well-designed mathematical transformation to obtain interval assessment results under typical preference. The formulation of intervals is inspired by the observation that it is extremely difficult to achieve a group consensus on the exact value of weights with respect to each ranking system, since different weight elicitation methods may produce different weight schemes. Finally, regarding the derived decision making problem involving interval-valued data, this paper utilizes the Stochastic Multicriteria Acceptability Analysis to obtain a comprehensive ranking of all National Olympic Committees. Instead of determining precise weights, this work probes the weight space to guarantee each alternative getting the most preferred one. The proposed method is illustrated by presenting a new ranking of 12 National Olympic Committees participating in the London 2012 Summer Olympic Games.
The Olympics medal table is a list of National Olympic Committees (NOCs) released by the International Olympic Committee (IOC), which ranks NOCs according to the number of gold medals won by single NOC. The number of silver medals is taken into account next and then the number of bronze medals. Meanwhile, total medal count with respect to each NOC is summed and shown in the Olympics medal table, which is widely accepted as the alternative criterion to sort NOCs. These two ranking mechanisms are defined as the lexicographic ranking system and the sum ranking system, respectively. However, the lexicographic ranking system is criticized as overvaluing gold medals. In particular case, the NOCs that won a large quantity of silver and bronze medals but without gold medal are ranked below the NOCs that won only one gold medal. For instance, the 2012 Summer Olympics Medal Table ranks India with zero gold medal, two silver medals, and four bronze medals behind Venezuela with only one gold medal. On the other hand, the sum ranking system takes into consideration of the total sum of the medals won by the NOCs, which inevitably fails to reveal the real value of gold medals and thus fails to discriminate NOCs that won equal number of medals. For instance, the sum ranking system at 2012 London Summer Olympics ranks Uzbekistan with one gold medal and two bronze medals and Thailand with two silver medals and one bronze medal at the same position. This is definitely full of conflict in real life.
Although the IOC publishes the quasi-official medal table during each Olympic Game, the IOC itself has not officially recognized and endorsed any ranking system. The former then-president of IOC, Jacques Rogge, says during the 2008 Beijing Summer Olympic Game:
I believe that each country will highlight what suits it best. One country will say, “Gold medal.” The other country will say, “The total tally counts.” We take no position on that.
The present study is motivated by his viewpoint and seeks to provide a comprehensive ranking system while simultaneously considering the preferences between the lexicographic and the sum ranking systems. Very few studies in literature have addressed the issue that measures the NOCs’ performance based only on the number of medals won. Sitarz [1, 2], for example, introduced two methods based on the weighted mean value and volume-based sensitivity analysis and used the concept of incenter of a convex cone to aid Olympic rankings. Cao et al.  aggregated both ranking systems by minimizing the difference between them and elicited exact weights associated with each system. Although this small body of research is somehow helpful to support Olympics ranking, it is crucial to understand the impact of distinct preferences between the lexicographic ranking system and the sum ranking system on constructing ranking system of NOCs. However, the extant literature has left this important and interesting research topic largely unexplored. This paper fills this gap by first aggregating the lexicographic ranking system through a sophisticated approach, next formulating intervals to assess the NOC-specific achievement under typical preference, and then applying Stochastic Multicriteria Acceptability Analysis (SMAA-2) to rank NOCs with interval input data. The formulation of interval input is motivated by the observation that in the domain of multicriteria decision making (MCDM) different weight determination methods may generate different weights even for the same problem, and it would be extremely difficult to achieve a group consensus about precise value of weights .
Apart from the previous studies about assessing Olympic achievements, this paper provides new research directions and more method options for ranking construction, based upon the methodology developed by Song et al. . The difference is that the rationale of Song et al.  is to determine intervals for the lexicographic ranking system, while this study takes the advantage of a widely accepted weight elicitation method for obtaining the precise weights associated with various medals and then considers different preferences between the lexicographic and sum ranking system. Such explorations go well beyond the general ideas of building ranking and shed much-needed light on potential incentives and directions for academic, managerial, and policy-related implications. This paper contributes to the growing literature on measuring Olympic achievements in the following points:(i)We modify the Olympics medal table published by IOC through jointly considering the lexicographic and the sum ranking systems, while almost all of the extant literature is interested in dealing with the lexicographic ranking system. The weights associated with gold, silver, and bronze medals are obtained through a well-designed mathematical approach to aggregate the lexicographic ranking system.(ii)Different preferences between the lexicographic and the sum ranking systems are proposed and investigated to obtain a holistic ranking. Regarding certain preference, a sophisticated mathematical transformation is developed to support the decision maker generating interval measurement with respect to each NOC. This gives rise to an interval decision matrix for aiding ultimate ranking.(iii)SMAA-2 is applied to determine holistic ranking acceptability index for the proposed interval decision matrix, by which all NOCs could be fully ranked taking into account both the lexicographic and the sum ranking systems.
The rest of this paper proceeds as follows. Section 2 reviews some related paper in literature. Section 3 presents the mathematical formulation of the problem. Section 4 describes SMAA-2 method and the related indices. Section 5 presents the application of SMAA-2 to rank NOCs. Finally, Section 6 concludes this research and provides the directions for future research.
2. Literature Review
2.1. Existing Approaches to Measuring Olympic Achievements
The majority of existing approaches to measuring Olympic achievement are related to Data Envelopment Analysis (DEA), evaluating input (i.e., GDP per capita and population) and output (i.e., the number of medals) efficiency using a family of DEA models and their variants. The pioneering work in this domain is presented by Lozano et al. . Lins et al.  take into consideration the fact that the sum of medals is constant and then develop zero-sum gains DEA model to rank NOCs in Olympics. Li et al.  introduce multiple sets of NOC-specific assurance regions into DEA and thus establish fair models for measuring and benchmarking the performance of NOCs. Soares De Mello et al.  consider that different sports should be of different importance and then propose a modified cross-evaluation DEA model with weight restrictions to generate a ranking for Athens Olympic Games. Wu et al. [10, 11] utilize DEA cross-efficiency approach to rank NOCs. Zhang et al.  incorporate lexicographic preference into DEA models to measure the performance of NOCs at the Olympic Games. Lei et al.  regard the Summer and Winter Olympic Games as a parallel system and then apply a parallel DEA approach to evaluate the efficiency of each NOC. Li et al.  develop a two-stage DEA model to evaluate the performance of NOCs at the 2012 London Summer Olympics.
There are also some complementary approaches that rank NOCs solely considering the number of medals, i.e., Copeland method , mean value and volume-based sensitivity analysis , the incenter of a convex cone , Borda multicriteria method , and distance-based approach .
2.2. Stochastic Multicriteria Acceptability Analysis (SMAA)
Initiated by Lahdelma et al. , SMAA denotes a methodology that intends to support multiperson, multicriteria decision making problem, in which extremely limited or even no weight knowledge is known, and the values of criteria are unknown as well. SMAA does not require any expert to precisely describe the input information and proposes three meaningful and useful indices, namely, acceptability index, central weight, and confidence factor. Lahdelma and Salminen  provide an extension of SMAA through taking all ranks into account and present a comprehensive SMAA-2 analysis to assess alternatives. Regarding the decision making problems including ordinal criteria knowledge, Lahdelma et al. (2003) provide a novel SMAA-O approach. Durbach  integrates SMAA and achievement functions to develop a SMAA-A model with respect to discrete-choice decision, which considers the significance of decision makers’ aspirations. Lahdelma and Salminen  propose cross confidence factor concept on the basis of computing the alternatives’ confidence factors using other’s central weights. Lahdelma and Salminen  integrate SMAA-2 and DEA to assess alternatives in the multicriteria framework. Lahdelma and Salminen  combine SMAA and prospect theory to develop a SMAA-P method. Lahdelma et al. [22, 23], respectively, develop and compare simulation technique and multivariate Gaussian distribution method for treating the uncertainty and dependency information with respect to the SMAA-2 MCDM problem. Tervonen and Lahdelma  provide efficient approaches to perform the computations by Monte Carlo simulation, conduct complexity analysis, and assess the accuracy of proposed algorithms. Corrente et al.  combine SMAA and PROMETHEE to investigate the parameters compatibility together with decision makers’ preference information. Angilella et al.  and Angilella et al. , respectively, integrate SMAA and Choquet integral preference model to achieve robust recommendations and robust ordinal regression. Durbach and Calder  explore the conditions under which decision makers are unable or unwilling to provide exact information in SMAA.
SMAA has been efficiently applied in the domain of decision making, i.e., facility location , forest planning , elevator planning , descriptive multiattribute choice model , estimation of a satisficing model of choice , DEA cross-efficiency aggregation , mutual fund performance assessment , project portfolio optimization , and energy performance evaluation .
To the best of our knowledge, almost all existing studies have ignored the fact that different NOCs may have different preferences between the lexicographic and the sum ranking systems. Even under typical preference, it is significantly difficult to achieve a group consensus on the exact weights with respect to each ranking system. Therefore, this paper pioneers the adventure to formulate intervals to represent the NOC-specific performances and then apply SMAA-2 to holistically rank NOCs in the presence of interval input data.
3. Problem Formulation
3.1. Aggregating the Lexicographic System
Regarding the lexicographic system, this section determines a system of points with respect to various medals. This is in line with the work performed by Sitarz . The conditions considered in this paper are presented as follows:(i)Gold medal should be assigned more points than silver medal, while silver medal should be given more points than bronze medal (ii)The difference between a gold medal and a silver medal is larger than that between a silver medal and a bronze medal [9, 37, 38]
These conditions definitely make sense in real life and could be mathematically expressed by a convex cone as follows:where represents the point for gold medal, denotes the point for silver medal, and is the point for bronze medal.
Inspired by the observations that MCDM and statistical problems usually use the mean value to support decision making , it is reasonable to determine the incenter of the set , which is defined by Henrion and Seeger  as the optimal solution to the following optimization problem:where indicates the unit sphere, represents the boundary of set , and dist means the distance in the Euclidean space. Details on theory justification and mathematical properties about the incenter of convex cone can be found in Henrion and Seeger [39, 40].
Following the numerical methods developed by Henrion and Seeger , the incenter of the set is obtained as follows:where the multiplier is a parameter and does not impact ranking. Therefore, the points associated with gold, silver, and bronze medals are determined as , and 1, respectively. The lexicographic ranking system is then aggregated by calculating the weighted sum of the number of medals and the corresponding points.
Based upon the results derived from aggregating the lexicographic system, we modify the Olympics medal table as shown in Table 1, where , are exact values and have been normalized to eliminate the effect of magnitude of data. Therefore, the evaluation results for each NOC are calculated by weighted sum of the ranking system measures; that is,where are the weights of ranking system associated with NOC , and .
Due to the fact that the preferences between the lexicographic and the sum ranking systems may change across NOCs, we exhaustively denote all possible preferences as follows:where LS denotes the lexicographic ranking system the sum ranking system, and SL represents the sum ranking system the lexicographic ranking system.
Previous studies have proposed a large number of objective and subjective as well as integrated approaches to implement weights determination . However, it is extremely difficult to achieve a group consensus about the exact values of weights for the same decision making problem. To tackle this issue, we use intervals to represent the performance of NOCs under typical preference, the lower and upper bounds of which could be obtained in terms of the least and most favorable performances of NOCs. In what follows, we first present a mathematical model to aggregate the most favorable performance of each NOC under preference :
Theorem 1. The optimal evaluation result of NOC derived from mathematical model (7) is
Proof. We denote and obtainWe incorporate and obtainTherefore, mathematical formulation (7) equals the following model: The dual of (15) is The optimal objective value of (16) is obtained at the point that , which is also the optimal value of (7).
Similarly, the least favorable performance of each NOC under preference can be obtained by the following mathematical formulation:
Theorem 2. The optimal evaluation result of NOC derived from mathematical model (17) is
In a word, the process to determine the lower and upper bounds of NOC-specific intervals is simple-to-understand and easy-to-execute, which could be effectively solved without the elicitation of the exact values of weights. Meanwhile, the results under preference would be derived similarly. Consequently, an interval decision matrix that considers both preferences can be constructed to rank NOCs. This logic to modify the ranking of NOCs is motivated by the work of Song et al. , which takes into account all possible preferences among subindicators for improving the constructing of composite indicators.Reasonable evaluation of NOC under typical preference would be distributed in This kind of decision making problem can be recognized as a stochastic decision making problem [5, 34]. Holistic ranking indices of SMAA-2 have been extensively applied in previous research to rank alternatives. In what follows, we briefly review the SMAA-2 method developed by Lahdelma and Salminen .
As for the MCDM problem with unknown, inexact, or partially missing information, SMAA denotes a set of approaches for support to find solutions. The logic of SMAA is discovering the weight space to obtain the preferences that make individual alternative the most preferred position or ensure a specified ranking position for a certain alternative. Lahdelma et al.  pioneer the trail about this piece of research and develop three useful concepts including rank acceptability index, central weight vector, and confidence factor to rank alternatives. Lahdelma and Salminen  present an extension of the SMAA approach in terms of taking all ranks into account and deliver a more comprehensive SMAA-2 analysis to vividly determine the preference among alternatives.
With respect to the mathematical formulation presented in Section 3, we presume that the decision maker’s judgement on two different preferences between two ranking systems for all NOCs, namely, and , could be denoted by a real-value function , in which the weight vector is proposed to quantitatively express the decision maker’s judgement on the two aforementioned preferences. In addition, the uncertain assessment results for certain NOC under specified preference can be indicated by the stochastic variables , and its density function is typically estimated or assumed in the space . Moreover, the uncertain weight vector would be described by a weight distribution function, the density function of which in the feasible weights set can be defined asTotal loss of weight vector knowledge is denoted in a “Bayesian” manner in terms of a uniform weight distribution in , i.e., . As a matter of course, a reasonable utility function is developed to map the weight distributions and the stochastic assessment results of the interval-valued decision matrix into the utility function. For the ease of clarification, this paper proposes a ranking function to denote the rank of each NOC as an integer ranging from the best position to the worst position as follows:where and .
The SMAA-2 approach is completely dependent on investigating the sets of preferable rank weights defined asin which a weight ensures that NOC achieves rank .
Some useful indices developed by SMAA-2 approach will be introduced in the present subsection. The first is rank acceptability index , alternatively known as the expected cubage of the set of preferable rank weights. Specifically speaking, is capable of measuring the variety of different valuations that guarantee NOC rank , which could be computed usingUndoubtedly, the rank acceptability index is distributed in the interval , and indicates that NOC never reaches rank , and shows that NOC always obtains rank , immune to the particular choice on weights. In case of large-scale problems, an iterative process is developed as follows, wherein the best ranks (mbr) acceptability would be evaluated at each interaction (m):The mbr-acceptability is an assessment of the variety of different judgements that ensure for NOC any of the (m) best rank. Such an analysis will not terminate until one or more NOC achieve a sufficiently acceptable weight.
The weight space associated with the (m) best rank with respect to a NOC could be described using the concept of central weight (mbr) vector as follows:Considering the predetermined weight distribution, the central weight (mbr) vector is regarded as the best single vector representation for the judgement of the decision maker who allocates a NOC any rank from 1 to .
The third index is the mbr confidence factor , the definition of which is the possibility that the NOC achieves any rank from 1 to if the central weight (mbr) vector is calculated by
Additional information about the proposed indices could be found in the paper published by Lahdelma and Salminen . A manual for applying SMAA in real life has been given by Tervonen and Lahdelma .
4.3. Holistic Evaluation of Rank Acceptability
Based upon the mentioned rank acceptability index, next step is to propose a complementary method that is able to integrate the rank acceptability index into holistic acceptability indices for all alternatives as follows:in which are recognized as metaweights to construct holistic acceptability indices and meet .
Barron and Barrett  develop rank-order centroid approach (ROC), i.e., , to assign ranked weights in a lexicographic system and claim that ROC is more straightforward, accurate, and efficacious.
In this section, we will measure the performance of NOCs using the data at the London 2012 Summer Olympic Games. To illustrate the effectiveness of applying SMAA-2 to rank NOCs, we select a set of 12 NOCs from the 2012 Olympics medal table and present them as shown in Table 2.
The published ranking is obtained according to the lexicographic ranking system. However, the sum ranking system generates different ranking from the lexicographic ranking system. This remains a controversy and definitely complicates the formulating of ranking. For the purpose of applying our method, we first aggregate the lexicographic ranking system using medal points derived from (3) and then normalize the results and that of the sum ranking system, which are reported in Table 3.
In addition, the metaweights to construct the holistic acceptability indices are presented asConsequently, the SMAA-2 method can be easily applied by using the open source software proposed by Tervonen .
5.1. Uniform Distribution
In the present subsection, the interval input data in Table 4 are assumed to satisfy the uniform distribution. We derive the rank acceptability indices and the holistic acceptability indices using SMAA-2, which are shown in Table 5 and graphically reported in Figure 1.
The new ranking when we apply SMAA-2 and assume that the interval input data are uniformly distributed is This is significantly different from the lexicographic ranking published by IOC, the difference of which resulted from different generating mechanisms. USA remains in the first position, whose holistic rank index is 1 and first rank support is 100% of the possibility, while KAZ is ranked at the end of the sequence, whose holistic rank index is 0.03, and the last rank support is 100% of the possibility. More specifically, it is observed that the rankings of USA, CHN, GBR, RUS, GER, KAZ, and NED are solidly guaranteed, since the rank acceptability indices with respect to these NOCs are 1.
5.2. Normal Distribution
The ranking in this subsection is the same as that in Section 5.1. This in a sense means that the ranking derived from applying SMAA-2 is robust and reliable, neglecting the exact distributions of the unknown input data. There exist some mild differences about the values of rank acceptability indices. However, there is not any impact on the final ranking.
Note: The sum ranking system ranks NOCs with same medal sum according to the number of gold medals next.
Compared with the lexicographic ranking system, SMAA-2 increases the ranking positions of RUS, GER, FRA, AUS, JPN, and NED and decreases that of GBR, KOR, ITA, and KAZ. Both USA and CHN simultaneously keep their status. Compared with the sum ranking system, SMAA-2 increases the ranking positions of FRA and decreases that of AUS and JPN. The rest of these NOCs keep their positions. Among these three ranking systems, only USA and CHN stay at their ranking positions. This reveals that the rankings of USA and CHN are robust and acceptable. Meanwhile, the ranking positions of FRA, AUS, and JPN change across three ranking systems. That is to say, the ranking positions of them are unreliable and full of conflict.
6. Concluding Remarks
The modern Olympic Games are the leading international sporting event and featured in terms of summer and winter sports competition, with the involvement of over 200 NOCs and thousands of athletes. However, measuring Olympics achievement still remains a controversy and is full of conflict. This paper comprehensively measures the NOC-specific performance in the following three steps. First, we use a sophisticated mathematical method based upon the incenter of a convex cone to aggregate the lexicographic ranking system. Second, we abstract the fact that the preference between the lexicographic and the sum ranking systems may change across NOCs and develop a well-designed mathematical transformation to obtain the NOC-specific evaluation results under certain preference. However, it is extremely difficult to achieve a group consensus about the exact weights associated with each ranking system, since different weight determination approaches may generate different weight results. Therefore, we formulate intervals to represent the NOC-specific achievement under typical preference. Third, regarding the proposed stochastic decision making problem with interval input data, we use SMAA-2 to provide a holistic ranking of all NOCs. Our analysis is illustrated by measuring the performance of 12 NOCs participating the London 2012 Summer Olympic Games. We find out that the final ranking is robust, irrespective of the distribution functions. In addition, comparisons with the lexicographic ranking and the sum ranking are performed to show the difference among these mechanisms. We notice that the majority of NOCs display different ranking positions among them.
Future research is suggested to investigate more options for aggregating the lexicographic ranking system and also study the comprehensive performance of NOCs by taking into account both Summer and Winter Olympics.
(1) This paper provides a new Olympic ranking scheme by considering different preferences between the lexicographic and the sum ranking systems. (2) The lexicographic ranking is aggregated using the concept of incenter of a convex cone. (3) An interval decision matrix is formulated to support Olympics ranking. (4) A Stochastic Multicriteria Acceptability Analysis is employed to obtain final ranking.
Conflicts of Interest
The author declares that there are no conflicts of interest regarding the publication of this paper.
This work is supported by the National Natural Science Foundation of China (NSFC no. 71661011).
- S. Sitarz, “Mean value and volume-based sensitivity analysis for Olympic rankings,” European Journal of Operational Research, vol. 216, no. 1, pp. 232–238, 2012.
- S. Sitarz, “The medal points' incenter for rankings in sport,” Applied Mathematics Letters, vol. 26, no. 4, pp. 408–412, 2013.
- X. Cao, Y. Fu, J. Du, J. Sun, and M. Wang, “Measuring Olympics performance based on a distance-based approach,” International Transactions in Operational Research, vol. 23, no. 5, pp. 979–990, 2016.
- R. Lahdelma and P. Salminen, “SMAA-2: stochastic multicriteria acceptability analysis for group decision making,” Operations Research, vol. 49, no. 3, pp. 444–454, 2001.
- L. Song, Y. Fu, P. Zhou, and K. K. Lai, “Measuring national energy performance via Energy Trilemma Index: A Stochastic Multicriteria Acceptability Analysis,” Energy Economics, vol. 66, pp. 313–319, 2017.
- S. Lozano, G. Villa, F. Guerrero, and P. Cortés, “Measuring the performance of nations at the summer olympics using data envelopment analysis,” Journal of the Operational Research Society, vol. 53, no. 5, pp. 501–511, 2002.
- M. P. E. Lins, E. G. Gomes, J. C. C. B. Soares de Mello, and A. J. R. Soares de Mello, “Olympic ranking based on a zero sum gains DEA model,” European Journal of Operational Research, vol. 148, no. 2, pp. 312–322, 2003.
- Y. Li, L. Liang, Y. Chen, and H. Morita, “Models for measuring and benchmarking olympics achievements,” Omega , vol. 36, no. 6, pp. 933–940, 2008.
- J. C. C. B. S. De Mello, L. Angulo-Meza, and B. P. Branco Da Silva, “A ranking for the Olympic Games with unitary input DEA models,” IMA Journal of Management Mathematics, vol. 20, no. 2, pp. 201–211, 2009.
- J. Wu, L. Liang, and Y. Chen, “DEA game cross-efficiency approach to Olympic rankings,” Omega , vol. 37, no. 4, pp. 909–918, 2009.
- J. Wu, L. Liang, and F. Yang, “Achievement and benchmarking of countries at the Summer Olympics using cross efficiency evaluation method,” European Journal of Operational Research, vol. 197, no. 2, pp. 722–730, 2009.
- D. Zhang, X. Li, W. Meng, and W. Liu, “Measuring the performance of nations at the olympic games using DEA models with different preferences,” Journal of the Operational Research Society, vol. 60, no. 7, pp. 983–990, 2009.
- X. Lei, Y. Li, Q. Xie, and L. Liang, “Measuring Olympics achievements based on a parallel DEA approach,” Annals of Operations Research, vol. 226, no. 1, pp. 1–18, 2015.
- Y. Li, X. Lei, Q. Dai, and L. Liang, “Performance evaluation of participating nations at the 2012 London Summer Olympics by a two-stage data envelopment analysis,” European Journal of Operational Research, vol. 243, no. 3, pp. 964–973, 2015.
- N. C. R. Bergiante and J. C. C. B. Soares de Mello, “A ranking for the Vancouver 2010 winter Olympic Games based on the Copeland method,” in Proceedings of the 3rd IMA International Conference on Mathematics in Sports, The Lowry, Salford Quays, UK, 2011.
- S. F. Gomes, J. C. C. B. Soares De Mello, and L. A. Meza, “Sequential use of ordinal multicriteria methods to obtain a ranking for the 2012 Summer Olympic Games,” WSEAS Transactions on Systems, vol. 13, no. 1, pp. 223–230, 2014.
- R. Lahdelma, J. Hokkanen, and P. Salminen, “SMAA—stochastic multiobjective acceptability analysis,” European Journal of Operational Research, vol. 106, no. 1, pp. 137–143, 1998.
- I. Durbach, “A simulation-based test of stochastic multicriteria acceptability analysis using achievement functions,” European Journal of Operational Research, vol. 170, no. 3, pp. 923–934, 2006.
- R. Lahdelma and P. Salminen, “Classifying efficient alternatives in SMAA using cross confidence factors,” European Journal of Operational Research, vol. 170, no. 1, pp. 228–240, 2005.
- R. Lahdelma and P. Salminen, “Stochastic multicriteria acceptability analysis using the data envelopment model,” European Journal of Operational Research, vol. 170, no. 1, pp. 241–252, 2005.
- R. Lahdelma and P. Salminen, “Prospect theory and stochastic multicriteria acceptability analysis (SMAA),” Omega , vol. 37, no. 5, pp. 961–971, 2009.
- R. Lahdelma, S. Makkonen, and P. Salminen, “Multivariate Gaussian criteria in SMAA,” European Journal of Operational Research, vol. 170, no. 3, pp. 957–970, 2006.
- R. Lahdelma, S. Makkonen, and P. Salminen, “Two ways to handle dependent uncertainties in multi-criteria decision problems,” Omega , vol. 37, no. 1, pp. 79–92, 2009.
- T. Tervonen and R. Lahdelma, “Implementing stochastic multicriteria acceptability analysis,” European Journal of Operational Research, vol. 178, no. 2, pp. 500–513, 2007.
- S. Corrente, J. R. Figueira, and S. Greco, “The SMAA-PROMETHEE method,” European Journal of Operational Research, vol. 239, no. 2, pp. 514–522, 2014.
- S. Angilella, S. Corrente, and S. Greco, “Stochastic multiobjective acceptability analysis for the Choquet integral preference model and the scale construction problem,” European Journal of Operational Research, vol. 240, no. 1, pp. 172–182, 2015.
- S. Angilella, S. Corrente, S. Greco, and R. Słowiński, “Robust Ordinal Regression and Stochastic Multiobjective Acceptability Analysis in multiple criteria hierarchy process for the Choquet integral preference model,” OMEGA - The International Journal of Management Science, vol. 63, pp. 154–169, 2016.
- I. N. Durbach and J. M. Calder, “Modelling uncertainty in stochastic multicriteria acceptability analysis,” OMEGA - The International Journal of Management Science, vol. 64, pp. 13–23, 2016.
- R. Lahdelma, P. Salminen, and J. Hokkanen, “Locating a waste treatment facility by using stochastic multicriteria acceptability analysis with ordinal criteria,” European Journal of Operational Research, vol. 142, no. 2, pp. 345–356, 2002.
- A. S. Kangas, J. Kangas, R. Lahdelma, and P. Salminen, “Using SMAA-2 method with dependent uncertainties for strategic forest planning,” Forest Policy and Economics, vol. 9, no. 2, pp. 113–125, 2006.
- T. Tervonen, H. Hakonen, and R. Lahdelma, “Elevator planning with stochastic multicriteria acceptability analysis,” Omega , vol. 36, no. 3, pp. 352–362, 2008.
- I. N. Durbach, “The use of the SMAA acceptability index in descriptive decision analysis,” European Journal of Operational Research, vol. 196, no. 3, pp. 1229–1237, 2009.
- I. Durbach, “On the estimation of a satisficing model of choice using stochastic multicriteria acceptability analysis,” Omega , vol. 37, no. 3, pp. 497–509, 2009.
- F. Yang, S. Ang, Q. Xia, and C. Yang, “Ranking DMUs by using interval DEA cross efficiency matrix with acceptability analysis,” European Journal of Operational Research, vol. 223, no. 2, pp. 483–488, 2012.
- V. Babalos, N. Philippas, M. Doumpos, and C. Zopounidis, “Mutual funds performance appraisal using stochastic multicriteria acceptability analysis,” Applied Mathematics and Computation, vol. 218, no. 9, pp. 5693–5703, 2012.
- F. Yang, S. Song, W. Huang, and Q. Xia, “SMAA-PO: project portfolio optimization problems based on stochastic multicriteria acceptability analysis,” Annals of Operations Research, vol. 233, no. 1, pp. 535–547, 2015.
- H. L. Hai, “Using vote-ranking and cross-evaluation methods to assess the performance of nations at the Olympics,” WSEAS Transactions on Systems, vol. 6, no. 6, pp. 1196–1205, 2007.
- J. C. C. B. S. de Mello, E. G. Gomes, L. A. Meza, and L. B. Neto, “Cross evaluation using weight restrictions in unitary input DEA models: Theoretical aspects and application to Olympic Games ranking,” WSEAS Transactions on Systems, vol. 7, no. 1, pp. 31–39, 2008.
- R. Henrion and A. Seeger, “On properties of different notions of centers for convex cones,” Set-Valued and Variational Analysis, vol. 18, no. 2, pp. 205–231, 2010.
- R. Henrion and A. Seeger, “Inradius and circumradius of various convex cones arising in applications,” Set-Valued and Variational Analysis, vol. 18, no. 3-4, pp. 483–511, 2010.
- Y.-M. Wang and Y. Luo, “Integration of correlations with standard deviations for determining attribute weights in multiple attribute decision making,” Mathematical and Computer Modelling, vol. 51, no. 1-2, pp. 1–12, 2010.
- F. H. Barron and B. E. Barrett, “Decision quality using ranked attribute weights,” Management Science, vol. 42, no. 11, pp. 1515–1523, 1996.
- T. Tervonen, “JSMAA: Open source software for SMAA computations,” International Journal of Systems Science, vol. 45, no. 1, pp. 69–81, 2014.
Copyright © 2018 Jiangze Du. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.