Table of Contents Author Guidelines Submit a Manuscript
Advances in Operations Research
Volume 2019, Article ID 3574263, 24 pages
https://doi.org/10.1155/2019/3574263
Research Article

An Examination of Ranking Quality for Simulated Pairwise Judgments in relation to Performance of the Selected Consistency Measure

Universite Internationale Jean-Paul II de Bafang, B.P. 213, Bafang, Cameroon

Correspondence should be addressed to Paul Thaddeus Kazibudzki; moc.liamg@atzcopliame

Received 16 October 2018; Revised 7 December 2018; Accepted 24 December 2018; Published 3 February 2019

Academic Editor: Eduardo Fernandez

Copyright © 2019 Paul Thaddeus Kazibudzki. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

An overview of current debates and contemporary research devoted to modeling decision making processes and their facilitation directs attention to techniques based on pairwise judgments. At the core of these techniques are various judgment consistency measures which, in a sense, control the prioritization process which leads to the establishment of decision makers’ unknown preferences. If judgments expressed by decision makers were perfectly consistent (cardinally transitive), all available prioritization techniques would deliver the same solution. However, human judgments are consistently inconsistent, as it were; thus the preference estimation quality significantly varies. The scale of these variations depends, among others, on the chosen consistency measure of pairwise judgments. That is why it seems important to examine relations among various consistency measures and the preferences estimation quality. This research reveals that there are consistency measures whose performance may confuse decision makers with the quality of their ranking outcome. Thus, it introduces a measure which is directly related to the quality of the preferences estimation process. The main problem of the research is studied via Monte Carlo simulations executed in Wolfram Mathematica Software. The research results argue that although the performance of examined consistency measures deviates from the exemplary ones in relation to the estimation quality of decision makers preferences, solutions proposed in this paper can significantly improve that quality.

1. Introduction

Overwhelming scientific evidence indicates that the unaided human brain is simply not capable of simultaneous analysis of many different, competing factors and then synthesizing the results for the purpose of making a rational decision. Indeed, numerous psychological experiments, e.g., [1], including the well-known Miller [2] study put forth the notion that humans are not capable of dealing accurately with more than about seven (±2) things at a time (the human brain is limited in its short term memory capacity, its discrimination ability, and its range of perception).

Humans learn about anything by two means: the first involves examining and studying some phenomenon from the perspective of its various properties and then synthesizing findings and drawing conclusions; the second entails studying some phenomenon in relation to other similar phenomena and relating them by making comparisons. The latter method leads directly to the essence of the matter, i.e., judgments regarding a phenomenon. Judgments can be relative or absolute. An absolute judgment is the relation between a single stimulus and some information held in short or long term memory. A relative judgment, on the other hand [3], is defined as identification of some relation between two stimuli both present to the observer. Certainly humans can make much better relative than absolute judgments. Thus, a pairwise comparisons method was proposed in order to facilitate the process of relative judgments.

Some authors proclaim [4, 5] that this method dates back to the beginning of the 20th century and was firstly applied by Thurstone [6]; however, its first scientific applications can be found in Fechner [7]. In reality, the method itself is much older and its idea goes back to Ramon Lull who lived in the end of 13th century. It is a fact that its popularity comes from an influential paper of Marquis de Condorcet [8]; see, e.g., [9, 10], who used this method in the election process where voters rank candidates based on their preference. It has been perfected in many papers, e.g., [4, 1120].

Also, the method is the core of a decision making methodology—the Analytic Hierarchy Process (AHP), developed by Saaty [21]. Although it is criticized (see [2225, 2528]) it is also continuously perfected and as such very often validated [29] and developed from the perspective of its utility (see [5, 3037]). Thus, considering the stable increase of the AHP application (see, e.g., [3844]) as well as its steady position among other theories of choice, e.g., [3137, 4547], the problematic issues of AHP should be continuously examined and questions which arise thereof should be simultaneously addressed.

The fundamental objective of this research is to determine the answer to the question: Does the reduction of PCM inconsistency lead to improvement of the priority ratios estimation process quality?

2. Background

The AHP uses the hierarchical structure of the decision problem, pairwise relative comparisons of the elements in the hierarchy, and a series of redundant judgments which enable measurement of judgment consistency. To make a proposed solution possible, i.e., derive ratio scale priorities on the basis of verbal judgments, a scale is utilized to evaluate the preferences for each pair of items. Probably, the most known scales are Saaty’s numerical scale which comprises integers, and their reciprocals, from one (equivalent to the verbal judgment, “equally preferred”) to nine (equivalent to the verbal judgment, “extremely preferred”), and a geometric scale which usually consists of the numbers computed in accordance with the formula where c denotes its parameter which commonly equals 2. Other arbitrarily defined numerical scales are also available, e.g., composed of arbitrary integers from one to n and their reciprocals.

The key issue in AHP is priority ranking on the basis of true or approximate weights, i.e., judgments. If the relative weights of a set of activities are known, they can be expressed as a Pairwise Comparison Matrix (PCM): A()=(/),  i, j=1,…, n. PCM in the AHP reflects decision makers’ preferences (their relative judgments) about considered activities (criteria, scenarios, players, alternatives, etc.). On the basis of A(), it is possible to derive true weights; i.e., decision makers priority ratios , where: i=1,…, n, are selected to be positive and normalized to unity: . For uniformity,   is referred hereafter to its normalized form. If the elements of a matrix A() satisfy the condition =1/ for all i, j=1,…, n then matrix A() is called reciprocal. If the elements of a matrix A() satisfy the condition = for all i, j, k=1,…, n, and the matrix is reciprocal, then it is called consistent or cardinally transitive.

Certainly, in real life situations when AHP is utilized, there is no A() which would reflect weights given by the vector of priority ratios. As was stated earlier, the human mind is not a reliable measurement device. Assignments, such as “Compare – applying a given ratio scale – your feelings concerning alternative 1 versus alternative 2”, do not produce accurate outcomes. Thus, A() is not established but only its estimate A(x) containing intuitive judgments, more or less close to A() in accordance with experience, skills, specific knowledge, personal taste, and even temporary mood or overall disposition. In such case, consistency property does not hold and the relation between elements of A(x) and A() can be expressed as = where is a perturbation factor fluctuating near unity. In the statistical approach, eij reflects a realization of a random variable with a given probability distribution.

Besides the prioritization procedure (PP) proposed by the creator of AHP, right principal eigenvector method (REV), there are alternative PPs devised to cope with the priority ratios estimation problem; their demonstrative review can be found, e.g., in [47]. Many of them are optimization based and seek a vector  , as a solution of the minimization problem given by the formula subject to some assigned constraints, such as positive coefficients and normalization condition. Because the distance function D  measures an interval between matrices A(x)   and A(), different definitions of the distance function lead to various prioritization concepts and prioritization results. As an example, eighteen PPs in [48] are described and compared for ranking purposes although some authors suggest there are only fifteen that are different. Furthermore, since the publication of the above-mentioned article, a few additional procedures have been introduced to the literature; see, e.g., [43, 4951]. Probably the most popular alternative to the REV is the Logarithmic Least Squares Method (LLSM) developed by Crawford and Williams [26, 52]. It is given by the following formula:The LLSM solution also has the following closed form and is given by the normalized products of the elements in each row:

Thus, it is also known as the geometric mean method and it is utilized in this research which strives to improve the reliability of the pairwise comparisons process which is also the core element of AHP.

3. Problem and Research Methodology

There are several PCM consistency measures (PCM-CMs) provided in literature called consistency or inconsistency indices (CIs). The most popular one is the PCM-CM proposed by Saaty [21]. He proposed his CI as determined by the formulawhere n indicates the number of alternatives within the particular PCM and denotes its maximal eigenvalue. The significant disadvantage of the PCM-CM is the fact that it can operate exclusively with reciprocal PCMs. In the case of nonreciprocal PCMs, this measure is useless (its values are meaningless) which in consequence seriously diminishes the value of the whole approach; see, e.g., [33]. It was also recently found to be incorrect; see, e.g., [4, 10, 53, 54].

However, as mentioned earlier, there are a number of additional PCM-CMs. Some of them, as in the case of , originate from the PPs devised for the purpose of the priority ratios estimation process. Their distinct feature is the fact that all of them can operate equally efficiently in conditions where reciprocal and nonreciprocal PCMs are accepted. Probably the most known example from that set of propositions is PCM-CM proposed by Aguaron and Moreno-Jimenez [55] given by the following formula:

Noticeably, there are a few definitions of PCM-CMs which are not connected with any PP and are devised on the basis of the PCM consistency definition. Koczkodaj’s idea [56] attracts attention and is the first to be scrutinized. Koczkodaj’s PCM-CM is grounded in his concept of triad consistency.

In order to clarify this, for any three distinguished decision alternatives A1, A2, and A3, there are three meaningful priority ratios, i.e., aij, ajk, and aik, which have their different locations in a particular  . For some different in, jn, and kn, the tuple (aij, ajk, aik) is called a triad. If the matrix   is consistent, then aijajk = aik for all triads.

In consequence, either of the equations and have to be true. Taking the above into consideration, Koczkodaj proposed his measure for triad inconsistency by the following formula:Following his idea, he then proposed the following PCM-CM of any reciprocal PCM=A:where the maximum value for K(A) is taken from the set of all possible triads in the upper triangle of a given PCM.

On the basis of Koczkodaj’s idea of triad inconsistency, Grzybowski [5] presented his PCM-CM determined by the following formula:

Finally, following the idea that , Kazibudzki [57] redefined triad inconsistency and proposed

– Two formulae for its measurement: – One meaningful formula for PCM-CM:

where x denotes the formula for triad inconsistency measurement, i.e., LTI1 or LTI2.

It behooves us to mention that ALTI(A) can be calculated on the basis of triads from the upper triangle of the given PCM when it is reciprocal or all triads within the given PCM when it is nonreciprocal.

As was already stated earlier, the fundamental question which should be asked by researchers who deal with the problem of priority ratios estimation quality in relation to a PCM consistency measure is as follows: Does the reduction of PCM inconsistency lead to improvement of the priority ratios estimation process quality?

The common reason why one strives to improve the consistency of the PCM, when it seems unsatisfactory, is to increase the quality of the priority ratios estimation process. However, the above question remains open and the answer to it is not evident. Even the creator of AHP stated once that improving consistency does not mean getting an answer closer to the “real” life solution [21]. It can be illustrated in the following example.

Considered is the true PV (denoting true weights of examined alternatives), i.e., =/20, 1/4, 1/4, 3/ and A() derived from that PV, which can be presented as follows:

Then two PCMs are considered, i.e., R(x) and A(x) produced by a hypothetical decision maker (DM). It is assumed that DM is very trustworthy and is able to express judgments very precisely at the same time being still somehow limited by the necessity of expressing judgments on a scale (the example utilizes Saaty’s scale). In the first scenario, entries of A() are rounded to Saaty’s scale and the entries are made reciprocal (a principal condition for a PCM in the AHP) producingIn the second scenario, only entries of A() are rounded to Saaty’s scale (nonreciprocal case) producing

It should be noted thatR(x) is perfectly consistent and A(x) is not. Tables 1 and 2 present selected values of the PPs related PCM-CMs (that is, and ) forR(x) and A(x) together with PVs derived from R(x)   and A(x); Mean Absolute Errors (MAEs), formula (14), among  (PP) and   for the case; Spearman Rank Correlation Coefficients (SRCs) among   (PP) and for the case.

Table 1: Values of the PCM-CMs denoted as CI(PP) for R(x) and proposed characteristics of PVs estimates () quality in relation to the true PV for the case.
Table 2: Values of the PCM-CMs denoted as CI(PP) for A(x) and proposed characteristics of PVs estimates () quality in relation to the true PV for the case.

Surprisingly, a very interesting phenomenon can be noted on the basis of information provided in Tables 1 and 2. The nonreciprocal version of the analyzed PCM contains nonzero values for the selected PCM-CMs. In cases similar to this example, the value of Saaty’s PCM-CM always becomes negative which makes it inexplicable and in consequence useless under such circumstances (as already mentioned earlier). The other exemplary measure taken into consideration is positive and higher values than zero which indicates that particular PCM is inconsistent. On the basis of the same indicators in the case of the reciprocal version of the analyzed PCM, its perfect consistency is apparent because all selected PCM-CMs in this case are equal to zero. However, the estimation precision measures (MAE and SRC), i.e., characteristics of the particular PV estimation quality, indicate something quite opposite. Surprisingly, smaller values of MAEs are apparent, as are perfect correlations of ranks between estimated and genuine PV for nonreciprocal version of the analyzed PCM. Certainly, this conclusion concerns all analyzed PPs and it is very true in the situation when the particular PCM is less consistent (on the basis of selected exemplary PCM-CMs). It remains to be mentioned that the inconsistency problem (hence errors) does not exist for simplified pairwise comparisons [53].

Taking into consideration only such a trivial example like the one above, it becomes apparent that the relation between performance of a consistency measure and the quality of the priority ratios estimation process is of great importance. That is why to examine the phenomenon further to improve the quality of the pairwise comparisons based prioritization process was decided. Thus, the simulation framework for this purpose was adopted from [5, 57] as the only way to examine the said phenomena through computer simulations. The simulation algorithm is comprised of the following phases.

Phase 1. Randomly generate a priority vector of assigned size [n x 1] and related perfect PCM()=K().

Phase 2. Randomly select an element for x<y ofK() and replace it with where eB is a relatively significant error, randomly drawn (uniform distribution) from the interval . Errors of that magnitude are basically considered “significant”; see, e.g., [50, 58].

Phase 3. For each other element , i<j≤n, select randomly a value eij for the relatively small error in accordance with the given probability distribution π (applied in equal proportions as gamma, log-normal, truncated normal, and uniform distribution) and replace the element with the element where eij is randomly drawn (uniform distribution) from the interval .

Phase 4. Round all values of for i<j ofK() to the nearest value of a considered scale.

Phase 5. Replace all elements for i>j ofK() with 1/.

Phase 6. After all replacements are done, return the value of the examined index as well as the estimate of the vector   denoted as  (PP) with application of assigned prioritization procedure. Then return the mean absolute error (MAE) and mean relative error (MRE) (formula (15)) between   and  (PP). Remember values computed in this phase as one record.

Phase 7. Repeat Phases from Phase 2 to Phase 6 Nn times.

Phase 8. Repeat Phases from Phase 1 to Phase 7 Nm times.

Phase 9. Save all records as one database file.

The above algorithm allows examining the performance of the selected CM in relation to the quality of the priority ratios estimation process. Its framework resembles steps scrutinized in the example provided earlier in this paper and was thoroughly described in [5]. Thus, for brevity, it will not be analyzed in detail herein.

For formality, all parameters of the applied PDs, gamma, log-normal, truncated normal, and uniform, in the simulation algorithm are set in such a way that the expected value EV(eij)=1. The simulation begins from n=4. Simulations for n=3 are not interesting due to direct interrelation of considered PCM consistency measures; see, e.g., [58, 59]. For the sake of objectivity, the simulation data is gathered in the following way: all values of selected consistency measures are split into 15 separate sets designated by the quantiles Q of order p from 1/15 to 14/15. The 15 intervals are defined as the first is from 0 to the quantile of order 1/15, i.e., VRCM1=[0, ), where VRCM represents a Value Range of the Selected PCM Consistency Measure; the second denotes VRCM2=[, ), and so on… to the last one which starts from the quantile of order 14/15 and goes on to infinity, i.e., VRCM15. The following variables are examined: mean , average MAE within between   and  (PP), MAE quantiles of the following orders, 0.05, 0.1, 0.5, 0.9, and 0.95, and relations between all of them. The application of the rounding procedure was also assumed which in this research operates according to Saaty’s scale. Lastly, the scenario takes into account the compulsory assumption in conventional AHP applications, i.e., the PCM reciprocity condition. The outcome of the simulation program is presented for the most popular PP=LLSM, and the most attractive PCM-CM=K(A). It must be emphasized that other PPs and PCM-CMs were also examined but results with their application will not be presented in this research paper because this is beyond its scope. The results are based on Nn=20, and Nm=500, i.e., 10,000 cases.

4. Results and Discussion

One could assume that MAE and MRE quantiles of any order should monotonically increase concurrently with the growth of the selected PCM-CM, e.g., VRCM index. The same relation should occur for mean and average MAE and MRE for . The results of the proposed simulation framework or any other similar simulation scenario which would contradict such a pertinent relationship would unequivocally lead to the conclusion that the examined PCM-CM is not a good indicator of the priority ratios estimation process quality and may mislead in further actions towards acceptance or rejection of the derived priority ratios vector.

An examination of the research problem depicts Figures 1 and 2 which present the performance of Koczkodaj K(A) (Plots A–H). In each case hereafter a horizontal axis represents values of examined PCM-CM, and a vertical axis denotes particular estimation errors.

Figure 1: Performance of K(A) as PCM-CM. The plots present the relation between mean values of K(A) within a given interval () and values of MAEs and quantiles of a given order for MAEs distribution concerning estimated and true PV for the case. The results are generated with application of LLSM as the PP and geometric scale, i.e., as the preference scale with its most popular parameter c=2. Plots are based on 10,000 random reciprocal PCMs for n=4 and n=6. MSRC denotes Mean Spearman Rank Correlation Coefficient between analyzed variables and describes their relation’s strength.
Figure 2: Performance of K(A) as PCM-CM. The plots present the relation between mean values of K(A) within a given interval () and values of MREs and quantiles of a given order for MREs distribution concerning estimated and true PV for the case. The results are generated with application of LLSM as the PP and geometric scale, i.e., as the preference scale with its most popular parameter c=2. Plots are based on 10,000 random reciprocal PCMs for n=4 and n=6. MSRC denotes Mean Spearman Rank Correlation Coefficient between analyzed variables and describes their relation’s strength.

Noticeably, when the quality of priority ratios estimation in a pairwise comparisons based process is taken into consideration, the presented relations indicate that the performance of selected PCM-CM varies from the assumption presented earlier. The phenomenon was thus far only examined in [5]. It may seem disturbing because the relation between priority ratios estimation errors and analyzed PCM-CM indicates that the analyzed index may sometimes misinform DMs about their judgments’ quality, affecting the vector of priorities which best converge with the true vector.

As seen similarly in the example provided earlier in this paper (Tables 1 and 2), taking the particular index as the measure of PCM consistency, especially when errors are small and inconsistency is not negligible, one can expect both, i.e., the improvement of priority ratios estimation quality (increase of the estimation accuracy) together with the increase of the particular CI (decrease of PCM consistency) and, inversely, the deterioration of priority ratios estimation quality (decrease of the estimation accuracy) together with the descent of the particular CI (improvement of PCM consistency).

This problem seems very troublesome especially when differences among derived priority ratios are insignificant. Then, it may occur that a ranking order of alternatives in the estimated priority vector can drastically differ from the true one because of estimation errors. From that perspective, a necessity of controlling that issue seems paramount. In order to evaluate that problem more thoroughly, detailed statistical characteristics for the examined K(A) are provided in Tables 36. It can be concluded that the performance of the presented PCM-CM varies (compare, e.g., [9]).

Table 3: Performance of the K(A) index. Statistical characteristics of the MAEs distribution in relation to various for i=1,…,15 of K(A) values. The results were generated for n=4 on the basis of as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and Saaty’s preference scale.
Table 4: Performance of the K(A) index. Statistical characteristics of the MREs distribution in relation to various for i=1,…,15 of K(A) values. The results were generated for n=4 on the basis of as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and Saaty’s preference scale.
Table 5: Performance of the K(A) index. Statistical characteristics of the MAEs distribution in relation to various for i=1,…,15 of K(A) values. The results were generated for n=6 on the basis of as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and the geometric preference scale.
Table 6: Performance of the K(A) index. Statistical characteristics of the MREs distribution in relation to various for i=1,…,15 of K(A) values. The results were generated for n=6 on the basis of as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and the geometric preference scale.

The motivation for this research was to develop a PCM-CM which could depict a more credible relation between the consistency of pairwise judgments and the priority ratios estimation quality, i.e., whose priority ratios estimation error, reflected by SRC, would be very close or equal to 1 for all their quantiles (the most desirable situation).

Successfully, a solution of the problem was generated and is presented as follows. On the basis of triad inconsistency measure introduced in [57], the following PCM-CM was devised:

The proposed PCM-CM is denoted as the Triads Squared Logarithm Corrected Mean and an examination of its performance on the basis of simulation algorithm   proposed earlier in this paper was carried out (Figures 3 and 4).

Figure 3: Performance of the new PCM-CM: TSL(A). The plots present the relation between a mean value of TSL(A) within a given interval () and quantiles of order 0.1 (Plot A), 0.5 (Plot B), and 0.9 (Plot C) of MAEs distribution as well as MAEs average values (Plot D) for estimated and true PVs. The results were generated with application of LLSM as the PP and Saaty’s scale. Plots are based on 10,000 random reciprocal PCMs for n=4.
Figure 4: Performance of the new PCM-CM: TSL(A). The plots present the relation between a mean value of TSL(A) within a given interval () and quantiles of order 0.1 (Plot A), 0.5 (Plot B), and 0.9 (Plot C) of MREs distribution as well as MREs average values (Plot D) for estimated and true PVs. The results were generated with application of LLSM as the PP and geometric scale. Plots are based on 10,000 random reciprocal PCMs for n=4.

As can be noticed, the proposed TSL(A) performs credibly from the perspective of the relation among the consistency of pairwise judgments and the priority ratios estimation quality. It is undeniably a positive piece of information which opens a new chapter in pairwise judgments based priority ratios estimation process embedded in many methodologies of decision making such as AHP. It behooves us to mention that TSL(A) is suitable for both reciprocal and nonreciprocal PCMs which prospectively may improve the pairwise judgments based priority ratios estimation quality when nonreciprocal PCMs are accepted.

Tables 7 and 8 provide detailed characteristics data for TSL(A) with application of LLSM (as the most popular alternative to REV), Saaty’s scale, and geometric scale as the most popular preference scales. Results for other PPs are similar; thus they are not presented in order to conserve the length of this paper. However, it is stressed that other PPs and preference scales were also tested and examined during the research and their results are not depicted in this paper because they coincide with results herein presented.

Table 7: Performance of the TSL(A) index. Statistical characteristics of the MAEs distribution in relation to various for i=1,…,15 of TSL(A) values. The results were generated for n=4 on the basis of as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and Saaty’s preference scale.
Table 8: Performance of the TSL(A) index. Statistical characteristics of the MREs distribution in relation to various for i=1,…,15 of TSL(A) values. The results were generated for n=4 on the basis of as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and geometric scale for preferences.

It is noted that all statistical characteristics of the MAEs and MREs distribution in relation to various VRCMi for i=1,…,15 of TSL(A) values, with few exceptions, monotonically grow. This examination ascertains that the proposed PCM-CM is a suitable measure of relation among pairwise comparisons consistency and the priority ratios estimation quality. The paramount position of the proposed TSL(A) is that it performs better than the other, evaluated here, PCM-CM, i.e., K(A). Its position is additionally strengthened by the fact that its performance is similar and independent from the applied PP and improves significantly for higher numbers of alternatives without regard to which PP is selected.

It should be noted that all characteristics presented herein are of great importance in the priority ratios estimation process, because one has to consider the potential of rejecting a “good” PCM, and vice versa, i.e., the possibility of accepting a “bad” PCM, as in the classic statistical hypothesis testing theory. However, for the first time in the course of pairwise judgments based prioritization development history, the possibility of selecting the level of certainty and basing decisions on statistical facts has been demonstrated.

For instance, considering some hypothetical PCM for n=4, with its mean TSL(A)≈0.319702 for LLSM as the PP (Table 7), one can expect with 95% confidence that the MAE should not exceed the value of 0.1208420. At the same time, one can expect with 95% confidence that it will be higher than 0.0201740 (Table 7). Whether one decides to accept such a PCM or reject it obviously depends on the quality requirements of the priority ratios estimation and the attitude regarding these errors. Indeed, the outcome of the research finally creates the potential for true consistency control in an unprecedented way, i.e., directly related to the priority ratios estimation quality.

For example, the following PV is considered as  = denoting DM preferences for alternatives A1, A2, A3, and A4, respectively. Taking into consideration the earlier assumed level of TSL(A)≈0.319702, the order of alternatives ranks A1=1, A2=2, A3=3, and A4=4 can be very deceptive and is rather meaningless. In such a situation one can expect with 95% confidence that the MAE>0.0201740 which makes one aware that the true rank order of examined preferences may appear otherwise, due to estimation errors related to DM inconsistency, e.g.,  =[(0.27–0.025), (0.26–0.025), (0.24+0.025), (0.23+0.025)] = [0.242, 0.231, 0.269, 0.258], with MAE=0.025, which designates a different order for alternatives ranks, A1=3, A2=4, A3=1, and A4=2. On the other hand, consider PV as  = of DM preferences for alternatives A1, A2, A3, and A4 consecutively, as previously. Assuming this time TSL(A)≈0.049049, it can be anticipated with 95% certainty that the MAE<0.072271 which rather insures that the order of alternatives ranks should remain unchanged.

For similar but even more detailed calculation, MRE can be applied. It is a more accurate measure of priority ratios deviation; however, its straightforward application for calculation of discrepancies within normalized priority vectors is problematic.

In order to enable other researchers to make similar analysis concerning different numbers of alternatives, the exemplary characteristics of TSL(A) performance are provided for n>4 in the Appendix of this article, computed with application of commonly known PP, i.e., LLSM and the most common Saaty's scale (Table 9) and geometric scale (Table 10) as prospective preferences scales selected by decision makers.

Table 9: Performance of TSL(A) index under the action of LLSM as the PP. Statistical characteristics of the MAEs distribution in relation to various levels of TSL(A) within a given for i=1,…,15. The results are based on 10,000 perturbed random reciprocal PCMs with application of Saaty’s scales and were generated on the basis of as the simulation . The table contains results for , presented consecutively.
Table 10: Performance of TSL(A) index under the action of LLSM as the PP. Statistical characteristics of the MREs distribution in relation to various levels of TSL(A) within a given for i=1,…,15. The results are based on 10,000 perturbed random reciprocal PCMs with application of geometric scale and were generated on the basis of as the simulation algorithm. The table contains results for , presented consecutively.

5. Conclusions

In this research, the performance of the selected PCM-CM from the perspective of its relation between pairwise judgments consistency and the quality of the priority ratios estimation process was examined with application of the most popular PP, i.e., LLSM, preference scales, and number of alternatives. The research findings can be stated as follows:(i)It is possible to improve the quality of pairwise judgments based priority ratios estimation when Triads Squared Logarithm Corrected Mean (TSL(A)) is applied as the PCM-CM(ii)From the perspective of TSL(A) performance, it is stable and its results differ insignificantly from the perspective of applied prioritization procedure (i.e., LLSM or any other selected PP)(iii)When the number of alternatives increases, the performance of the examined TSL(A) improves independently from the applied PP(iv)The examined TSL(A) can be applied to both reciprocal and nonreciprocal PCMs broadening their applications and perfecting the quality of decision makers’ preferences estimation process

Certainly, there is a place for further research in the field; e.g., advantages of TSL(A) in comparison to other consistency measures can be examined theoretically (which was beyond the scope of this research), and the performance of TSL(A) can be examined when nonreciprocal PCMs are applied what encourage further development of pairwise judgments methodology.

Appendix

See Tables 9 and 10.

Data Availability

All necessary data concerning the manuscript are included within its content. Not included to the manuscript, additional data might be delivered upon request of interested parties in the form of text files.

Conflicts of Interest

The author declares that there are no conflicts of interest regarding the publication of this paper.

References

  1. J. Martin, Design of Man-Computer Dialogues, Prentice Hall, Englewood Cliffs, NY, USA, 1973.
  2. G. A. Miller, “The magical number seven, plus or minus two: some limits on our capacity for processing information,” The Psychological Review, vol. 63, no. 2, pp. 81–97, 1956. View at Publisher · View at Google Scholar · View at Scopus
  3. A. L. Blumenthal, The Process of Cognition, Prentice Hall, Englewood Cliffs, New York, 1977.
  4. W. W. Koczkodaj and R. Urban, “Axiomatization of inconsistency indicators for pairwise comparisons,” International Journal of Approximate Reasoning, vol. 94, pp. 18–29, 2018. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  5. A. Z. Grzybowski, “New results on inconsistency indices and their relationship with the quality of priority vector estimation,” Expert Systems with Applications, vol. 43, pp. 197–212, 2016. View at Google Scholar · View at Scopus
  6. L. L. Thurstone, “A law of comparative judgment,” Psychological Review, vol. 34, no. 4, pp. 273–286, 1927. View at Publisher · View at Google Scholar · View at Scopus
  7. G. T. Fechner, Elementeder Psychophysik, Breitkopf und Hartel, Leipzig, Germany, 1860.
  8. M. J. A. de Condorcet, Essai Sur L'application De L'analyse À La Probabilité Des Décisionsrendues À La Pluralité Des Voix, fDe l'Imprimerie Royale, Paris, France, 1785, [fascimile edition New York: Chelsea, 1972].
  9. I. L. Tomashevskii, “Eigenvector ranking method as a measuring tool: formulas for errors,” European Journal of Operational Research, vol. 240, no. 3, pp. 774–780, 2015. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  10. W. Xu, Y. Dong, and W. Xiao, “Is It Reasonable for Saaty's Consistency Test in the Pairwise Comparison Method?” in Proceedings of the 2008 ISECS International Colloquium on Computing, Communication, Control, and Management, vol. 3, pp. 294–298, Guangzhou, China, August 2008. View at Publisher · View at Google Scholar
  11. L. Csató, “Characterization of an inconsistency ranking for pairwise comparison matrices,” Annals of Operations Research, vol. 261, no. 1-2, pp. 155–165, 2018. View at Publisher · View at Google Scholar · View at MathSciNet
  12. É. Orbán-Mihálykó, C. Mihálykó, and L. Koltay, “A generalization of the Thurstone method for multiple choice and incomplete paired comparisons,” Central European Journal of Operations Research, pp. 1–27, 2017. View at Publisher · View at Google Scholar
  13. M. Thimm, “Measuring inconsistency with many-valued logics,” International Journal of Approximate Reasoning, vol. 86, pp. 1–23, 2017. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  14. W. W. Koczkodaj, J. Szybowski, and E. Wajch, “Inconsistency indicator maps on groups for pairwise comparisons,” International Journal of Approximate Reasoning, vol. 69, pp. 81–90, 2016. View at Publisher · View at Google Scholar · View at Scopus
  15. L. Cholvy, L. Perrussel, and J.-M. Thevenin, “Using inconsistency measures for estimating reliability,” International Journal of Approximate Reasoning, vol. 89, pp. 41–57, 2017. View at Publisher · View at Google Scholar · View at MathSciNet
  16. J. Grant and A. Hunter, “Analysing inconsistent information using distance-based measures,” International Journal of Approximate Reasoning, vol. 89, pp. 3–26, 2017. View at Publisher · View at Google Scholar · View at MathSciNet
  17. M. Salvador, A. Altuzarra, P. Gargallo, and J. M. Moreno-Jiménez, “A Bayesian Approach to Maximising Inner Compatibility in AHP-Systemic Decision Making,” Group Decision and Negotiation, vol. 24, no. 4, pp. 655–673, 2015. View at Publisher · View at Google Scholar · View at Scopus
  18. W. Qian, K. W. Li, and Z. Wang, “Approaches to improving consistency of interval fuzzy preference relations,” Journal of Systems Science and Systems Engineering, vol. 23, no. 4, pp. 460–479, 2014. View at Publisher · View at Google Scholar
  19. Y. Wang, Y. Luo, and Y. Xu, “Cross-Weight Evaluation for Pairwise Comparison Matrices,” Group Decision and Negotiation, vol. 22, no. 3, pp. 483–497, 2013. View at Publisher · View at Google Scholar
  20. S. S. Hosseinian, H. Navidi, and A. Hajfathaliha, “A new linear programming method for weights generation and group decision making in the analytic hierarchy process,” Group Decision and Negotiation, vol. 21, no. 3, pp. 233–254, 2012. View at Publisher · View at Google Scholar · View at Scopus
  21. T. L. Saaty, “A scaling method for priorities in hierarchical structures,” Journal of Mathematical Psychology, vol. 15, no. 3, pp. 234–281, 1977. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  22. C. A. Bana e Costa and J.-C. Vansnick, “A critical analysis of the eigenvalue method used to derive priorities in AHP,” European Journal of Operational Research, vol. 187, no. 3, pp. 1422–1428, 2008. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  23. A. Farkas, “The analysis of the principal eigenvector of pairwise comparison matrices,” Acta Polytechnica Hungarica, vol. 4, no. 2, 2007, http://uni-obuda.hu/journal/Farkas_10.pdf. View at Google Scholar
  24. J. Barzilai, “Measurement and preference function modelling,” International Transactions in Operational Research, vol. 12, no. 2, pp. 173–183, 2005. View at Publisher · View at Google Scholar · View at MathSciNet
  25. D. V. Budescu, R. Zwick, and A. Rapoport, “A Comparison of the Eigenvalue Method and The Geometric Mean Procedure for Ratio Scaling,” Applied Psychological Measurement, vol. 10, no. 1, pp. 69–78, 1986. View at Publisher · View at Google Scholar · View at Scopus
  26. G. Crawford and C. Williams, “A note on the analysis of subjective judgment matrices,” Journal of Mathematical Psychology, vol. 29, no. 4, pp. 387–405, 1985. View at Publisher · View at Google Scholar · View at Scopus
  27. V. Belton and T. Gear, “On a short-coming of Saaty's method of analytic hierarchies,” Omega , vol. 11, no. 3, pp. 228–230, 1983. View at Publisher · View at Google Scholar · View at Scopus
  28. N. V. Hovanov, J. W. Kolari, and M. . Sokolov, “Deriving weights from general pairwise comparison matrices,” Mathematical Social Sciences, vol. 55, no. 2, pp. 205–220, 2008. View at Publisher · View at Google Scholar · View at MathSciNet
  29. T. L. Saaty, “There is no mathematical validity for using fuzzy number crunching in the analytic hierarchy process,” Journal of Systems Science and Systems Engineering, vol. 15, no. 4, pp. 457–464, 2006. View at Publisher · View at Google Scholar · View at Scopus
  30. P. T. Kazibudzki, “An examination of performance relations among selected consistency measures for simulated pairwise judgments,” Annals of Operations Research, vol. 244, no. 2, pp. 525–544, 2016. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  31. K. Chen, G. Kou, J. M. Tarn, and Y. Song, “Bridging the gap between missing and inconsistent values in eliciting preference from pairwise comparison matrices,” Annals of Operations Research, vol. 235, pp. 155–175, 2015. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  32. V. Pereira and H. G. Costa, “Nonlinear programming applied to the reduction of inconsistency in the AHP method,” Annals of Operations Research, vol. 229, pp. 635–655, 2015. View at Publisher · View at Google Scholar · View at MathSciNet
  33. P. Linares, S. Lumbreras, A. Santamaría, and A. Veiga, “How relevant is the lack of reciprocity in pairwise comparisons? An experiment with AHP,” Annals of Operations Research, vol. 245, no. 1-2, pp. 227–244, 2016. View at Publisher · View at Google Scholar · View at MathSciNet
  34. J. M. Moreno-Jiménez, M. Salvador, P. Gargallo, and A. Altuzarra, “Systemic decision making in AHP: a Bayesian approach,” Annals of Operations Research, vol. 245, no. 1-2, pp. 261–284, 2016. View at Publisher · View at Google Scholar · View at MathSciNet
  35. J. Aguarón, M. T. Escobar, and J. M. Moreno-Jiménez, “The precise consistency consensus matrix in a local AHP-group decision making context,” Annals of Operations Research, vol. 245, no. 1-2, pp. 245–259, 2016. View at Publisher · View at Google Scholar · View at MathSciNet
  36. C. Lin, G. Kou, and D. Ergu, “An improved statistical approach for consistency test in AHP,” Annals of Operations Research, vol. 211, no. 1, pp. 289–299, 2013. View at Publisher · View at Google Scholar · View at MathSciNet
  37. M. Brunelli, L. Canal, and M. Fedrizzi, “Inconsistency indices for pairwise comparison matrices: a numerical study,” Annals of Operations Research, vol. 211, pp. 493–509, 2013. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  38. L. Lidinska and J. Jablonsky, “AHP model for performance evaluation of employees in a Czech management consulting company,” Central European Journal of Operations Research, vol. 26, no. 1, pp. 239–258, 2018. View at Publisher · View at Google Scholar · View at Scopus
  39. R. K. Goyal and S. Kaushal, “Deriving crisp and consistent priorities for fuzzy AHP-based multicriteria systems using non-linear constrained optimization,” Fuzzy Optimization and Decision Making, vol. 17, no. 2, pp. 195–209, 2017. View at Publisher · View at Google Scholar · View at MathSciNet
  40. J. Krejčí, O. Pavlačka, and K. Talašová, “A fuzzy extension of analytic hierarchy process based on the constrained fuzzy arithmetic,” Fuzzy Optimization and Decision Making, vol. 16, no. 1, pp. 89–110, 2017. View at Publisher · View at Google Scholar · View at MathSciNet
  41. T. F. Abdelmaguid and W. Elrashidy, “Halting decisions for gas pipeline construction projects using AHP: a case study,” International Journal of Operational Research, pp. 1–21, 2016. View at Google Scholar
  42. M. García-Melón, B. Pérez-Gladish, T. Gómez-Navarro, and P. Mendez-Rodriguez, “Assessing mutual funds' corporate social responsibility: a multistakeholder-AHP based methodology,” Annals of Operations Research, vol. 244, no. 2, pp. 475–503, 2016. View at Publisher · View at Google Scholar · View at MathSciNet
  43. G. Kou and C. Lin, “A cosine maximization method for the priority vector derivation in AHP,” European Journal of Operational Research, vol. 235, no. 1, pp. 225–232, 2014. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  44. S. T. Ponis, S. P. Gayialis, I. P. Tatsiopoulos, N. A. Panayiotou, D.-R. I. Stamatiou, and A. C. Ntalla, “An application of AHP in the development process of a supply chain reference model focusing on demand variability,” Operational Research, vol. 15, no. 3, pp. 337–357, 2014. View at Publisher · View at Google Scholar · View at Scopus
  45. J. Mazurek and R. Perzina, “On the inconsistency of pairwise comparisons: an experimental study. Scientific papers of the Series D,” Faculty of Economics and Administration, vol. 41, pp. 1–8, 2017, https://dk.upce.cz/handle/10195/69597. View at Google Scholar
  46. G. Kou, D. Ergu, and J. Shang, “Enhancing data consistency in decision matrix: adapting Hadamard model to mitigate judgment contradiction,” European Journal of Operational Research, vol. 236, no. 1, pp. 261–271, 2014. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  47. G. Kou, D. Ergu, C. Lin, and Y. Chen, “Pairwise comparison matrix in multiple criteria decision making,” Technological and Economic Development of Economy, vol. 22, no. 5, pp. 738–765, 2016. View at Publisher · View at Google Scholar · View at Scopus
  48. E. Choo and W. Wedley, “A common framework for deriving preference values from pairwise comparison matrices,” Computers & Operations Research, vol. 31, no. 6, pp. 893–908, 2004. View at Publisher · View at Google Scholar
  49. K. Kułakowski, “A heuristic rating estimation algorithm for the pairwise comparisons method,” Central European Journal of Operations Research, vol. 23, no. 1, pp. 187–203, 2015. View at Publisher · View at Google Scholar · View at MathSciNet
  50. A. Z. Grzybowski, “Note on a new optimization based approach for estimating priority weights and related consistency index,” Expert Systems with Applications, vol. 39, no. 14, pp. 11699–11708, 2012. View at Publisher · View at Google Scholar · View at Scopus
  51. C. Lin, “A revised framework for deriving preference values from pairwise comparison matrices,” European Journal of Operational Research, vol. 176, no. 2, pp. 1145–1150, 2007. View at Publisher · View at Google Scholar
  52. G. B. Crawford, “The geometric mean procedure for estimating the scale of a judgment matrix,” Applied Mathematical Modelling: Simulation and Computation for Engineering and Environmental Systems, vol. 9, no. 3-5, pp. 327–334, 1987. View at Publisher · View at Google Scholar · View at MathSciNet
  53. W. W. Koczkodaj and J. Szybowski, “Pairwise comparisons simplified,” Applied Mathematics and Computation, vol. 253, pp. 387–394, 2015. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  54. W. W. Koczkodaj, J.-P. Magnot, J. Mazurek et al., “On normalization of inconsistency indicators in pairwise comparisons,” International Journal of Approximate Reasoning, vol. 86, pp. 73–79, 2017. View at Publisher · View at Google Scholar · View at Scopus
  55. J. Aguarón and J. M. Moreno-Jiménez, “The geometric consistency index: Approximated thresholds,” European Journal of Operational Research, vol. 147, no. 1, pp. 137–145, 2003. View at Publisher · View at Google Scholar
  56. W. W. Koczkodaj, “A new definition of consistency of pairwise comparisons,” Mathematical and Computer Modelling, vol. 18, no. 7, pp. 79–84, 1993. View at Publisher · View at Google Scholar · View at Scopus
  57. P. T. Kazibudzki, “Redefinition of triad's inconsistency and its impact on the consistency measurement of pairwise comparison matrix,” Journal of Applied Mathematics and Computational Mechanics, vol. 15, no. 1, pp. 71–78, 2016. View at Publisher · View at Google Scholar
  58. T. K. Dijkstra, “On the extraction of weights from pairwise comparison matrices,” Central European Journal of Operations Research, vol. 21, no. 1, pp. 103–123, 2013. View at Publisher · View at Google Scholar · View at MathSciNet
  59. S. Bozóki and T. Rapcsák, “On Saaty's and Koczkodaj's inconsistencies of pairwise comparison matrices,” Journal of Global Optimization, vol. 42, no. 2, pp. 157–175, 2008. View at Publisher · View at Google Scholar · View at MathSciNet