Advances in Operations Research

Volume 2019, Article ID 3574263, 24 pages

https://doi.org/10.1155/2019/3574263

## An Examination of Ranking Quality for Simulated Pairwise Judgments in relation to Performance of the Selected Consistency Measure

Universite Internationale Jean-Paul II de Bafang, B.P. 213, Bafang, Cameroon

Correspondence should be addressed to Paul Thaddeus Kazibudzki; moc.liamg@atzcopliame

Received 16 October 2018; Revised 7 December 2018; Accepted 24 December 2018; Published 3 February 2019

Academic Editor: Eduardo Fernandez

Copyright © 2019 Paul Thaddeus Kazibudzki. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

An overview of current debates and contemporary research devoted to modeling decision making processes and their facilitation directs attention to techniques based on pairwise judgments. At the core of these techniques are various judgment consistency measures which, in a sense, control the prioritization process which leads to the establishment of decision makers’ unknown preferences. If judgments expressed by decision makers were perfectly consistent (cardinally transitive), all available prioritization techniques would deliver the same solution. However, human judgments are consistently inconsistent, as it were; thus the preference estimation quality significantly varies. The scale of these variations depends, among others, on the chosen consistency measure of pairwise judgments. That is why it seems important to examine relations among various consistency measures and the preferences estimation quality. This research reveals that there are consistency measures whose performance may confuse decision makers with the quality of their ranking outcome. Thus, it introduces a measure which is directly related to the quality of the preferences estimation process. The main problem of the research is studied via Monte Carlo simulations executed in* Wolfram Mathematica Software.* The research results argue that although the performance of examined consistency measures deviates from the exemplary ones in relation to the estimation quality of decision makers preferences, solutions proposed in this paper can significantly improve that quality.

#### 1. Introduction

Overwhelming scientific evidence indicates that the unaided human brain is simply not capable of simultaneous analysis of many different, competing factors and then synthesizing the results for the purpose of making a rational decision. Indeed, numerous psychological experiments, e.g., [1], including the well-known Miller [2] study put forth the notion that humans are not capable of dealing accurately with more than about seven (±2) things at a time (the human brain is limited in its short term memory capacity, its discrimination ability, and its range of perception).

Humans learn about anything by two means: the first involves examining and studying some phenomenon from the perspective of its various properties and then synthesizing findings and drawing conclusions; the second entails studying some phenomenon in relation to other similar phenomena and relating them by making comparisons. The latter method leads directly to the essence of the matter, i.e., judgments regarding a phenomenon. Judgments can be relative or absolute. An absolute judgment is the relation between a single stimulus and some information held in short or long term memory. A relative judgment, on the other hand [3], is defined as identification of some relation between two stimuli both present to the observer. Certainly humans can make much better relative than absolute judgments. Thus, a pairwise comparisons method was proposed in order to facilitate the process of relative judgments.

Some authors proclaim [4, 5] that this method dates back to the beginning of the 20th century and was firstly applied by Thurstone [6]; however, its first scientific applications can be found in Fechner [7]. In reality, the method itself is much older and its idea goes back to Ramon Lull who lived in the end of 13th century. It is a fact that its popularity comes from an influential paper of Marquis de Condorcet [8]; see, e.g., [9, 10], who used this method in the election process where voters rank candidates based on their preference. It has been perfected in many papers, e.g., [4, 11–20].

Also, the method is the core of a decision making methodology—the Analytic Hierarchy Process (AHP), developed by Saaty [21]. Although it is criticized (see [22–25, 25–28]) it is also continuously perfected and as such very often validated [29] and developed from the perspective of its utility (see [5, 30–37]). Thus, considering the stable increase of the AHP application (see, e.g., [38–44]) as well as its steady position among other theories of choice, e.g., [31–37, 45–47], the problematic issues of AHP should be continuously examined and questions which arise thereof should be simultaneously addressed.

The fundamental objective of this research is to determine the answer to the question:* Does the reduction of PCM inconsistency lead to improvement of the priority ratios estimation process quality?*

#### 2. Background

The AHP uses the hierarchical structure of the decision problem, pairwise relative comparisons of the elements in the hierarchy, and a series of redundant judgments which enable measurement of judgment consistency. To make a proposed solution possible, i.e., derive ratio scale priorities on the basis of verbal judgments, a scale is utilized to evaluate the preferences for each pair of items. Probably, the most known scales are Saaty’s numerical scale which comprises integers, and their reciprocals, from one (equivalent to the verbal judgment, “equally preferred”) to nine (equivalent to the verbal judgment, “extremely preferred”), and a geometric scale which usually consists of the numbers computed in accordance with the formula where* c* denotes its parameter which commonly equals 2. Other arbitrarily defined numerical scales are also available, e.g., composed of arbitrary integers from one to* n* and their reciprocals.

The key issue in AHP is priority ranking on the basis of true or approximate weights, i.e., judgments. If the relative weights of a set of activities are known, they can be expressed as a Pairwise Comparison Matrix (PCM): ** A**()=(/)

*, i, j=*1,…,

*n*. PCM in the AHP reflects decision makers’ preferences (their relative judgments) about considered activities (criteria, scenarios, players, alternatives, etc.). On the basis of

**(), it is possible to derive true weights; i.e., decision makers priority ratios , where:**

*A**i*=1

*,…, n*, are selected to be positive and normalized to unity: . For uniformity, is referred hereafter to its normalized form. If the elements of a matrix

**() satisfy the condition**

*A**=1/*for all

*i, j=*1,…,

*n*then matrix

**() is called**

*A**reciprocal*. If the elements of a matrix

**() satisfy the condition = for all**

*A**i, j, k=*1,…,

*n*, and the matrix is

*reciprocal*, then it is called

*consistent*or

*cardinally transitive*.

Certainly, in real life situations when AHP is utilized, there is no ** A**() which would reflect weights given by the vector of priority ratios. As was stated earlier, the human mind is not a reliable measurement device. Assignments, such as “Compare – applying a given ratio scale – your feelings concerning alternative 1 versus alternative 2”, do not produce accurate outcomes. Thus,

**() is not established but only its estimate**

*A***(**

*A**x*) containing intuitive judgments, more or less close to

**() in accordance with experience, skills, specific knowledge, personal taste, and even temporary mood or overall disposition. In such case, consistency property does not hold and the relation between elements of**

*A***(**

*A**x*) and

**() can be expressed as = where is a perturbation factor fluctuating near unity. In the statistical approach,**

*A**e*

_{ij}reflects a realization of a random variable with a given probability distribution.

Besides the prioritization procedure (PP) proposed by the creator of AHP, right principal eigenvector method (REV), there are alternative PPs devised to cope with the priority ratios estimation problem; their demonstrative review can be found, e.g., in [47]. Many of them are optimization based and seek a vector , as a solution of the minimization problem given by the formula subject to some assigned constraints, such as positive coefficients and normalization condition. Because the distance function ** D** measures an interval between matrices

**(**

*A**x*) and

**(), different definitions of the distance function lead to various prioritization concepts and prioritization results. As an example, eighteen PPs in [48] are described and compared for ranking purposes although some authors suggest there are only fifteen that are different. Furthermore, since the publication of the above-mentioned article, a few additional procedures have been introduced to the literature; see, e.g., [43, 49–51]. Probably the most popular alternative to the REV is the Logarithmic Least Squares Method (LLSM) developed by Crawford and Williams [26, 52]. It is given by the following formula:The LLSM solution also has the following closed form and is given by the normalized products of the elements in each row:**

*A*Thus, it is also known as the geometric mean method and it is utilized in this research which strives to improve the reliability of the pairwise comparisons process which is also the core element of AHP.

#### 3. Problem and Research Methodology

There are several PCM consistency measures (PCM-CMs) provided in literature called consistency or inconsistency indices (CIs). The most popular one is the PCM-CM proposed by Saaty [21]. He proposed his CI as determined by the formulawhere* n* indicates the number of alternatives within the particular PCM and denotes its maximal eigenvalue. The significant disadvantage of the PCM-CM is the fact that it can operate exclusively with reciprocal PCMs. In the case of nonreciprocal PCMs, this measure is useless (its values are meaningless) which in consequence seriously diminishes the value of the whole approach; see, e.g., [33]. It was also recently found to be incorrect; see, e.g., [4, 10, 53, 54].

However, as mentioned earlier, there are a number of additional PCM-CMs. Some of them, as in the case of , originate from the PPs devised for the purpose of the priority ratios estimation process. Their distinct feature is the fact that all of them can operate equally efficiently in conditions where reciprocal and nonreciprocal PCMs are accepted. Probably the most known example from that set of propositions is PCM-CM proposed by Aguaron and Moreno-Jimenez [55] given by the following formula:

Noticeably, there are a few definitions of PCM-CMs which are not connected with any PP and are devised on the basis of the PCM consistency definition. Koczkodaj’s idea [56] attracts attention and is the first to be scrutinized. Koczkodaj’s PCM-CM is grounded in his concept of triad consistency.

In order to clarify this, for any three distinguished decision alternatives A_{1}, A_{2}, and A_{3}, there are three meaningful priority ratios, i.e.,* a*_{ij},* a*_{jk}, and* a*_{ik}, which have their different locations in a particular . For some different* i*≤*n, j*≤*n,* and* k*≤*n,* the tuple (*a*_{ij},* a*_{jk},* a*_{ik}) is called a* triad.* If the matrix is consistent, then* a*_{ij}*a*_{jk} =* a*_{ik} for all triads.

In consequence, either of the equations and have to be true. Taking the above into consideration, Koczkodaj proposed his measure for triad inconsistency by the following formula:Following his idea, he then proposed the following PCM-CM of any reciprocal PCM=** A**:where the maximum value for

*K*(

*A*) is taken from the set of all possible triads in the upper triangle of a given PCM.

On the basis of Koczkodaj’s idea of triad inconsistency, Grzybowski [5] presented his PCM-CM determined by the following formula:

Finally, following the idea that , Kazibudzki [57] redefined triad inconsistency and proposed

– Two formulae for its measurement: – One meaningful formula for PCM-CM:

where* x *denotes the formula for triad inconsistency measurement, i.e.,* LTI*_{1} or* LTI*_{2}.

It behooves us to mention that* ALTI*(*A*) can be calculated on the basis of triads from the upper triangle of the given PCM when it is reciprocal or all triads within the given PCM when it is nonreciprocal.

As was already stated earlier, the fundamental question which should be asked by researchers who deal with the problem of priority ratios estimation quality in relation to a PCM consistency measure is as follows:* Does the reduction of PCM inconsistency lead to improvement of the priority ratios estimation process quality?*

The common reason why one strives to improve the consistency of the PCM, when it seems unsatisfactory, is to increase the quality of the priority ratios estimation process. However, the above question remains open and the answer to it is not evident. Even the creator of AHP stated once that improving consistency does not mean getting an answer closer to the “real” life solution [21]. It can be illustrated in the following example.

Considered is the true PV (denoting true weights of examined alternatives), i.e., =/20, 1/4, 1/4, 3/ and ** A**() derived from that PV, which can be presented as follows:

Then two PCMs are considered, i.e., ** R**(

*x*) and

**(**

*A**x*) produced by a hypothetical decision maker (DM). It is assumed that DM is very trustworthy and is able to express judgments very precisely at the same time being still somehow limited by the necessity of expressing judgments on a scale (the example utilizes Saaty’s scale). In the first scenario, entries of

**() are rounded to Saaty’s scale and the entries are made reciprocal (a principal condition for a PCM in the AHP) producingIn the second scenario, only entries of**

*A***() are rounded to Saaty’s scale (nonreciprocal case) producing**

*A*It should be noted that** R**(

*x*) is perfectly consistent and

**(**

*A**x*) is not. Tables 1 and 2 present selected values of the PPs related PCM-CMs (that is, and ) for

**(**

*R**x*) and

**(**

*A**x*) together with PVs derived from

**(**

*R**x*) and

**(**

*A**x*); Mean Absolute Errors (MAEs), formula (14), among (PP) and for the case; Spearman Rank Correlation Coefficients (SRCs) among (PP) and for the case.