Abstract
This paper arose from the perceived need to make a contribution towards assessing a strategic information system by using a new method for eliciting the weights of criteria. This is considered one of the most complex and important stages in multicriteria models. Multicriteria models have been proposed to support decisions in the context of information systems given that problems in this field deal with many conflicting criteria. The new procedure for eliciting the weights of the criteria has the advantage of requiring less effort from the decision-maker and, thus, the risk of inconsistent answers is minimized. Therefore, a model based on this new procedure is proposed and applied using data from a glass packaging factory that needs to select a single information system from a set of systems previously identified as relevant. The results obtained are consistent both with the performance of alternatives and with the additive model used to evaluate the alternatives.
1. Introduction
As observed by Arvidsson et al. [1], information systems (IS) have a strategic role within organizations since they are used to bring about strategic intent. Industrial production managers, service providers, and business administrators invest in IS with a view to obtaining greater efficiency, agility, and security in their operations. Although the market offers a variety of IS which serve different areas within organizations, it is not possible, in most organizations, to invest simultaneously in all the systems that they require. Usually, the resources (whether these have to do with financial resources, time, workforce, infrastructure, and so on) required for these investments are scarce.
Due to the relevance of this subject, academic researchers have devoted special attention to selecting IS, which is one of the three strands within IS strategy research, identified by Chen et al. [2]:(i)IS and business strategy alignment: where models to provide a strategic alignment between IS and business strategy [3] and models to analyze the positive impact of this alignment [4] are developed.(ii)Strategic IS planning to identify portfolios of systems: where the aim is to define a set of systems that jointly contribute to the organization [5, 6]. In some models, depending on the objective and particularities of the organization, the aim is not to define a portfolio but simply to select or to rank the systems [7].(iii)To use a specific system and assess its contribution towards adding to competitive advantage [1, 8].
Regarding the second strand, a large number of intuitive and analytical models have evolved over the last twenty or so years to assist decision-makers (DMs) in evaluating IS projects [9]. According to Zandi and Tavana [9], the main methodologies used for selecting and prioritizing IS projects can be divided into single-criterion cost benefit analysis, multicriteria scoring models and ranking methods, and portfolio methods. A subjective committee is also identified by Chen [10] as a class of methodologies used for IS project selection. The single-criterion cost benefit analysis is illustrated in [11], where Research and Development (R&D) projects are evaluated, and in [12], where R&D projects are selected by using an expected utility approach. The use of multicriteria scoring models and ranking methods for IS project selection is demonstrated in [13], which uses a multiobjective decision model, and in [14], where an analytic network process and goal programming are applied. Currently, the use of multicriteria methods designed to handle portfolio problems has gained special attention within the context of IS. de Almeida and Vetschera [15] propose a method to correct a problem on scale transformations in the PROMETHEE V method, which can be used to support portfolio decision problems. Other papers demonstrate the use of a portfolio method such as [16]. A fuzzy set approach for R&D portfolio selection is proposed by [17].
Initially models that evaluate the financial return on investment in IS were proposed to assist the selection of IS projects. However, the process for doing so requires several conflicting objectives to be analyzed but these cannot be measured monetarily. Thus, multicriteria decision-making/aiding (MCDM/A) methods are gaining in importance due to their inherent ability to judge different alternatives and to rank them.
The aim of this paper is to put forward a contribution in the field of strategic IS selection by using the FITradeoff (Flexible and Interactive Tradeoff) method, proposed by de Almeida et al. [18]. The application presented here was conducted using data from IS investments in a factory of one of the largest manufacturers of glass packaging in the world. These data were also used in an application presented by de Gusmão et al. [19] when the ELECTRE IV method was used by different DMs to rank the alternatives and, at the end of the model, the ranks were aggregated. Here, the alternatives are ranked according to an additive model in MAVT (Multiattribute Value Theory) scope, using the FITradeoff method for eliciting scaling constants or weights of criteria. The results of the two applications cannot be compared since the first application is conducted using a noncompensatory method while the value function of MAVT is compensatory. However, in both cases, the issue of defining the weights of criteria is undertaken (automatically by the method used) so as to minimize the effort that DMs need to make with regard to this.
This paper is organized as follows: Section 2 discusses the importance of multicriteria models in many instances of decision-making and presents an overview of the FITradeoff procedures. The model proposed is briefly presented in Section 3 and applied in Section 4, which discusses details of the steps of the model. The results area is also analyzed in Section 4, thereby allowing the DM to test the robustness of the model. Finally some conclusions and insights that may prompt future research are considered.
2. Background
2.1. Multicriteria Models
When planning investments in IS, not only the financial implications of such investment, but also other objectives such as competitive advantage, market share, and future growth need to be assessed. Thus, MCDM/A methods are gaining in importance since they allow different alternatives to be evaluated and ranked or one subset to be selected [20, 21]. These methods consider several points of view, characterized as conflicting issues, thereby enabling an integrated assessment of the problem in question to be made [22].
According to Vincke [23], MCDM/A can be considered a set of methods developed to support decision problems faced by organizations and individuals. The problems can include selecting suppliers [24], planning maintenance [25, 26], IS selection [20], and evaluating critical technology for generating energy [27].
According to de Almeida et al. [18], defining the weight of criteria, in multicriteria decision, mainly in additive models, is considered a problem either because the DM does not understand the meaning of weight or because the DM does not have sufficient knowledge and information to define the weights. Due to some major difficulties and challenges in procedures for eliciting weights that have been identified and considering that the tradeoff elicitation procedure—the procedure most used in additive models—has a strong axiomatic foundation besides some inconsistencies, de Almeida et al. [18] propose the FITradeoff method.
Unlike traditional procedures, in the flexible elicitation procedure, the DM does not need to provide imprecise or incomplete information a priori. In the flexible elicitation process, there is an attempt to minimize the effort that the DM must make, when compared with considering the traditional procedures. Thus it is expected that less inconsistency occurs during the process [18].
The search for ways to make it easier to determine the parameters required in decision-making, including the weights of the criteria, is not something new. The ELECTRE TRI Assistant method, proposed by Mousseau et al. [28], for example, requires from the DM much less cognitive effort. In this case, the parameters are defined indirectly using holistic information given by the DM through assignment examples, which are alternatives assigned by the DM to categories according to his/her comprehensive preferences. The use of assignment examples makes sense since the ELECTRE TRI is a multicriteria model whose goal is to assign each alternative to one of the categories which are predefined. Also, Mousseau and Dias [29] propose a slight adaptation of the valued outranking relation used in ELECTRE III and ELECTRE TRI. Although the modified outranking relation keeps the complexity of inferring the weights and cutting level, the veto thresholds are inferred easily.
The last two examples refer to methods of outranking. Regarding additive models, Melo Brito et al. [30], for example, present the application of a multicriteria methodology to support the selection of repair contracts in a context where information is imprecise, that is, when it is not possible to assign precise values to importance parameters of the criteria used for contract selection, but unlike the FITradeoff there is no elicitation procedure. A decision support system (DSS) was proposed by de Almeida et al. [31] to support a DM to establish the weights of criteria in a multicriteria decision problem by using a flexible elicitation procedure.
2.2. Overview of the FITradeoff Procedure
The usual notations adopted in elicitation of weights processes are used [32, 33]:(i) represents the vector of consequences of an alternative, considering all criteria.(ii) represents the weight for the criteria .(iii) represents the value function of the consequences for the criteria.Thus, according to Keeney and Raiffa [32] and Keeney [34], aggregates the value functions asAssuming that it is well known that, in additive models, the most appropriate denomination for is as a scaling constant and not as a weight, but, as in [18], in this paper the term weight will be used for the sake of simplification.
The procedure for applying the FITradeoff, like that for the traditional model, is divided into two parts:(i)Obtaining the orders of the weights , using the preference .(ii)Obtaining the values of , using the indifference relation .The first part allows an -dimension weight space () to be defined which is given byIn this notation, and from now on, it was assumed, as assumed in de Almeida et al. [18], that the criteria are ordered from the most relevant to the least relevant.
The second part is now begun and it is in this part that the difference between the procedure for the traditional model and the FITradeoff is seen. In the FITradeoff, it is not necessary for the DM to define an exact value (), which would denote the outcome of criterion for which the indifference is obtained between consequences, whereas the traditional method requires this. In the FITradeoff, this procedure requires the DM to specify a range from to that represents, respectively, the upper and lower limit that can assume. Thus, given any criterion , the following relations can be established:Thus, as shown in [18], as a result of the second part, a new weight space ( may be obtained, which is a subspace of (3), in which all the valid relations of type (4) are considered:For more details on the definition of , see Keeney and Raiffa [32] and Keeney [34].
The FITradeoff is operationalized by a DSS, which includes the following stages [18]:(1)Evaluating the intracriteria.(2)Ranking the weights of the criteria.(3)Attempting to solve the problem using the available set of weights.(4)Evaluating the DM’s preferences.At the end of Stage , a check is made on whether or not a unique solution has been obtained, that is, if an optimal alternative has been identified. The DSS classifies the alternatives in three situations: potentially optimal, dominated, or optimal. If a single solution is not found, the DM goes on to the next stage, namely, that of evaluating the DM’s preferences which can be divided into four steps:(4.1)Setting values for testing the distribution of weights.(4.2)Asking the DM to state his/her preferences.(4.3)Computing LPP.(4.4)Finalization.These four steps constitute the main stage of the FITradeoff [18]. The aim of using the heuristic presented in Stage (4.1) is to compute the value of , thereby minimizing the number of questions to the DM. The output of this step, and the input for Stage (4.2), is a new set of values for and . Based on this new set of values, the DM, in Stage (4.2), has three options: to see partial results; not to proceed in the system; or to proceed with a view to making a choice. This choice consists of defining if there is a preference or an indifference relation between two consequences. If there is a preference relation, the DM has to signal the preferred consequence. Depending on the consequence chosen, either or assumes the value. In the case of indifference, assumes the value.
Having made this choice, Stage (4.3) is started and a Linear Program Problem (LPP) model is run [18]. It is important to note that this LPP model is also applied in Step assuming and = 0 for all criteria . This LPP has the following objective function:Thus, the aim is to find an alternative , from the set of alternatives, that has the maximum value given in (1) in accordance with the weight of criteria space given by (5). Therefore, it is necessary to consider some constraints in the LPP. The relations (4) are introduced as constraints on how strict inequality is avoided.
Also, what should be considered to solve the problem (the optimal alternative) is that the maximum value of the alternative should be greater than (or equal to) any other alternative in the subset. Thus, the following constraint has to be considered:This LPP runs until an optimal alternative is found. If this does not happen, the dominated alternatives are eliminated and the process is started again, from Step (4.1). Now, just the alternatives identified as potentially optimal are considered in the subsequent steps.
3. Model Proposed
As already explained, the aim of this paper is to put forward a contribution in the field of strategic IS selection by using FITradeoff and based on the data presented in [19]. Thus, based on the framework presented in [19] and on the general procedure of the FITradeoff, the model proposed can be structured as shown in Figure 1.
First of all, the IS that are identified as relevant for the organization and also the criteria that will be used to evaluate the IS must be defined. This step can be supported, as proposed in [19], by some methods, for example, using the BSC (Balanced Score Card), as a result of which, from the strategic viewpoint of the organization, objectives and also some criteria can be identified. The identification of strategic criteria is particularly important when decisions are being made about IS. The model allows working with either continuous or discrete criteria.
Thereafter, the intracriteria are evaluated. Here, it is important to emphasize that, in the FITradeoff, consequences may be dealt with monotonically by either increasing or decreasing them [18].
As explained in the previous section, the FITradeoff is divided into two parts, the ultimate goal of which is to determine the values of , for which the global evaluation of the alternatives is maximized. Using the language of the DSS constructed by de Almeida et al. [18] to operationalize the FITradeoff, the aim is to find an alternative (IS) from the set of alternatives that has the maximum value given in (1) and considering the weight space given by (5). Since the DSS conducts the elicitation process in a flexible way, only essential information is required from the DM.
In the following section, the proposed model is applied in the context of IS selection and the steps used are discussed in greater detail.
4. An Application of IS Selection Using FITradeoff
The data from a factory of one of the largest manufacturers of glass packaging in the world were used to validate the model proposed. These data were used before when applying ELECTRE IV in a decision group problem presented in [19]. The biggest difference between the two models resides in the method used to evaluate the IS. In the model proposed by [19], ELECTRE IV was used to prioritize IS, so the application was conducted as a ranking problem. However, in the model proposed in this paper, an additive mode, MAVT, is used, that, by also using FITradeoff, enables the best IS according to the criteria considered to be selected. So, in this paper, the problem is regarded as a multicriteria problem about choice. To do so, the value judgment of a single DM, expert in the field, is taken into consideration. In addition, this application enables the contribution of the model in the field of IS selection to be assessed. Moreover, it is also important to evaluate this kind of problem from different perspectives as it stands to reason that these methods (ELECTRE IV and MAVT) have specific assumptions that need to be validated before they are used and also to bear in mind that their results cannot be compared.
The organization identified and analyzed, in the process of managing its IS, some IS projects but, due to lack of resources, cannot invest in them all. Therefore, it needs to select which one to invest in, first. Considering the strategic importance of the IS for the organization, they should be assessed in the light of strategic criteria, which could be obtained directly from the DM or by a method such as the BSC.
In 2011, four IS projects were identified as relevant for the manufacturer, namely,(i)CSS: the aim of this IS is to store and manage customer’s complaints in a database, thus allowing the main problems to be classified, solutions to be prioritized, and responses to clients to be generated;(ii)CRS: it serves to create routes using digital maps that lead to delivery times and the cost of fuel and spare parts being reduced and to deploying software so that loads can be assembled quickly;(iii)SAP: its aim is to complete the implementation of SA, thus ensuring greater agility and consistency in providing monthly reports to the Executive Board;(iv)EL: this is needed to develop employees and to empower and train new leaders.The criteria, the same as those used in [19], which were defined using the BSC method, are cost reduction (CR), improving functional qualifications (IFQ), improving customer relationships (ICR), improving the quality of products and services (IQPS), and improving the efficiency and control of processes (IECP).
According to the criterion CR, the alternatives are assessed considering their contribution to reducing the company’s total costs, including workforce costs, transportation costs, and time savings. The aim of criterion IFQ is to assess the contribution of the system to improving functional qualifications. The contribution to improving the quality of products and services offered by the company is evaluated by using the third criterion. In the same way, the criterion IECP enables an evaluation to be made of how systems influence the efficiency gain. All the criteria are to be maximized.
In the second step of the proposed model, in which the intracriteria are evaluated, the four alternatives are assessed for all criteria using a 5-point Likert scale. In order to obtain initial references for the criteria weights, the traditional tradeoff procedure was applied. The order obtained was . Thus, the first three steps of the DSS were run and three alternatives were identified as nondominated alternatives (CCS, CRS, and EL). This result means that more than one solution was found, assuming and for all criteria in the LPP model.
Then Step (4.1) is started and the DM was requested to supply information. In Step (4.2), two alternatives (A and B) were presented to the DM. The criterion “cost reduction” has the greater weight of the two criteria being analyzed, is given the value indicated for , and is assumed by Alternative A. Alternative B represents the best consequence for the criterion “efficiency and control of processes,” the weight of which is less than that of the criterion “cost reduction.” If the DM prefers Alternative A, assumes value. If B is chosen, assumes value. Also, if the DM is indifferent to the alternatives, then assumes value. Thereafter, the LPP model is run until it presents the same three nondominated alternatives.
Figure 2 shows the result of the LPP model from the first cycle.
Finally, after seven cycles, the CRS system is identified as the unique solution. The partial results from the DSS are presented in Table 1.
For each output of the LPP model, a bar graph is provided. For example, the bar graph presented after the sixth cycle is illustrated in Figure 3.
To exemplify how the consequences are presented to the DM in each cycle, Figure 4 illustrates the question that the DSS runs in the seventh cycle.
The great advantage of this procedure is its flexibility with regard to analyzing the alternatives. It does so by producing bar graphs during the process. These display the weights of the criteria which are identified according to the nondominated alternatives. The DM’s answer in the seventh cycle, for example, was based on information from Figure 4.
The DSS of FITradeoff was applied using the software available, on request, at http://fitradeoff.org/.
4.1. Analyses of the Results
Alternative CRS has the best performance for criteria CR and IECP, which is highlighted by the orange bars in the graph of Figure 3, and alternative CCS has the best performance for the criteria ICR and IQPS, which is highlighted by the blue bars in the graph. Thus these alternatives persist for several cycles as nondominated alternatives. In fact these alternatives have a balanced global performance, which makes them preferred in an additive model such as MAVT. In [19], the application made using ELECTRE IV presented SAP as the alternative to be prioritized. The difference between these results was expected not only due to the fact that the alternatives that present a balanced performance are prioritized in additive models, but also because the application presented in [19] was conducted by a decision-making group while the application presented here takes into account the value of judgment of a single DM.
Also, although the problem was not solved in the first three steps and the subsequent steps needed to be undertaken, it was nevertheless possible to reach the best alternative setting in a few cycles. This corroborates that the goal of the method had been achieved, which is to minimize the effort that the DM requires to make.
5. Conclusions
Investments in IS are strategic for many organizations. Thus decision in this area requires the evaluation of both quantitative and qualitative criteria. Therefore, the use of multicriteria decision models was proposed to support the DM or a group of DMs in selecting IS that have the potential to contribute most to an organization’s business.
In this paper a model, based on the FITradeoff method, is proposed. It supports defining which is the best IS to invest in, from a set of alternatives and a set of strategic criteria. The model is validated using data from a glass packaging factory, which identified some IS relevant in 2011. However, due to the lack of resources, it needed to identify the most critical IS.
For the application, the DSS of the FITradeoff is used. This DSS interacts easily and flexibly with the DM. The bar graphs provided by the software also enable the DM to analyze the performance of the alternatives and weights in each cycle.
The results obtained are consistent with input data and the range of the criteria weights obtained initially with the traditional tradeoff procedure. The effort required and the time spent on this by the DM are consistent with the objectives targeted by the method.
Thus, the FITradeoff consists of a more sophisticated additive model that uses a more sophisticated approach to conduct an evaluation that uses multicriteria, as in the case of IS assessment. The need to compile portfolios from the alternatives, in this kind of problem, may well prompt future research into how to improve this method.
Competing Interests
The authors declare that they have no competing interests.
Acknowledgments
The authors would like to acknowledge the Center for Decision Systems and Information Development (CDSID) for having provided the software.