Using the Fuzzy Linguistic Preference Relation Approach for Assessing the Importance of Risk Factors in a Software Development Project

Abstract

This study employs fuzzy linguistic preference relation (Fuzzy LinPreRa) approach to assess the relative degree of impact of risk factors in software development project for two expert groups working in technology enterprises and software development companies. For the identified risk dimensions, the results show the same rankings for these two groups. “Organization function risk” is considered the most important dimension influencing the software development project performance, with the others, in order, being “developing technology risk,” “resources integration risk,” “personnel system risk” and “system requirement risk.” The proposed approach not only facilitates the information collecting for making pairwise comparisons, but it also eliminates the inconsistencies in the collected information.

1. Introduction

Software development projects are usually designed by highly skilled and trained experts due to projects’ complicate, professional, and technicality nature. However, these types of projects often have series of setbacks such as high cost, schedule overruns, quality issues, and usability problems [1, 2]. From the past experience, most software development projects use more resources than originally planned, take longer time to complete, and unfortunately, often deliver poor functionality and low quality than expected [3]. It is obvious that the life cycle of software development process is inherent numerous uncertainties and risk factors. Therefore, there are some researchers that focused on the study of risk evaluation, risk assessment, and risk management of software development.

Since there are numerous risks in software development, the proposed assessment procedure has designed using multiple criteria decision making (MCDM) approach. Besides, associated risk evaluation is often determined by experts subjectively; however, it is not easy for the experts to achieve precise judgments with limited time. Therefore, to solve the problem of incorporating more assessment factors and working within time limitations for interviews with experts, this study applies the fuzzy linguistic preference relation (Fuzzy LinPreRa) proposed by Wang and Chen [5]. This method is based on the consistent fuzzy preference relations (CFPRs) approach which was proposed by Herrera-Viedma et al. [6] to deal with the relative degree of impact of software development risk factors. The results can provide decision makers with the ability to propose risk management strategies in advance of implementing a software development project.

2. Risk Factors for a Software Development Project

Many uncertain problems could confront whether done by oneself or by outsourcing groups when the software development projects are performed. In order to reduce failure rate of software development, software engineers and information system researchers should always identify as many risk factors affecting project outcome as they can. In the literatures review, ten risk items for software development projects were identified by Boehm [7], which include personnel shortfalls, unrealistic schedules and budgets, developing the wrong functions and properties, developing the wrong user interface, adding more functionality/features, continual requirement changes, shortfalls in externally furnished components, shortfalls in externally performed tasks, real-time performance shortfalls, and straining of computer-science capabilities. A hierarchical structure model for evaluating software development risk was constructed via fuzzy set theory by Lee and Lin [8]. They proposed six risk attributes in order to evaluate the aggregative risk: personnel, system requirements, schedules and budgets, developing technology, external resource, and performance. The fourteen risk items can be categorized into six attributes.

The twelve risk categories identified by Jiang and Klein [9] include technological acquisition, project size, lack of general team expertise, lack of team’s expertise with the task, lack of team’s development expertise, lack of user support, intensity of conflicts, extent of changes brought, resources insufficient, lack of clarity of role definitions, application complexity, and lack of user experience. In addition, Houston et al. [10] listed thirty software development risk factors and further studied the effects of six common and significant risk factors. They are creeping user requirements, lack of staff commitment, low morale, instability and lack of continuity in project staffing, inaccurate cost estimation, excessive schedule pressure, and lack of senior management commitment. Büyüközkan and Ruan [11] applied the Choquet integral aggregation approach to analyze the effects of importance and interactions among software development risks. They extracted software development risk factors by surveying a large amount of literature, and they explored several types of risks including product engineering risks, development environment risks, and program constraint-related risks such as the main risk dimensions. Factors associated with product engineering risks include requirements, design, code and unit tests, integration and testing, engineering specialists. Factors in development environment risks include development process, development system, management process, management methods, and work environment. Resource, contract, and program interfaces are those factors that influence the program constraint risks. Lu and Ma [12] identified the risk factors according to the life cycle of a software development project for owners and contractors. For owners, the causes of risk are distinguished into conceive stage, development stage, execution stage, and finish stage. In the conceive stage and development stage there are eight risk factors, respectively. Six risk factors belong to execution stage, but in finish stage, the main risk factor is that there is no standard and criterion of the information system. For contractors, risk factors in software development include four aspects: policy risk, technique risk, scope change risk, and management risk. The management risk consists of cost management risk, quality management risk, schedule management risk, integration management risk, human resources management risk, and communication management risk. Lu and Yu [4] constructed a hierarchical structure of five dimensions and associated twenty-two risk factors for software development project and presented two approaches to assess the importance and the ranking of these risk factors by fuzzy set theory. The five risk dimensions include organization function risk, developing technology risk, personnel system risk, resources integration risk, and system requirement risk.

From these literatures review, the software development risk factors were further screened and synthesized for this study. We can adopt the hierarchical structure of risk factors developed by [4] for applying the proposed Fuzzy LinPreRa approach conveniently. Figure 1 shows the hierarchical structure of risk factors for a software development project.

Figure 1 

Hierarchical structure of risk factors for a software development project [4].

3. Method for Assessing the Degree of Impact of Risk Factors

This study not only would like to realize the risk factors in the process of software development, but is also considers which of these factors are important for the project managers. It is impractical to assume that the different project risk factors equally affect project success. Therefore, the impact degree of the risk factors on project success should be carefully evaluated for better managing of the project risk and increasing the chance of project success. That is, the varying effects of project risk factors on project success provide valuable information needed to allocate software development project resources. This study would like to assess the relative impact degree among software development risk factors for helping developers to draw up the risk management strategies. However, the largest possible number of evaluation criteria of a dimension is five. If we apply the conventional analytic hierarchical process (AHP) proposed by Saaty [13], or fuzzy AHP proposed by Buckley [14], to design the questionnaire, we may encounter great difficulties and challenges in collecting information. In addition, the model also needs to consider the fuzziness of the judgments or opinions of selected experts when they answer the questionnaires. Thus, this study applies the fuzzy linguistic preference relation (Fuzzy LinPreRa) approach for constructing the decision matrices of pairwise comparisons. The FLPR approach was constructed by Wang and Chen [5] based on the consistent fuzzy preference relations (CFPRs) proposed by Herrera-Viedma et al. [6]. The Fuzzy LinPreRa not only makes it easy for interviewers to use linguistic variables to present a set of criteria with the least amount of subjective judgment, but it also avoids the necessity to check for consistency in the decision-making process. More importantly, it is more convenient to acquire the judgments of practitioners or experts of software development industry through a questionnaire. A brief introduction of the definitions and steps in the adopted Fuzzy LinPreRa method is given.

3.1. Consistent Fuzzy Preference Relations (CFPRs)

The fuzzy preference relation 𝑃 on a set of evaluation criteria 𝑋 is a fuzzy set of the product 𝑋×𝑋 with a membership function 𝑃∶𝑋×𝑋→[0,1]. The preference relation is represented by the 𝑛×𝑛 matrix 𝐏=[𝑝_𝑖𝑗], where 𝑝_𝑖𝑗=𝑃(𝑥_𝑖,𝑥_𝑗)  for  all  𝑖,𝑗∈{1,…,𝑛}. Herein, 𝑝_𝑖𝑗  is interpreted as the degree of importance of criteria 𝑥_𝑖  over  𝑥_𝑗. If  𝑝_𝑖𝑗=0.5, it means that  𝑥_𝑖  and  𝑥_𝑗  are equally important; 𝑝_𝑖𝑗=1 indicates that 𝑥_𝑖 is absolutely important to  𝑥_𝑗; and  𝑝_𝑖𝑗>0.5  shows that  𝑥_𝑖  is more important than  𝑥_𝑗. In this case, the preference matrix, 𝐏, is usually assumed to be additive reciprocal; that is, 𝑝_𝑖𝑗+𝑝_𝑗𝑖=1,  for  all  𝑖,𝑗∈{1,…,𝑛}. When the reciprocal fuzzy preference relation 𝐏=[𝑝_𝑖𝑗]   is consistent and verifies additive consistency, there exists a relationship equation such that [6] 𝑝_𝑖(𝑖+1)+𝑝_{(𝑖+1)(𝑖+2)}+⋯+𝑝_{(𝑗−1)𝑗}+𝑝_𝑗𝑖=𝑗−𝑖+12,∀𝑖<𝑗. (1)

Equation (1) is very important because it can be used to construct a consistent fuzzy preference relation from the set of 𝑛−1  values  {𝑝₁₂,𝑝₂₃,…,𝑝_{(𝑛−1)𝑛}}. If a decision matrix with entries in the interval [−𝑎,1+𝑎],  𝑎>0 is outside the interval  [0,1], it can be constructed by transforming the obtained values using a transformation function that preserves reciprocity and additive consistency. The transforming function is 𝑓∶[−𝑎,1+𝑎]→[0,1],  𝑓(𝑥)=(𝑥+𝑎)/(1+2𝑎). In such a way, we can facilitate the expression by the evaluator of consistent preferences in the decision process.

3.2. Linguistic Variables and Fuzzy Numbers

A linguistic variable is the one whose values are words or sentences expressed in a natural or artificial language. Here, we use this kind of expression to compare the impact degree of two risk dimensions or factors for a software development project using linguistic terms: “absolutely (not) important,” “very strongly (not) important,” “essentially (not) important,” “weakly (not) important,” “equally important,” with respect to a triangle fuzzy number (TFN) proposed by [15]. A TFN is a fuzzy number 𝑃 on ℝ if its membership function 𝜇𝑃(𝑥)∶ℝ→[0,1] is equal to (2), following the definition in [14] 𝜇𝑃(𝑥)=(𝑥−𝑙)()()𝑚−𝑙,𝑙≤𝑥≤𝑚,𝑢−𝑥(𝑢−𝑚),𝑚≤𝑥≤𝑢,0,otherwise, (2) where  𝑙 and 𝑢 stand for the lower and upper bounds of the fuzzy number 𝑃, respectively, and 𝑚 stands for the median value. The TFN can be denoted by 𝑃=(𝑙,𝑚,𝑢), and Table 1 shows the corresponding TFN for each linguistic assessment variable. The table of converting linguistic terms to fuzzy numbers is constructed from [15]. Detailed conversion scales can contain and match all the verbal terms given by the evaluators, and it is very important for the evaluators to be familiar with the decision problem context.

Table 1 

Fuzzy linguistic assessment variables.

3.3. Fuzzy Linguistic Preference Relations (Fuzzy LinPreRa)

Given a set of criteria, 𝑋={𝑥₁,…,𝑥_𝑛} is associated with the fuzzy linguistic preference relations matrix 𝐏=[𝑝_𝑖𝑗]=(𝑝^𝑙_𝑖𝑗,𝑝^𝑚_𝑖𝑗,𝑝^𝑢_𝑖𝑗) based on consistent fuzzy preference relations and fuzzy linguistic assessment variables. If the matrix above complies with additive reciprocal consistency, then the following statements must be equivalent: 𝑝^𝑙_𝑖𝑗+𝑝^𝑢_𝑗𝑖{},𝑝=1∀𝑖,𝑗∈1,…,𝑛_𝑚𝑖𝑗+𝑝^𝑚_𝑖𝑗{},𝑝=1∀𝑖,𝑗∈1,…,𝑛_𝑢𝑖𝑗+𝑝^𝑙_𝑖𝑗{},𝑝=1∀𝑖,𝑗∈1,…,𝑛_𝑙𝑖𝑗+𝑝^𝑙_𝑗𝑘+𝑝^𝑢_𝑘𝑖{},𝑝=1.5∀𝑖,𝑗∈1,…,𝑛_𝑚𝑖𝑗+𝑝^𝑚_𝑗𝑘+𝑝^𝑚_𝑘𝑖𝑝=1.5∀𝑖<𝑗<𝑘,_𝑢𝑖𝑗+𝑝^𝑢_𝑗𝑘+𝑝^𝑙_𝑘𝑖𝑝=1.5∀𝑖<𝑗<𝑘,_{𝑙𝑖(𝑖+1)}+𝑝^𝑙_{(𝑖+1)(𝑖+2)}+⋯+𝑝^𝑙_𝑗𝑗−1+𝑝^𝑢_𝑗𝑖=𝑗−𝑖+12𝑝∀𝑖<𝑗,_{𝑚𝑖(𝑖+1)}+𝑝^𝑚_{(𝑖+1)(𝑖+2)}+⋯+𝑝^𝑚_𝑗𝑗−1+𝑝^𝑚_𝑗𝑖=𝑗−𝑖+12𝑝∀𝑖<𝑗,_{𝑢𝑖(𝑖+1)}+𝑝^𝑢_{(𝑖+1)(𝑖+2)}+⋯+𝑝^𝑢_𝑗𝑗−1+𝑝^𝑙_𝑗𝑖=𝑗−𝑖+12∀𝑖<𝑗. (3)

Notably, if the values of the obtained matrix 𝐏 with elements in the interval [−𝑐,1+𝑐](𝑐>0) are not in the interval  [0,1], the obtained fuzzy numbers will need to be transformed by way of a transformation function to preserve the reciprocity and additive consistency. The transformation functions for left, median, and upper values of TFN in every element of decision matrix  𝐏  are given in (4). Finally, we can obtain a consistent additive reciprocal decision matrix  𝐏^′𝑓𝑥^𝑙=𝑥^𝑙+𝑐(),𝑓𝑥1+2𝑐_𝑚=𝑥^𝑚+𝑐(),𝑓𝑥1+2𝑐_𝑢=𝑥^𝑢−𝑐().1+2𝑐 (4)

3.4. Determination of Criteria Weights

If there are 𝑚 evaluators to participate in the judgments, this study uses the notion of the average value to integrate the fuzzy judgment values of 𝑚 evaluators from the set of 𝑛−1 values  {𝑝₁₂,𝑝₂₃,…,𝑝_{(𝑛−1)𝑛}}; that is, 𝑝_𝑖𝑗=1𝑚⊗𝑝¹_𝑖𝑗⊕𝑝²_𝑖𝑗⊕⋯⊕𝑝^𝑚_𝑖𝑗. (5)

Here, we can establish an original synthetic fuzzy judgment matrix  𝐏 and then use (3) and (4) to obtain the consistent synthetic fuzzy judgment matrix 𝐏^′. In order to obtain the weights of each criterion, this study adopts column vector average approach proposed by Saaty and Vargas [16] as follows: 𝑟̃_𝑖𝑗=𝑝_𝑖𝑗𝑝_1𝑗⊕𝑝_2𝑗⊕⋯⊕𝑝_𝑛𝑗,𝑤_𝑖=1𝑛𝑟̃_𝑖1⊕𝑟̃_𝑖2⊕⋯⊕𝑟̃_𝑖𝑛=𝑤^𝑙_𝑖,𝑤^𝑚_𝑖,𝑤^𝑢_𝑖. (6)

The result of the fuzzy synthetic decision reached by each criterion is a fuzzy number, (𝑤^𝑙_𝑖,𝑤^𝑚_𝑖,𝑤^𝑢_𝑖). Therefore, it is necessary that a defuzzification method for fuzzy numbers is used. Methods for such defuzzified weighting generally include the mean of maximal (MOM), center of area (COA), and α-cut methods [17, 18]. This study utilizes the COA method to find that the nonfuzzy value is a simple and practical method. The defuzzified weights of the fuzzy number can be found by the following equation: 𝑤_𝑖=𝑤^𝑙_𝑖+𝑤^𝑚_𝑖+𝑤^𝑢_𝑖3∀𝑖. (7)

4. Case Study

Two companies were invited to answer the questionnaire related to the risk of a software development project. The first group is comprised of five experts with 5 to 15 years’ experience in the information management division from a medium scale technology enterprise. The second group is also comprised of five experts with 5 to 10 years’ experience in establishing enterprise resource planning (ERP) systems for a software development company. The individuals in the two groups were invited to provide responses to the questionnaire survey and were asked to compare the relative degree of impact for the risk dimensions and factors identified in this study in pairwise sequential order. That is, they only needed to compare the relative impact degree from 𝐹₁ to 𝐹₂,  from  𝐹₂ to 𝐹₃,  from  𝐹₃ to 𝐹₄,  from  𝐹₄ to 𝐹₅,  from  𝐹₁₁ to 𝐹₁₂,  from  𝐹₁₂ to 𝐹₁₃, and so on. Tables 2 and 3 list the original evaluation results of this pairwise comparison for the risk dimensions given by the five experts of the two groups. Similarly, Tables 4 and 5 display the original pairwise comparison matrix for the twenty-two risk factors given by the two groups. Since the preferences and experiences of these experts are different, we use the fuzzy linguistic assessment variables, as in Table 1 and from (3) to (5), to aggregate the experts’ subjective judgments toward the impact of the degree of risk dimensions and factors, yielding the synthesized triangular fuzzy numbers listed in Tables 6, 7, 8, and 9, respectively. The ranking of the impact degree is determined by constructing a crisp value from the fuzzy number. Hence, defuzzification needs to be performed to arrange the fuzzy numbers for ranking. Table 10 lists the impact degree and ranking of risk dimensions and factors for software development projects obtained by using (6) and (7).

Table 2 

Original evaluation results for the risk dimensions by the first group.

Table 3 

Original evaluation results for the risk dimensions by the second group.

Table 4 

Original evaluation results for the risk factors by the first group.

Table 5 

Original evaluation results for the risk factors by the second group.

Table 6 

Aggregation decision matrix of the risk dimensions form the first group.

Table 7 

Aggregation decision matrix of the risk factors from the first group.

Table 8 

Aggregation decision matrix of the risk dimensions from the second group.

Table 9 

Aggregation decision matrix of the risk factors from the second group.

Table 10 

Impact degree and ranking of risk dimensions and factors for the two groups.

For experts in the first group, who are staff members of a technology enterprise, among the identified five dimensions, “organization function risk” was found to be the most important risk dimension to influence software development performance and success. “Developing technology risk” and “resources integration risk” are respectively the second and third highest impact dimensions affecting software development project performance. “Personnel system risk” and “system requirement risk” are the last two dimensions in sequence in relative importance affecting performance and success of the software development project.

For each dimension, among the first risk dimensions (𝐹₁), incapable organization management (𝐹₁₁) was considered the most impact risk factor, and it is also the most important factor for the global aspect. Among the second, fourth, and fifth dimensions (i.e., 𝐹₂, 𝐹₄, and  𝐹₅), immature technology development (𝐹₂₁), improper coordination with related divisions (𝐹₄₂), and continuing stream of requirements changes (𝐹₅₁) were all found to be the highest impact risk factors influencing the software development project success. Unavailability of key staff or project managers (𝐹₃₂) and lack of staff commitment and low morale (𝐹₃₄), respectively, are the most observed prominent risk factors which harm software development project success in each dimension.

The points of view in relation to the risk dimensions expressed by the experts of the second group, who are staff members of a software development company, are the same as the first group. For each dimension, the ranking results of the impact degree for risk factors are somewhat different from the first group. For example, among the first risk dimension (𝐹₁), incapable organization management (𝐹₁₁) is also considered as the most impacting risk factor in the first group, but it is the factor having the second greatest impact for global aspects, which is not the same as the first group. The most difference in ranking for factors of dimensions between these two groups was for the third risk dimension (𝐹₃). The second group considers reliance on a few key personnel (𝐹₃₁) as the most important factor in the “personnel system risk” aspect, but it has the least impact based on the evaluations of the first group.

According to the results of this case study, whether it is technology enterprise (demander) or software development company (supplier), they should realize that organization and communication management is the upmost restrict condition for the success of software development project; that is, organizational culture, organizational structure, organizational processes, organization’s established communications channels, and so forth. The internal and external enterprise environmental factors will critically influence the performance of the software development project. It indicates that the application of appropriate project management skills and knowledge is necessary.

5. Conclusions

There is an old saying that goes “An ounce of prevention is worth a pound of cure.” Therefore, if we can understand the encountered risks and pay more attention in advance, it probably can mitigate loss for the results. In this study, we adopt the five risk dimensions, and associated twenty-two risk factors were developed by [4]. This can help the project managers to have an overview of the global picture of risk. Then, we apply the Fuzzy LinPreRa method which can simplify the Fuzzy AHP method and also avoid the necessity of checking the consistency in the decision-making process to assess project risk for software development. Because in the Fuzzy AHP, if there are 𝑛 criteria/attributes, it needs 𝑛(𝑛−1)/2  pairwise comparisons, but the Fuzzy LinPreRa just requires 𝑛−1  comparisons and maintains the decision matrices consistency. The advantage of applying Fuzzy LinPreRa approach compared with Fuzzy AHP is that the information demand has been significantly reduced, especially in obtaining information from practitioners. That is, it is more convenient to acquire the judgments of practitioners or experts of software development industry through a questionnaire.

The evaluation results of the experts in both groups implied that the impact degrees of the risk dimensions are the same whatever the rule on the software development project. Although there are some differences in the ranking of risk factors for each risk dimension, the results should prove to be a valuable reference for software developers. According to the results of the model built, it can be seen that the proposed model can benefit the stakeholders of software development projects and help them recognize what risk factors they may encounter and to facilitate drawing up project risk management strategies for the adequate allocation of resources.