A Hybrid Z-Based MADM Model for the Evaluation of Urban Resilience

Huang, Chun-Nen; Lo, Huai-Wei

doi:https://doi.org/10.1155/2021/9474753

Mathematical Problems in Engineering

On this page

Abstract Introduction Preliminaries Discussion and Conclusions Data Availability Conflicts of Interest Authors’ Contributions References Copyright Related Articles

Special Issue

Fuzzy Applications in Engineering and Risk Management

View this Special Issue

Research Article | Open Access

Volume 2021 | Article ID 9474753 | https://doi.org/10.1155/2021/9474753

A Hybrid Z-Based MADM Model for the Evaluation of Urban Resilience

Chun-Nen Huang¹and Huai-Wei Lo²

Academic Editor: Lazim Abdullah

Received10 Aug 2021

Revised13 Sept 2021

Accepted15 Sept 2021

Published29 Sept 2021

Abstract

Natural disasters and man-made incidents have many negative impacts on major cities, including casualties, economic losses, disruption of social order, and environmental contamination. Cities need to be resilient in order to protect people’s lives and property. Although research on urban resilience has been rapidly emerging in recent years, there are still some research gaps. The interplay of attributes for assessing urban resilience has not been explored, and the Multiple Attribute Decision Making- (MADM-) based framework for evaluating urban resilience is rarely studied. Therefore, this study proposes a novel model to evaluate urban resilience, the Z number-based Decision-Making Trial, and Evaluation Laboratory (Z-DEMATEL), to identify the mutual influential relationships and the weights of the attributes. In addition, the Z number-based Reference Ideal Method (Z-RIM) is used to determine the resilience capacity of cities and to suggest improvements for decision makers to develop appropriate strategies. In this study, we not only use trapezoidal fuzzy numbers to reflect the uncertainty of information but also measure the reliability/confidence of experts in the assessment. The integrated methodology is presented for the first time in this study, and we use the firmness data of six major metropolitan cities in Taiwan as an example of model demonstration. The results of the study show that population density, value of business activity, healthcare facilities, electricity supply, and number of business registrations are the most important attributes influencing the resilience of cities. Taipei City and Taichung City are the two major cities with better resilience in Taiwan based on the analysis of this study.

1. Introduction

Many disasters caused by extreme weather and human factors negatively affect human life, and this situation is becoming more and more frequent and serious. Cities need to not only build a defensive system that can withstand disasters, but also protect the lives and property of their residents. Cities need to be sufficiently defensible, resilient, and able to adapt to numerous disturbances [1]. In recent years, research on urban resilience has been rapidly emerging, and the number of topics related to the terms “urban resilience,” “resilient city,” and “resilient cities” is growing every year [2]. The definition of “resilience” has always been a controversial issue, leading to conceptual and definitional differences in many studies. How to measure “resilience” is of interest to researchers [2–4], and Meerow et al. [5] provide a clear definition of urban resilience through an extensive literature review. Urban resilience is the ability of an urban system and all its constituent organizations (including cross-functional departments, societies, and technologies) to maintain or rapidly recover required functions in the face of turbulence, to adapt to change, and to rapidly transform the system’s current limits or future adaptive capacity [5, 6]. Ribeiro and Pena Jardim Gonçalves [7] argue that enhancing the adaptive capacity of cities is one of the most important needs of urban communities, especially those areas that are often at risk. Although natural or man-made disasters are always tragic, postdisaster recovery can provide a rare opportunity for societies to rebuild, address serious structural problems, and prevent affected people from suffering the same hardships again, thereby increasing the resilience of local communities. The strength of urban resilience determines the social and economic resilience of cities and the speed of recovery [3].

In recent years, there has been a growing body of research on urban resilience assessment. Researchers’ research topics cover communities and cities and even extend to different levels such as cross-regional level. The research methods involve both mathematical and spatial analysis methods [8]. For example, Cariolet et al. [9] use maps and geographic information systems (GIS) to assist in mapping urban resilience. McClymont et al. [10] developed a multiobjective optimization system for urban resilience and green infrastructure systems, using policy, performance, connectivity, and socializing as the four main dimensions of the system. Meerow and Newell [2] further define and discuss urban resilience through an extensive literature review and compilation. However, most scholars have focused on urban performance in response to individual disasters (e.g., earthquakes, floods, epidemics, and transportation disasters) [11], but participation in comprehensive urban resilience assessment is still limited. Furthermore, it is necessary to consider the mutual influences of various factors/attributes in the overall assessment of urban resilience [8].

In this study, we discuss the following four key urban resilience assessment questions: (i) Which attributes are suitable for assessing urban resilience? (ii) What are the mutually influential relationships of attributes? And what is their importance? (iii) How can an assessment model be constructed to measure the resilience of a city? (iv) Finally, how can poorly resilient cities be improved? These questions constitute our research motivation. The four points make up a typical MADM problem, and the MADM approach has excellent analytical performance in complex evaluation environments. It does not require the assumptions of normal distribution and independence that traditional statistics require for its use [12, 13]. The process performed by MADM includes identifying assessment attributes, calculating attribute weights, and integrating the performance of the evaluated items [14]. The urban resilience evaluation framework relies on the judgment of multiple experts. Only a few studies have used the MADM concept for urban resilience assessment [15]. In addition, most of the studies using qualitative surveys lack the exploration of the interaction of factors affecting urban resilience. Urban communities can be viewed as complex and dynamic interactions of physical, facility, social, economic, and environmental systems. People living in cities move, work, and engage in activities among communities regularly. Therefore, the resilience of a neighborhood cannot be considered completely independent of the resilience of its surrounding neighborhoods [3].

In the past decades, MADM techniques have been playing an important role in decision-making issues in many research areas, such as supplier selection [16], renewable power sources evaluation [17], sustainable sports tourism planning [18], and risk assessment [19]. When evaluating alternatives based on criteria/attributes/indicators, experts/decision makers usually refer to extreme values of attributes (the-larger-the-better and the-smaller-the-better characteristics) in many MADM methods, such as COmplex PRoportional ASsessment (COPRAS), Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS), VIseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR), ELimination Et Choice Translating REality (ELECTRE), and Preference Ranking Organization METHod for Enrichment Evaluation (PROMETHEE). In these methods, TOPSIS and VIKOR use the same principles to determine positive and negative ideal solutions based on the extreme values of the attributes [20]. However, in the real world, many attributes do not belong to the-larger-the-better or the-smaller-the-better characteristics, or there are ranges of preferences for these attributes rather than simply extreme values, e.g., temperature and humidity for human habitation, optimal age for athletes, human blood pressure and heart rate, and height of ideal partners. Obviously, the optimal values of these attributes are not extreme, and they should be of the nominal-the-best characteristic; in addition, these values should have a specific interval range. In these cases, the expert cannot make the most appropriate decision based on extreme values. Moreover, when the number of alternatives increases or decreases, it may affect the ranking results of these traditional MADM methods, most often by reversing the ranking [21]. Cables et al. [22] proposed the Reference Ideal Method (RIM) to overcome these problems. The RIM provides a reasonable way of calculating the attributes of the nominal-the-best characteristic, which can set the ideal solution considered by the experts as the reference ideal solution and set the reference ideal solution as an interval value to avoid the problem of ranking reversal of evaluated items.

However, in the current complex and uncertain decision-making environment, expert judgments and opinions are difficult to interpret using crisp values [23]. Furthermore, when experts are faced with problems that they cannot fully grasp, the confidence level of their assessments may not be 100%. Many researchers have proposed various fuzzy theories to describe the ambiguities and uncertainty of the information expressed by experts. Common fuzzy theories include general fuzzy set theory, intuitionistic fuzzy set theory, hesitation fuzzy set theory, Fermatean fuzzy set theory, and Pythagorean fuzzy set theory. However, little attention has been paid to the reliability or confidence of the messages of these theories. A theory called Z-number theory, proposed by Zadeh [24], takes into account these limitations by using two sets of fuzzy numbers to evaluate the events and to measure the reliability of the evaluated values. Z-number has been widely used in various decision-making problems based on uncertain environments. For example, Peng et al. [25] used Z-TOPSIS to select the most suitable location for an inland nuclear power station in Hunan Province, China. Hsu et al. [26] proposed a modified Z-number-based Decision-Making Trial and Evaluation Laboratory (Z-DEMATEL) to explore the interrelationship and priority of Taiwan’s medical industry development trends. Garg et al. [27] developed granulized Z-VIKOR to provide a novel failure mode and effect analysis model in the field of risk management. Although MADM methods combined with Z-number overcome the problem of reliability measurement in expert assessment, they only consider the maximum or minimum value of the attribute when evaluating the attribute and do not reasonably address the attribute of the nominal-the-best characteristic. In addition, the evaluated items do not necessarily have to meet the extreme values for some attributes of the-larger-the-better or the-smaller-the-better characteristic, they only have to reach a certain range to be accepted. In fact, it is costly to achieve the attribute extremes.

Therefore, in order to fill this research gap, this paper proposes a novel MADM model that combines Z-DEMATEL and Z-RIM to evaluate urban resilience. The proposed modified Z-DEMATEL extends the study of Hsu et al. [26] by using trapezoidal fuzzy numbers instead of the general triangular fuzzy numbers in assessing the mutually influential relationships between attributes. Compared to triangular fuzzy numbers, trapezoidal fuzzy numbers may cover a wider range of ambiguities. The improved Z-DEMATEL determines which attributes are the main factors affecting the resilience system and generates an influential network relationship map (INRM). Besides, the improved Z-RIM extends the concept of TOPSIS by using ideal and nonideal solutions to map the relative position of each evaluation item when determining the variation in each alternative relative to the normalized reference ideal. The improved Z-RIM not only overcomes the limitations of Z-VIKOR and Z-TOPSIS in practical applications, but also facilitates the examination of which attributes of the evaluated items are underperforming [22]. The proposed model is valid and reliable for assessing urban resilience. This study provides information on which attributes are the main factors affecting the resilience of cities, and the government and relevant ministries can focus on these attributes to develop relevant regulations. The results of the RIM can be also used to observe which attributes are underperforming in their current state, and decision makers can develop improvement strategies to enhance city resilience based on these underperforming attributes.

The MADM model proposed in this study provides a novel soft calculation method for urban resilience assessment. As far as we know, the proposed model is novel, and no other studies have proposed this integration method. In addition, both the conventional DEMATEL and RIM have been improved in this study. The characteristics, innovations, and contributions of this work are summarized as follows.(i)This paper introduces an effective integrated model, which provides a reference for the government and relevant ministries to evaluate urban resilience.(ii)This study uses Z-number to measure the reliability of experts in the evaluation. Both Z-DEMATEL and Z-RIM methods reflect the decision-making process in an uncertain environment.(iii)Experts could understand the causal relationship between the attributes more clearly through the INRM constructed by Z-DEMATEL. In addition, underperforming attributes can be observed through Z-RIM. Therefore, decision makers can formulate strategies for improving urban resilience.(iv)In this study, six major metropolitan cities in Taiwan are taken as examples to demonstrate the feasibility of the model.

The other sections are arranged as follows: Section 2 introduces the description of the evaluation framework and attributes. Section 3 explains the concept and calculation of trapezoidal fuzzy numbers and Z-number theory. Section 4 introduces the proposed MADM model, including calculation procedures describing Z-DEMATEL and Z-RIM. Section 5 uses survey data from six major cities in Taiwan as a practical application case to prove the feasibility and practicability of the proposed model. Section 6 discusses and describes the implications for management and finally gives conclusions and future research directions.

2. Description of the Evaluation Framework and Attributes

This section reviews the literature related to urban resilience. Many researchers have developed some indicators, frameworks, and conceptual models to quantify resilience analysis [3, 28–35]. Our first task is to establish a suitable evaluation framework to measure the resilience of cities and then, by collecting and sorting out relevant data sets, to create an urban resilience knowledge system. We identify and select widely available data sources.

Very few scholars have explored the research on urban resilience assessment related to Taiwan. In particular, the evaluation framework constructed with the MADM concept has not yet been fully studied. This study establishes the initial evaluation attributes suitable for assessing urban resilience in Taiwan based on relevant academic literature and opinions from disaster prevention-related ministries (including police and firefighting units, relevant departments of universities, and research institutions) and then selects relatively important attributes to be included in the evaluation system to reflect the characteristics and connotations of urban resilience. The main framework consists of four dimensions, namely, social infrastructure and community connectivity (SI), contingency capacity (CC), economic strength (ES), and environmental conditions (GC). Each of these dimensions can be divided into several attributes, and a total of 24 attributes are used to build the evaluation framework, as shown in Table 1. The proposed city resilience attributes can be used to examine the resilience of the evaluated cities.

3. Preliminaries

In this section, the basic concepts and calculation logic of trapezoidal fuzzy numbers and Z-number are introduced.

3.1. The Definition and Operation Laws of Trapezoidal Fuzzy Number

In the real world, most qualitative evaluations involve ambiguity and uncertainty due to the presence of many unknown and unidentified information in the decision-making process. Moreover, in an uncertain environment, ambiguity and subjective judgment can greatly affect the decision-making process. To alleviate this problem, researchers have proposed the fuzzy theory to express the information uncertainty encountered in decision making [36]. Often, linguistic variables are used to describe information about an expert’s evaluation, which is a convenient way for humans to express their evaluation ideas. Linguistic variables are effective in converting qualitative content into fuzzy forms of quantitative data [37]. Many studies use trapezoidal fuzzy numbers for modeling fuzzy information. A trapezoidal fuzzy number can be symmetric or asymmetric, and it covers a wider range of uncertainty than the conventional triangular fuzzy number [38].

A fuzzy set E on a universe discourse X can be written as a pair of (x, μ_E), where μ_E: X[0, 1] is the membership function. The fuzzy number on the real set S can be defined as a trapezoidal fuzzy number; the membership function is shown in the following equation:

A trapezoidal fuzzy number can be denoted as , where e₁ < e₂ < e₃ < e₄. Suppose there are two trapezoidal fuzzy numbers and , and is a positive constant number. Then, the operation of the trapezoidal fuzzy number can be defined as follows [20]:

(i) Addition:

(ii) Subtraction:

(iii) Division:

(iv) Multiplication:

(v) Multiplication by a positive constant number:

3.2. The Transformation Rules of Z-Number

Zadeh [24] proposed a variation of fuzzy numbers called Z-number, which adds “reliability/confidence” as a parameter in fuzzy operations. A Z-number covers two kinds of fuzzy information: one is the judgment of the experts/decision makers (), and the other is the reliability of the judgment (). The Z-number can be recorded as , where is the trapezoidal fuzzy number of the judgment value and is a measure of the reliability of the fuzzy number , and they can be expressed as and . In this paper, and are the trapezoidal fuzzy number and triangular fuzzy number, respectively. The reliability of the Z-number can be converted into a reliability weight by the following equation:

Next, the reliability weights obtained according to (7) are integrated into the judgment value , and the weighted Z-number is as in the following equation:

A simple example is used to illustrate the procedure of Z-number calculation. Assuming a Z-number with the judgment value and reliability , it forms . According to (7), the reliability weight is calculated as follows:

Then, integrate with the judgment value , and is as follows:

The Z-number is converted to a regular fuzzy number .

In this paper, the linguistic variables to measure the reliability of experts’ judgment are based on the research of Hsu et al. [26], as shown in Table 2. More Z-number calculation examples can be found in Zadeh [24] and Dong et al. [20].

4. The Proposed MADM Model

This section introduces the proposed MADM model. First, the influence weights for evaluating the attributes are obtained through Z-DEMATEL. Then, the results of the Z-DEMATEL analysis are incorporated into the Z-RIM algorithm to calculate the performance of the alternative solutions.

4.1. Z-DEMATEL

DEMATEL is a technology that defines the mutually influential relationships between attributes. It can construct a structured INRM to help decision makers understand the complex relationships of the system, to identify which factors are the main factors that affect others and which are the factors that are affected [26]. In an environment full of uncertainty, it is difficult to use crisp values to reflect the true judgments and ideas of experts. Although many fuzzy theory methods incorporate DEMATEL to overcome the problem of uncertainty evaluation, there is a lack of reliability measurement for these evaluation values. In this study, Z-number is introduced into DEMATEL; not only can we know the reliability of experts in the evaluation process, but also we can use trapezoidal fuzzy numbers to evaluate the uncertainty of the influential relationships of attributes. The detailed steps of the modified Z-DEMATEL technique are as follows. Step 1. Identify a set of attributes suitable for the evaluation issue. The potential evaluation attributes are compiled through literature review, and then a decision-making team is formed by experts to screen the attributes to create an appropriate set of evaluation attributes a_j, j = 1, 2, …, n. Step 2. Construct the average direct relation matrix . There are n attributes that need to be evaluated for influence. Each expert evaluates the direct influence degree of the jth attribute on the j'th attribute according to the evaluation level of Table 3; j = j' = 1, 2, …, n. Next, the confidence level in the evaluation is measured according to the reliability level in Table 2. According to the judgment of expert k (k = 1, 2,…, K), the direct relation matrix can be constructed. The arithmetic mean is used to integrate an average direct relation matrix , as shown in the following equation: where and . Here, DEMATEL requires the diagonal elements in the matrix to be 0; that is, (when j = j'). According to the Z-number operation described in Section 3.2, the direct relation matrix is transformed into the matrix as shown in (12). is the reliability weight of the evaluation attribute j to the attribute j'. The matrix of the k experts is integrated into the average direct relation matrix by the arithmetic mean, as shown in equation (13). where . Step 3. Obtain the normalized direct relation matrix through normalization. Equation (14) is used for normalization calculation to obtain the normalized direct relation matrix . where , j = j' = 1, 2, …, n. Step 4. Construct the total influence matrix . The total influence matrix , (15), uses (16) to aggregate all the direct and indirect influence relationships of the normalized direct relation matrix . The accumulation from the first power to the infinite power of matrix can reflect all potential influence relationships. Because the operating procedure of (16) is cumbersome, a faster solution can be derived from equation (17). where . where and I are unit matrices. The superscript “-1” indicates the inverse matrix. Step 5. Plot INRM to identify the mutually influential relationships between attributes. Each column and each row of the total influence matrix is summed up to obtain (18) and (19). is the influence of attribute j on other attributes, and is the influence on attribute j by other attributes. where , , , and . The superscript “T” represents the matrix transposition. is the index of the strength of influences given and received, which is called the total influence. On the other hand, represents the net influence. The larger , the greater the influence of the attribute j on the evaluation system. If > 0, this means that attribute j has a more significant influence on other attributes, which is called a causal factor; conversely, if < 0, this means that attribute j is more influenced by other attributes, which is called an affected factor. In this paper, the centroid method is used to defuzzify the fuzzy values (e.g., ) to obtain the crisp value () as shown in the following equation [20]. Next, and are obtained as r_j and s_j, respectively, by the defuzzification procedure of (20). The relative coordinate positions of each attribute are clearly plotted using as the horizontal axis and as the vertical axis. The matrix is used to identify the influence between each attribute, and the arrows (indicating the direction of influence) are drawn to generate a systematic INRM. Step 6. Obtain the influence weights of the attributes. Here, reflects the total influence of the attribute on the evaluation system, so the influence weight of the attribute, , can be constructed by (21). Here, the total weight is required to be 1.

4.2. Z-RIM

Z-number is introduced into the traditional RIM method, and a practical MADM method is proposed to rank the alternatives, called Z-RIM. The proposed Z-RIM method can handle the uncertainty of the performance rank given by the decision maker. At the same time, the ranking reversal problem can be effectively avoided. The main steps of the proposed Z-RIM method are as follows. Step 1. Set the Z-RIM parameter. First, the required execution parameters of Z-RIM are established, and the following parameters are set for each attribute a_j [22]:(i)The value range of the attribute a_j: (ii)The reference ideal (RI) of the attribute a_j: (iii)The weight of the attribute a_j: w_j Step 2. Obtain the evaluation matrix . Assuming that there are m evaluated items and n attributes , each performance value of evaluated item V_i under attribute a_j is investigated. In this paper, the data are collected based on the public information provided by the police and fire agencies of the Ministry of Internal Affairs. Some of the attribute data are ambiguous in nature, so Z-numbers are added to reflect their uncertainty. After converting the Z-numbers in Section 3.2, the evaluation matrix is shown in the following equation: where . Step 3. Calculate the normalized evaluation matrix F. It is calculated by referring to the fuzzy RIM normalization program proposed by Cables et al. [40] to obtain the normalized evaluation matrix F, as shown in the following equation: where f_ij is the crisp value obtained by normalization. Fuzzy numbers are converted to crisp numbers in the normalization process. Step 4. Construct the weighted normalized evaluation matrix G. Taking into account the different importance of each attribute, the attribute weights obtained from Z-DEMATEL are multiplied with matrix F to obtain the weighted normalized evaluation matrix G, as shown in the following equation: Step 5. Calculate the distance to the ideal and nonideal evaluated items. The definition of Euclidean distance is used to calculate the distance to the ideal and nonideal evaluated items, as shown in the following equations: Step 6. Calculate the relative index to the reference ideal of each evaluated item. The ranking index of the evaluated item can be calculated as (27). The closer I_i is to 1, the closer it is to the reference ideal. On the contrary, the closer I_i is to 0, the closer the performance is to be extremely poor.

5. Illustration of a Real Case

5.1. Problem Description

Taiwan is located in the eastern part of Asia, at the intersection of Northeast Asia and Southeast Asia. Due to the unique geographical location of Taiwan, natural disasters occur frequently. Typhoons generated from tropical cyclones bring instantaneous rainfall, resulting in heavy rains, floods, and landslides. Taiwan is located at the junction of plates (in the Pacific Rim seismic zone), so earthquakes occur frequently, with an average of over 100 felt earthquakes per year. Besides, Taiwan’s high population density makes some human-induced disasters also prone to high casualties [41]. Therefore, it is an urgent task to evaluate the resilience of Taiwan’s major urban areas. This study focuses on four questions: What are the interdependencies of urban resilience attributes? What are their weights? How resilient are these cities? And how can the resilience of cities be improved and strengthened? The answers to these questions will help the government to develop strategies to improve the disaster preparedness of Taiwan. We extensively invite experts on urban resilience and disaster prevention in Taiwan to join the research group. A total of 13 experts were surveyed for this study, including those from police and firefighting units, relevant departments of universities, and research institutions. These experts have sufficient expertise and years of experience in urban resilience. They have worked on the topic for more than 10 years. In addition, these experts have contributed to the promotion of critical infrastructure protection policies in Taiwan. Table 4 shows the organization, working experience, and academic qualifications of the experts.

This study analyzed the evaluated projects according to Section 2 evaluation framework. According to the definition of municipality promulgated by the Executive Yuan of Taiwan, cities belonging to municipalities directly under the Central Government are included in the evaluation items, including Taipei City (V1), New Taipei City (V2), Taoyuan City (V3), Taichung City (V4), Tainan City (V5), and Kaohsiung City (V6), commonly known as the “six direct-controlled municipalities”. Direct-controlled municipalities are the first-level administrative divisions of Taiwan and are directly under the jurisdiction of the Executive Yuan. According to the Local Government Act, municipalities are established in areas with a population of over 1.25 million and special needs in political, economic, cultural, and metropolitan development.

5.2. Using Z-DEMATEL to Generate Attribute Weights and Identify Their Causal Relationships

In Section 4.1, the calculation process of Z-DEMATEL is described, and the data from the 13 experts’ surveys are executed according to the process to identify which attributes are influential. The level of consensus among experts can be examined by the average sample gap index (), where n is the sample size, k is the number of experts, and is the evaluation value in the matrix . The average gap of 13 experts calculated by the index is 4.7%, which means that there is a 95.3% confidence level that these experts have a consensus of about 95%. Tables 5 and 6 present the average direct relation matrix and the normalized direct relation matrix , respectively. The diagonal elements of the matrix are all 0. In addition, all elements are Z-numbers, and the total influence relationship of the attributes can be seen in the total influence matrix (Table 7), where the elements of the matrix encompass the direct and indirect influences of the attributes.

Table 8 shows the analysis results of Z-DEMATEL, including the attributes’ total influence (r_j + s_j) and net influence (r_j − s_j). The greater the net influence, the greater the influence of the attribute on other attributes. r_j + s_j can indicate the total influence in the overall evaluation system to express the importance ratio (the weights of the attributes). The top five attributes’ weights generated by Z-DEMATEL are ranked as SI1ES3CC1GC4ES4.

Moreover, with r_j + s_j as the horizontal axis and r_j − s_j as the vertical axis, the INRM of the attributes is plotted, as shown in Figure 1. In Figure 1, the attributes on the upper right indicate high total and net influences, being the main causes. On the other hand, the attributes on the lower left indicate lower total and net influences, being the effects [18]. Obviously, the cause attributes are SI1, ES3, GC4, ES4, ES5, and GC5. In addition, CC3, CC2, SI2, CC6, GC3, SI4, SI7, and SI3 are factors that are more likely to be influenced by other attributes. The management implications derived from the Z-DEMATEL analysis are discussed in Section 6.

5.3. Applying Z-RIM to Integrate the Resilience Performance of the Six Major Cities

The performance values of the six evaluated objects (six major cities in Taiwan) in the proposed evaluation framework were investigated as shown in Tables 9–12. These data were collected from publicly available data provided by the relevant government agencies in Taiwan, and the parameters required by Z-RIM were set by 13 experts. Most of the attributes are quantitative data from the actual survey and are therefore of the crisp type. For example, the survey value of V1 under the SI1 attribute is 9731.6 (person/km2). Nevertheless, GC1 and GC2 are uncertain data and thus are Z-numbers. For example, the Z-numbers of V1 under the GC1 attribute are (16937697, 16985131, 17032565, 17079999). The overall performance (ranking index) of the evaluated objects was obtained by following the Z-RIM execution steps described in Section 4.2.

The degree of separation of the evaluated objects from the ideal and nonideal solutions can be determined. What is certain is that the evaluated objects with better performance will be closer to the ideal solution, while the objects with poor performance will be far away from the ideal solution. On the contrary, the evaluated items with better performance will be far away from the nonideal solution. Table 13 shows the calculation results of Z-RIM. The performance of the evaluated objects is V1V4V5V3V2V6 in order.

6. Discussion and Conclusions

The analysis of urban resilience is an important and urgent task. Advanced countries have invested huge amounts of money in this issue to strengthen the resilience of cities. In order to improve the level of urban resilience, many researchers have proposed advanced methodologies to effectively assess urban resilience and provide feasible improvement strategies as a reference [3, 5, 6]. Based on the literature review, previous studies rarely used MADM models to explore the mutual influences and weights of urban resilience attributes. In addition, an urban resilience evaluation framework suitable for Taiwan has not yet been fully established. This paper proposes a novel MADM model to bring a more complete evaluation framework and analysis method to the problem of urban resilience measurement. Firstly, through a large number of literature reviews and expert interviews, 4 dimensions and 24 attributes were determined to establish an evaluation framework. Secondly, this paper uses Z-DEMATEL to obtain the mutually influential relationships and weights of attributes and then generates an INRM so that decision makers can easily identify the key influencing factors in the evaluation system. The Z-DEMATEL proposed in this paper is an extension of the research of Hsu et al. [26]. It uses trapezoidal fuzzy numbers to envelop a wider range of uncertainty and effectively measures the confidence of experts in the evaluation. Finally, we also improved the conventional RIM [22], combined with Z-numbers to form the Z-RIM, and further optimized the practicability of the conventional RIM in a fuzzy environment.

Based on the Z-DEMATEL results in Table 8, in terms of total influence (r_j + s_j), the top five are “population density in the urban area (SI1) (4.941),” “outcome of business activities (ES3) (4.541),” “number of healthcare services (CC1) (4.474),” “density of buildings (GC4) (4.393),” and “number of existing businesses in the area (ES4) (4.310)”. Besides, the total weight of these five attributes is approximately 0.25 (0.053 + 0.049 + 0.048 + 0.047 + 0.046), which accounts for about a quarter of the overall evaluation system. The INRM shows that “population density in the urban area (SI1),” “outcome of business activities (ES3),” “density of buildings (GC4),” “number of existing businesses in the area (ES4),” “disposable income on average households (ES5),” and “number of buildings over 30 years old (GC5)” are the main factors affecting the relationship system. The results of Z-DEMATEL analysis echo the research of Monteiro et al. [28], Borsekova et al. [29], Kontokosta and Malik [3], and Zheng et al. [30]. This study focuses on the results of Z-DEMATEL analysis. Several feasible improvement measures for resilience are proposed and described as follows.(i)The population density in urban areas: Many resilience-related documents mention the impact of population density and population composition on urban resilience, and most of them focus on the vulnerable population, such as the elderly, the young, and the disadvantaged. These populations have higher vulnerability and lower resilience in the face of natural disasters [42–45]. In terms of improvement measures, the possibility of urban migration can be considered. Taipei is the capital of Taiwan, and capitals are often the economic, political, and cultural centers of countries. However, as the scale of capital cities continues to expand, many city capacity problems will arise, for example, huge problems such as traffic congestion, environmental pollution, overconcentration of population, and disaster multiplication. Relocating the capital city, or moving part of the government to a different city, can solve the problems of uneven economic development and urban disaster multiplication in a country and strengthen the resilience of the city, which is a better policy choice. There are many successful cases in the world, such as the Seoul metropolitan area, which has nearly 50% of the country’s population, and the problems of metropolitan areas are getting more and more serious, such as high housing prices, traffic congestion, and environmental pollution. In order to solve the urban development problem, Korea decentralized Seoul’s urban capacity and relocated the capital to become the first choice to decentralize Seoul’s politics and economy. Therefore, the National Assembly passed the New Administrative Capital Special Act in December 2003 to provide legal protection for the relocation of Korea’s capital.(ii)The outcome of business activities and the number of existing businesses in the urban area: The output value of urban industrial activities and the number of industrial companies in the urban area are important indicators to measure urban resilience. Different industries have different influences on disaster resilience. For example, agriculture may be less resilient than the commercial and industrial sectors because of its lower value and less attention received [43]. For example, the key focus of Taipei City is the development of high-tech industries such as communications, electronics, information, computers, and machinery, located mainly in the Neihu and Nangang districts of Taipei City, which are in line with the current key construction of Taipei’s urban development and the creation of high-tech industrial clusters. The impact of large-scale disasters on the industry is relatively large for sure. In particular, earthquakes and floods in Taiwan have a significant impact on urban resilience. The stability of electricity is also an important determinant of industrial development and urban resilience. Therefore, it is recommended that the government should pay attention to the industrial development characteristics of a city, such as Taipei City as a high-tech development city, and also pay attention to the industrial clustering effect, the more industrial clusters, what is more favorable to the development of industries, what is more favorable to the city income, and what is more favorable to the city resilience.(iii)Density of buildings/number of buildings over 30 years old: The distribution of buildings affects the impact level of earthquakes and relevant disasters (e.g., floods), as well as the response and recovery ability during and after disasters. In general, the higher the number of buildings and age of buildings, the lower the urban resilience; on the contrary, stronger buildings are expected to have a positive relationship with urban resilience [46]. Strategies for improvement include accelerating urban renewal and introducing disaster-resistant urban building plans, such as the design of sponge cities and seismic retrofitting of 30-year-old houses, to strengthen urban resilience and disaster resistance.(iv)Number of healthcare services: In the event of a disaster, many casualties may occur, and the distribution of healthcare facilities is important in measuring the disaster response capability of a region. In terms of disaster prevention and relief facilities, in addition to the number and distribution of medical institutions, it is also necessary to consider the distribution of fire and police agencies and the number of disaster response sites. The more these facilities are adequate, the more the public resources and space are available to flexibly deal with or recover from disaster-related impacts and therefore have a higher degree of urban resilience.(v)Disposable income on average households: When a disaster occurs, having more disposable income allows people to purchase different disaster prevention and relief equipment or to have more flexibility in disaster avoidance and relief, to reduce the occurrence of damage and to have more resources to rebuild after the disaster. The higher the disposable income, the higher the resilience.

It is feasible and effective to use MADM methodologies to discuss the comprehensive assessment of alternatives [47–49]. The results of the Z-DEMATEL analysis were shared and fed back to all experts, who indicated that the information assisted them in their decision making in the strategic alliance. In addition, the results of the Z-RIM analysis show that Taipei City (V1) and Taichung City (V4) are relatively more resilient in terms of urban capacity. We use index I_i as attributes to determine the resilience of the cities. I₁ to I₅ are all greater than 0.5, which means that the resilience of the five cities is above the average of the assessment system; only I₆ of V6 is 0.4978, which is less than 0.5. Therefore, appropriate improvement strategies should be given to Kaohsiung City (V6). Kaohsiung City has a diverse landscape and is a large industrial city in southern Taiwan and therefore faces complex and diverse types of disasters. Every year, Kaohsiung is threatened by typhoons, earthquakes, and factory explosions. Highly developed urban areas need to be protected from flooding, hillside areas have the potential for landslides and collapses, and oceanfront areas need to be protected from tsunamis. In addition to natural disasters, there are also threats from man-made fires, gas, firecrackers, and dangerous waters. In terms of improving policies, the Kaohsiung government should conduct disaster potential surveys, use and update disaster risk maps, deepen local disaster prevention and relief capabilities, establish various promotion mechanisms for disaster prevention, conduct education and training, improve the capabilities of relevant disaster prevention and relief personnel, and enhance the local government’s ability to promote disaster prevention and relief work. Furthermore, Kaohsiung should also strengthen the chain of industrial activities, including introducing foreign investment, strengthening the formation of industrial clusters (such as science parks), and strengthening the stability of water and electricity infrastructure. Moreover, the care and immediate evacuation of vulnerable and low resilience populations (such as the elderly, the young, and the disabled) will help to improve the resilience of the city.

This study performed a model comparison, including Z-SAW (Simple Additive Weighting), Z-WASPAS (Weighted Aggregated Sum Product ASsessment), Z-COPRAS, Z-TOPSIS, Z-VIKOR, and Z-RIM methods. The calculation results of multiple models are shown in Table 14. Obviously, the three methods Z-TOPSIS, Z-VIKOR, and Z-RIM have the same sorting result because they are all based on the concept of compromise sorting. However, Z-RIM considers more potential management information than Z-TOPSIS and Z-VIKOR. In addition, we also performed a sensitivity analysis to check the robustness of the proposed model. The weight of the SI1 (the most important attribute) was changed from 0.1 to 0.9, and the other attributes were weighted proportionally. Table 15 shows the ranking results of the nine runs. It can be confirmed that the sensitivity of the proposed model will not be affected by the single attribute weight.

Our proposed MADM model provides a systematic analysis process to provide a complete resilience evaluation for the six major cities in Taiwan. In summary, this study provides an effective tool for cities on the issue of resilience evaluation. This effective soft calculation method can be combined with expert/decision maker’s judgment to propose more realistic management implications. The methodology of this study has not yet been developed in the academic community. We prove the validity and reliability of the proposed model. Several advantages of the model integrating Z-DEMATEL and Z-RIM are as follows: (i) Z-number takes into account the uncertainty and reliability in expert judgment; (ii) Z-DEMATEL explores the mutually influential relationships of urban resilience attributes and generates influence weights for the attributes; (iii) Z-RIM overcomes the shortcomings of conventional TOPSIS, which can be applied to the-larger-the-better, the-smaller-the-better, and the-nominal-the-best attributes; (iv) it helps decision makers to be more systematic in the decision-making process. Moreover, the proposed model is suitable for various evaluation and selection problems based on multiple criteria, especially under uncertain information environment, for example, site selection, corporate performance evaluation, and talent recruitment.

Although the study makes an important contribution to solving the problem of urban resilience, it still has some limitations. The proposed model uses the arithmetic mean to integrate the survey data provided by multiple experts, which makes it impossible to detect any anomalies in the data. In the future, it can be combined with the rough set theory to overcome the aforementioned problem. Furthermore, the concept of aspiration level is not introduced in Z-RIM, so it is impossible to determine the gap between the evaluated items and the aspiration level. At present, the experts suggested evaluating the six major cities in Taiwan. In the future, the model can be applied to evaluate more cities.

Data Availability

All data generated or analysed during the study are included in this published article.

Conflicts of Interest

The authors declare no conflicts of interest.

Authors’ Contributions

H.-W. L. conceptualized the study and developed the methodology. C.-N. H. investigated the study and wrote, reviewed, and edited the manuscript. Both authors prepared the original draft and read and agreed to the published version of the manuscript.

References

I. Tumini, P. Villagra-Islas, and G. Herrmann-Lunecke, “Evaluating reconstruction effects on urban resilience: a comparison between two Chilean tsunami-prone cities,” Natural Hazards, vol. 85, no. 3, pp. 1363–1392, 2017.
View at: Publisher Site | Google Scholar
S. Meerow and J. P. Newell, “Urban resilience for whom, what, when, where, and why?” Urban Geography, vol. 40, no. 3, pp. 309–329, 2019.
View at: Publisher Site | Google Scholar
C. E. Kontokosta and A. Malik, “The Resilience to Emergencies and Disasters Index: applying big data to benchmark and validate neighborhood resilience capacity,” Sustainable Cities and Society, vol. 36, pp. 272–285, 2018.
View at: Publisher Site | Google Scholar
S. S. Fainstein, “Resilience and justice: planning for New York city,” Urban Geography, vol. 39, no. 8, pp. 1268–1275, 2018.
View at: Publisher Site | Google Scholar
S. Meerow, J. P. Newell, and M. Stults, “Defining urban resilience: a review,” Landscape and Urban Planning, vol. 147, pp. 38–49, 2016.
View at: Publisher Site | Google Scholar
C. Simone, F. Iandolo, I. Fulco, and F. Loia, “Rome was not built in a day. Resilience and the eternal city: insights for urban management,” Cities, vol. 110, Article ID 103070, 2021.
View at: Publisher Site | Google Scholar
P. J. G. Ribeiro and L. A. Pena Jardim Gonçalves, “Urban resilience: a conceptual framework,” Sustainable Cities and Society, vol. 50, Article ID 101625, 2019.
View at: Publisher Site | Google Scholar
Z. Wang, X. Deng, C. Wong, Z. Li, and J. Chen, “Learning urban resilience from a social-economic-ecological system perspective: a case study of Beijing from 1978 to 2015,” Journal of Cleaner Production, vol. 183, pp. 343–357, 2018.
View at: Publisher Site | Google Scholar
J.-M. Cariolet, M. Vuillet, and Y. Diab, “Mapping urban resilience to disasters - a review,” Sustainable Cities and Society, vol. 51, p. 101746, 2019.
View at: Publisher Site | Google Scholar
K. McClymont, D. G. Fernandes Cunha, C. Maidment, B. Ashagre, A. F. Vasconcelos, and M. Batalini de Macedo, “Towards urban resilience through Sustainable Drainage Systems: a multi-objective optimisation problem,” Journal of Environmental Management, vol. 275, Article ID 111173, 2020.
View at: Publisher Site | Google Scholar
H. M. Lyu, W. J. Sun, S. L. Shen, and A. Arulrajah, “Flood risk assessment in metro systems of mega-cities using a GIS-based modeling approach,” The Science of the Total Environment, vol. 626, pp. 1012–1025, 2018.
View at: Publisher Site | Google Scholar
Q. Shao, S. S. Weng, J. J. Liou, H. W. Lo, and H. Jiang, “Developing a sustainable urban-environmental quality evaluation system in China based on a hybrid model,” International Journal of Environmental Research and Public Health, vol. 16, no. 8, p. 1434, 2019.
View at: Publisher Site | Google Scholar
M. H. Chang, J. J. Liou, and H. W. Lo, “A hybrid MCDM model for evaluating strategic alliance partners in the green biopharmaceutical industry,” Sustainability, vol. 11, no. 15, p. 4065, 2019.
View at: Publisher Site | Google Scholar
C. F. Liaw, W. C. J. Hsu, and H. W. Lo, “A hybrid MCDM model to evaluate and classify outsourcing providers in manufacturing,” Symmetry, vol. 12, no. 12, p. 1962, 2020.
View at: Publisher Site | Google Scholar
X. Xun and Y. Yuan, “Research on the urban resilience evaluation with hybrid multiple attribute TOPSIS method: an example in China,” Natural Hazards, vol. 103, pp. 557–577, 2020.
View at: Publisher Site | Google Scholar
J. J. Liou, J. Tamošaitienė, E. K. Zavadskas, and G. H. Tzeng, “New hybrid COPRAS-G MADM Model for improving and selecting suppliers in green supply chain management,” International Journal of Production Research, vol. 54, no. 1, pp. 114–134, 2016.
View at: Publisher Site | Google Scholar
Y. Wu, C. Xu, and T. Zhang, “Evaluation of renewable power sources using a fuzzy MCDM based on cumulative prospect theory: a case in China,” Energy, vol. 147, pp. 1227–1239, 2018.
View at: Publisher Site | Google Scholar
J. J. Yang, Y. C. Chuang, H. W. Lo, and T. I. Lee, “A two-stage MCDM model for exploring the influential relationships of sustainable sports tourism criteria in Taichung city,” International Journal of Environmental Research and Public Health, vol. 17, no. 7, p. 2319, 2020.
View at: Publisher Site | Google Scholar
T. W. Chang, H. W. Lo, K. Y. Chen, and J. J. Liou, “A novel FMEA model based on rough BWM and rough TOPSIS-AL for risk assessment,” Mathematics, vol. 7, no. 10, p. 874, 2019.
View at: Publisher Site | Google Scholar
P. Dong, T. Zhang, Y. Ju, and A. Wang, “A novel multi-attribute decision-making framework based on Z-RIM: an illustrative example of cloud service selection,” Soft Comput, vol. 24, no. 23, pp. 18233–18247, 2020.
View at: Publisher Site | Google Scholar
J. M. Sánchez-Lozano and O. N. Rodríguez, “Application of Fuzzy Reference Ideal Method (FRIM) to the military advanced training aircraft selection,” Applied Soft Computing, vol. 88, Article ID 106061, 2020.
View at: Publisher Site | Google Scholar
E. Cables, M. T. Lamata, and J. L. Verdegay, “RIM-reference ideal method in multicriteria decision making,” Information Science, vol. 337, pp. 1–10, 2016.
View at: Publisher Site | Google Scholar
M. Keshavarz-Ghorabaee, M. Amiri, M. Hashemi-Tabatabaei, E. K. Zavadskas, and A. Kaklauskas, “A new decision-making approach based on fermatean fuzzy sets and WASPAS for green construction supplier evaluation,” Mathematics, vol. 8, no. 12, p. 2202, 2020.
View at: Publisher Site | Google Scholar
L. A. Zadeh, “A note on Z-numbers,” Information Science, vol. 181, no. 14, pp. 2923–2932, 2011.
View at: Publisher Site | Google Scholar
H. M. Peng, X. K. Wang, T. L. Wang, Y. H. Liu, and J. Q. Wang, “A multi-criteria decision support framework for inland nuclear power plant site selection under Z-information: a case study in hunan Province of China,” Mathematics, vol. 8, no. 2, p. 252, 2020.
View at: Publisher Site | Google Scholar
W. C. J. Hsu, J. J. Liou, and H. W. Lo, “A group decision-making approach for exploring trends in the development of the healthcare industry in Taiwan,” Decision Support Systems, vol. 141, Article ID 113447, 2020.
View at: Google Scholar
A. Garg, S. Das, J. Maiti, and S. K. Pal, “Granulized Z-VIKOR model for failure mode and effect analysis,” IEEE Transactions on Fuzzy Systems, pp. 1063–6706, 2020.
View at: Publisher Site | Google Scholar
A. Monteiro, V. Carvalho, S. Velho, and C. Sousa, “Assessing and monitoring urban resilience using COPD in Porto,” The Science of the Total Environment, vol. 414, pp. 113–119, 2012.
View at: Publisher Site | Google Scholar
K. Borsekova, P. Nijkamp, and P. Guevara, “Urban resilience patterns after an external shock: an exploratory study,” International Journal of Disaster Risk Reduction, vol. 31, pp. 381–392, 2018.
View at: Publisher Site | Google Scholar
Y. Zheng, X. Xin-Lu, L. Chen-Zhen, W. Mou, and H. E. Xiao-Jia, “Development as adaptation: framing and measuring urban resilience in Beijing,” Advances in Climate Change Research, vol. 9, no. 4, pp. 234–242, 2018.
View at: Publisher Site | Google Scholar
O. Caldarice, G. Brunetta, and N. Tollin, “The challenge of urban resilience: Operationalization,” in Urban Resilience for Risk and Adaptation Governance, pp. 1–6, Springer, Cham, Switzerland, 2019.
View at: Publisher Site | Google Scholar
C. Chen, L. Xu, D. Zhao, T. Xu, and P. Lei, “A new model for describing the urban resilience considering adaptability, resistance and recovery,” Safety Science, vol. 128, Article ID 104756, 2020.
View at: Publisher Site | Google Scholar
M. D. C. Almeida, M. J. Telhado, M. Morais, J. Barreiro, and R. Lopes, “Urban resilience to flooding: triangulation of methods for hazard identification in urban areas,” Sustainability, vol. 12, no. 6, p. 2227, 2020.
View at: Publisher Site | Google Scholar
M. Ghouchani, M. Taji, A. Y. Roshan, and M. S. Chehr, “Identification and assessment of hidden capacities of urban resilience,” Environment, Development and Sustainability, vol. 23, no. 3, pp. 3966–3993, 2021.
View at: Publisher Site | Google Scholar
E. Okada, “Urban resilience and opportunity identification of social entrepreneurs,” in Management of Science-Intensive Organizations, pp. 39–59, Palgrave Macmillan, Cham, Switzerland, 2021.
View at: Publisher Site | Google Scholar
R. E. Bellman and L. A. Zadeh, “Decision-making in a fuzzy environment,” Management Science, vol. 17, no. 4, p. B141, 1970.
View at: Publisher Site | Google Scholar
H. W. Lo, J. J. Liou, C. N. Huang, Y. C. Chuang, and G. H. Tzeng, “A new soft computing approach for analyzing the influential relationships of critical infrastructures,” International Journal of Critical Infrastructure Protection, vol. 28, Article ID 100336, 2020.
View at: Publisher Site | Google Scholar
E. Ayyildiz, A. T. Gumus, and M. Erkan, “Individual credit ranking by an integrated interval type-2 trapezoidal fuzzy Electro methodology,” Soft Computing, vol. 24, no. 21, pp. 16149–16163, 2020.
View at: Publisher Site | Google Scholar
M. Hiete, M. Merz, T. Comes, and F. Schultmann, “Trapezoidal fuzzy DEMATEL method to analyze and correct for relations between variables in a composite indicator for disaster resilience,” Spectrum, vol. 34, no. 4, pp. 971–995, 2012.
View at: Publisher Site | Google Scholar
E. Cables, M. T. Lamata, and J. L. Verdegay, “FRIM—fuzzy reference ideal method in multicriteria decision making,” in Soft Computing Applications for Group Decision-Making and Consensus Modeling, pp. 305–317, Springer, Cham, Switzerland, 2018.
View at: Publisher Site | Google Scholar
L. C. Chen, J. Y. Wu, and M. J. Lai, “The evolution of the natural disaster management system in Taiwan,” Journal of the Chinese Institute of Engineers, vol. 29, no. 4, pp. 633–638, 2006.
View at: Publisher Site | Google Scholar
B. L. Murphy, “Locating social capital in resilient community-level emergency management,” Natural Hazards, vol. 41, no. 2, pp. 297–315, 2007.
View at: Publisher Site | Google Scholar
S. L. Cutter, L. Barnes, M. Berry et al., “A place-based model for understanding community resilience to natural disasters,” Global Environmental Change, vol. 18, no. 4, pp. 598–606, 2008.
View at: Publisher Site | Google Scholar
S. L. Cutter, C. G. Burton, and C. T. Emrich, “Disaster resilience indicators for benchmarking baseline conditions,” Journal of Homeland Security and Emergency Management, vol. 7, no. 1, 2010.
View at: Publisher Site | Google Scholar
S. L. Cutter, K. D. Ash, and C. T. Emrich, “The geographies of community disaster resilience,” Global Environmental Change, vol. 29, pp. 65–77, 2014.
View at: Publisher Site | Google Scholar
H. C. Hung and L. Y. Chen, “Incorporating stakeholders’ knowledge into assessing vulnerability to climatic hazards: application to the river basin management in Taiwan,” Climatic Change, vol. 120, no. 1, pp. 491–507, 2013.
View at: Publisher Site | Google Scholar
S. Bekesiene, A. V. Vasiliauskas, Š. Hošková-Mayerová, and V. Vasilienė-Vasiliauskienė, “Comprehensive assessment of distance learning modules by fuzzy AHP-TOPSIS method,” Mathematics, vol. 9, no. 4, p. 409, 2021.
View at: Publisher Site | Google Scholar
S. Ashraf and S. Abdullah, “Spherical aggregation operators and their application in multiattribute group decision‐making,” International Journal of Intelligent Systems, vol. 34, no. 3, pp. 493–523, 2019.
View at: Publisher Site | Google Scholar
Š. Hošková-Mayerová, V. Talhofer, P. Otřísal, and M. Rybanský, “Influence of weights of geographical factors on the results of multicriteria analysis in solving spatial analyses,” ISPRS International Journal of Geo-Information, vol. 9, no. 8, p. 489, 2020.
View at: Google Scholar

Copyright

Copyright © 2021 Chun-Nen Huang and Huai-Wei Lo. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

537

Downloads

496

Citations