Table of Contents Author Guidelines Submit a Manuscript
Computational and Mathematical Methods in Medicine
Volume 2015, Article ID 257856, 13 pages
http://dx.doi.org/10.1155/2015/257856
Research Article

The Prioritization of Clinical Risk Factors of Obstructive Sleep Apnea Severity Using Fuzzy Analytic Hierarchy Process

1Department of Mathematics, King Mongkut’s University of Technology Thonburi, Bangkok 10140, Thailand
2Department of Medicine, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand

Received 23 October 2014; Accepted 10 May 2015

Academic Editor: Kaan Yetilmezsoy

Copyright © 2015 Thaya Maranate et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Recently, there has been a problem of shortage of sleep laboratories that can accommodate the patients in a timely manner. Delayed diagnosis and treatment may lead to worse outcomes particularly in patients with severe obstructive sleep apnea (OSA). For this reason, the prioritization in polysomnography (PSG) queueing should be endorsed based on disease severity. To date, there have been conflicting data whether clinical information can predict OSA severity. The 1,042 suspected OSA patients underwent diagnostic PSG study at Siriraj Sleep Center during 2010-2011. A total of 113 variables were obtained from sleep questionnaires and anthropometric measurements. The 19 groups of clinical risk factors consisting of 42 variables were categorized into each OSA severity. This study aimed to array these factors by employing Fuzzy Analytic Hierarchy Process approach based on normalized weight vector. The results revealed that the first rank of clinical risk factors in Severe, Moderate, Mild, and No OSA was nighttime symptoms. The overall sensitivity/specificity of the approach to these groups was 92.32%/91.76%, 89.52%/88.18%, 91.08%/84.58%, and 96.49%/81.23%, respectively. We propose that the urgent PSG appointment should include clinical risk factors of Severe OSA group. In addition, the screening for Mild from No OSA patients in sleep center setting using symptoms during sleep is also recommended (sensitivity = 87.12% and specificity = 72.22%).

1. Introduction

Obstructive sleep apnea (OSA) is a common medical disorder characterized by repetitive partial or complete collapse of the upper airway during sleep, resulting in sleep fragmentation and cyclic oxygen desaturation. The prevalence of OSA in adult population is approximately 3–7% for males and 2–5% for females [14]. This can be as high as 50–98% in the morbidly obese population [5]. OSA leads to neurocognitive consequences for example, unrefreshing sleep, excessive daytime sleepiness, motor vehicle accidents and work performance [6], impaired quality of life, and considerable morbidity for cardiovascular diseases [7, 8] as well as a substantial economic impact [9]. OSA can progress to more severe if it is left untreated. Diagnosis and severity assessment required Respiratory Disturbance Index (RDI) obtained from polysomnography (PSG). The RDI is defined as the total numbers of apneas, hypopneas, and respiratory-effort related arousals (RERAs) per hour of sleep (events/hour).

Unfortunately, PSG is not widely available in Thailand because of the relative lack of sleep laboratories and results in a very long waiting list. Delayed diagnosis and treatment may lead to worse outcomes particularly in patients with Severe OSA. Therefore, the appointment for PSG should be based on OSA severity rather than the first-come-first-serve basis. Moreover, clinicians prefer a simple and nonexpensive tool to predict the severity of OSA. To date, no single clinical information can predict the severity of OSA. We believe that if symptoms and anthropometric data are categorized into groups, this may solve the problem. We previously studied 113 variables of 1,042 sleep questionnaires, anthropometric measurements, and PSGs data from suspected OSA patients of Siriraj Sleep Center. Using factor analysis, the 19 groups of clinical risk factors consisting of 42 variables were categorized into each OSA severity [10]. This research is the extended study aiming to prioritize these 19 groups of factors in each level of OSA severity by using Fuzzy Analytic Hierarchy Process (FAHP) approach. The study has been approved by the Ethics Committee of Siriraj Institutional Review Board, Faculty of Medicine Siriraj Hospital, Mahidol University, Thailand.

2. Materials and Methods

2.1. Patients, Sleep Questionnaire, and Polysomnography

The 1,042 suspected OSA patients who underwent diagnostic PSGs at Siriraj Sleep Center, Siriraj Hospital, during 2010-2011 were studied. Sleep questionnaires and anthropometric measurements were obtained in all patients in the evening prior to undergoing PSG. Self-administered sleep questionnaire consisted of 5 domains (113 questions), including (1) patients’ characteristics; (2) work and sleep pattern; (3) nighttime symptoms; (4) daytime symptoms, and (5) Epworth Sleepiness Scale. Anthropometric measurements including body weight, height, neck, waist, and hip circumferences, and thyromental distance were measured by sleep technicians. All patients underwent PSGs using standard acquisition techniques. All PSGs were manually scored by certified advanced sleep technicians and reviewed by certified sleep specialists using standard scoring rule AASM 2007 [16]. Patients with central sleep apnea were excluded. Patients with RDI of <5.0, 5.0–14.9, 15.0–30.0, and >30 events/hour are classified as having No OSA, Mild OSA, Moderate OSA, and Severe OSA, respectively. The distribution of OSA severity is shown in Table 1.

Table 1: Patients’ characteristics.
2.2. The Hierarchical Structure of 19 Clinical Risk Factors of All OSA Severities from Factor Analysis [10]

The abovementioned 42 variables which were categorized into No, Mild, Moderate, and Severe OSA group comprised 3, 5, 5, and 6 clinical risk factors, respectively. All these factors will be prioritized according to their importance to OSA (Figure 1).

Figure 1: The preprioritized hierarchical structure of 19 clinical risk factors of all OSA severities.
2.3. A Questionnaire of Analytic Hierarchy Process (AHP)

We created questionnaire of AHP using 9-point scale [1115] (Table 2). The pairwise comparisons between clinical risk factors were performed within each level of OSA severity. Therefore, there were 3, 10, 10, and 15 comparative questions in No, Mild, Moderate, and Severe OSA, respectively.

Table 2: 9-point intensity of relative weight (importance or well-being) scale (adapted from [1115]).
2.4. A Specialist Team

The questionnaire of AHP was taken to 3 sleep specialists. They were individually face-to-face interviewed by author to determine which groups of factors they thought to be more important when compared to another. Then, the decision data were collected.

2.5. The Construction and Consistency Check of Pairwise Comparison Matrix [17]

Step 1 (establishing the hierarchical structure (Figure 1)). Then, the decision-makers are requested to make pairwise comparisons between decision alternatives and criteria using a nine-point scale from Table 2. Subsequently, all matrices are developed and all pairwise comparisons will be obtained from each decision-maker.

Step 2 (calculating the consistency). To ensure that the priority of elements is consistent, the maximum eigenvector or relative weights and are calculated. Then, the consistency index (CI) for each matrix order using (1) is computed. Based on the CI and random index (RI), the consistency ratio (CR) is calculated by (2): where RI is the random consistency index obtained from a randomly generated pairwise comparison matrix. Table 3 shows the values of the RI for matrices of orders 1 to 15 [12]. If the value of CR is 0.1 or less, the pairwise comparisons will be considered as having an acceptable consistency.

Table 3: Random index (RI) [12].

Then, we construct a fuzzy pairwise comparison matrix in each criterion.

2.6. The Mathematics of Fuzzy Sets and Triangular Fuzzy Number (TFN) [18]

The fuzzy set theory is an effective instrument for modeling in the lack of comprehensive and accurate information. A TFN is a particular fuzzy set , and its membership function is a continuous linear function. A TFN is defined by its basic particular equation which is [19]where and correspond to the lower and upper bounds of the fuzzy number , respectively, and is the midpoint. A TFN is indicated as . Arithmetic operations between fuzzy numbers or a fuzzy number and crisp number have been defined elsewhere in Bulut and Zadeh [18, 19] by standard fuzzy arithmetic operations.

In this research, we use TFN to prioritize clinical risk factors of No, Mild, Moderate, and Severe OSA groups with fuzziness. A TFN is designated as .

Considering two TFNs, let and . Their operation laws are as follows [20]:

2.7. The Construction of Fuzzy Pairwise Comparison Matrix

Consider the following:where VM1, PM2, and NM3 are pairwise comparison matrix of each decision-maker and is the pairwise comparison score of each decision-maker . Integrating 3 decision-makers’ grades through (6) and yields TFN:

By this procedure, decision-makers’ pairwise comparison values are transformed into TFN. After forming fuzzy pairwise comparison matrix, weights of all factors are determined by FAHP method.

In this study, the extent FAHP which was originally introduced by Chang [20] is utilized. Let be an object set and a decision set. According to Chang’s extent analysis, each decision is taken and extent analysis for each goal is performed, respectively. Therefore, extent analysis values for each decision can be obtained with the following: If then .

2.8. Procedure of FAHP [2127]

Chang’s extent analysis can be performed in the following steps.

Step 1. The value of fuzzy synthetic extent with respect to the decision is defined asWe calculate fuzzy criteria weights of TFN of by (8).

Step 2. Let and be 2 TFNs; the degree of possibility of is defined asSince the height of a fuzzy set hgt () is the supremum (maximum) of the membership grades of , therefore,

Step 3. The degree of possibility for a convex fuzzy number to be greater than convex fuzzy numbers can be defined byAssume that for ; ; then the weight vector is given by where are elements.

Step 4. Via normalization, the normalized weight vectors arewhere is a nonfuzzy number. Then, weights of main criteria and attributes () can be calculated by

2.9. Diagnostic Test Evaluation of Sensitivity, Specificity, and 95% Confidence Interval (95% CI) [28, 29]

Sensitivity and specificity are statistical measures of the performance of a binary classification test. Sensitivity is the proportion of people with the target disorder in whom the test result is positive. Specificity is the proportion of people without the target disorder in whom test result is negative. To use these concepts, we divide test results into normal and abnormal to create a 2 × 2 table (Table 4):The 95% CI of a proportion is estimated based on the binomial theorem:where is the observed proportion and is the number of people observed.

Table 4: The relation between a diagnostic test result and occurrence of disease.
2.10. The Summarization of All Steps for the Prioritization of Clinical Risk Factors

See (Figure 2).

Figure 2: Flow chart for the prioritization of clinical risk factors.

3. Results

3.1. Decision-Makers’ Data

Questionnaires of AHP in No OSA and OSA group were taken to the 3 decision-makers, VM1, PM2, and NM3, to weight according to the significance to OSA. Main criteria of pairwise comparison matrices of No OSA and OSA groups by 3 sleep specialists (VM1, PM2, and NM3) are as follows. A1–A19 are clinical risk factors in each OSA severity.

No OSA

Mild OSA

Moderate OSA

Severe OSA

3.2. Results of Consistency Test

Consistency ratio for each specialist’s decision was calculated and checked. The results revealed that all CR values were ≤0.1. Thus, the consistency of all the decisions were satisfactory (Table 5).

Table 5: Consistency test for each sleep specialist’s decision.

From Table 5, the decision comparison matrices of the 3 sleep specialists in each severity group are then transformed into TFN by using (6) (Tables 69).

Table 6: Fuzzy pairwise comparison matrix for the clinical risk factors A1–A3 in No OSA group in terms of TFN.
Table 7: Fuzzy pairwise comparison matrix for the clinical risk factors A4–A8 in Mild OSA in terms of TFN.
Table 8: Fuzzy pairwise comparison matrix for the clinical risk factors A9–A13 in Moderate OSA in terms of TFN.
Table 9: Fuzzy pairwise comparison matrix for the clinical risk factors A14–A19 in Severe OSA in terms of TFN.
3.3. Procedure of Fuzzy Analytic Hierarchy Process (FAHP)

Step 1. We employ the calculation of fuzzy synthetic extents with respect to factors where the results of are calculated in detail (Table 10).

Table 10: Fuzzy criteria weights of TFN in No OSA and OSA group.

From the calculation, the weights of significance of decision factors in terms of triangle fuzzy number with lower, mean, and upper bounds in each OSA severity are obtained.

Step 2. We compare the values of , respectively, and calculate the degree of possibility of , yielding both values of and in all groups (Tables 1114).

Table 11: Degree of possibility of in No OSA.
Table 12: Degree of possibility of in Mild OSA.
Table 13: Degree of possibility of in Moderate OSA.
Table 14: Degree of possibility of in Severe OSA.

Step 3. The minimum degree of possibilities values of clinical risk factors in No OSA to Severe OSA are calculated by (11) as in Table 15.

Table 15: Minimum degree of possibilities value of each clinical risk factor in No OSA to Severe OSA.

Step 3 to Step 4. We obtain and of all criteria where is a nonfuzzy number as shown in Table 16.

Table 16: Weight vector () and normalized weight vector () of each clinical risk factor.

At this stage, factors affecting each level of OSA severity have been prioritized by using the FAHP methodology, which is a scientific procedure of multicriteria decision-making method. It can reflect effectively the human thoughts with vagueness of real world decision-making. The results of this research have finally provided the optimal factors.

3.4. Final Ranking, Choosing the Optimal Factors Sensitivity/Specificity and 95% CI

Next, we put them in an order from highest to lowest based on what the priority weight of each factor is and on their corresponding normalized weights vector. An optimal factor that has the highest score in a priority rating is selected. Finally, the overall sensitivity, specificity, and 95% CI of all OSA severities are calculated (Table 17).

Table 17: Normalized weight vector of each clinical risk factor, its ranking in OSA, and sensitivity/specificity with 95% CI of each severity group.

As can be seen in Table 17, in No OSA, first rank clinical risk factor is symptoms during sleep and the last one is underlying diseases and sleep posture. In Mild OSA, first rank clinical risk factor is choking and witnessed snoring and the last one is lung diseases and sleep-wake pattern. In Moderate OSA, first rank clinical risk factor is witnessed snoring and apnea plus awakening due to chest discomfort, whereas the last one is related personal variables. In Severe OSA, first rank clinical risk factor is witnessed snoring and apnea and the last one is underlying diseases and personal variables. It is observed that the sensibility and specificity of the approach to each group are high.

4. Discussion

Regarding the concept of factor analysis, each factor has its members (variable or symptom); whenever any variable is found, there is high tendency of the rest members to occur because they belong to the same factor. The FAHP methodology can provide the flexibility and robustness needed for the decision-maker to understand the decision problem as well as a standard control of consistency on the decision matrix for them. These merits of the approach lead to the developed FAHP questionnaire for detecting OSA patient in each level.

In Severe OSA group, the most related clinical risk factor is witnessed snoring and apnea that includes the witnessing of frequency of periodically stopped breathing and snoring as well as the intensity of the loudness of snoring. From Table 1 the Severe OSA patients have very high RDI (range 30.1–168.2 events/hour and mean ± S.D. = events/hour). Thus, in clinical practice, we propose that the appointment for the urgent sleep study and prompt management should include all clinical risk factors of Severe group (the sensitivity of 92.32% and specificity of 91.76%) (Table 18).

Table 18: The clinical risk factors and variables for urgent sleep study appointment.

In Moderate OSA, the overall sensitivity and specificity of the approach are 89.52% and 88.18%. Furthermore, the most at-risk factor is witnessed snoring and apnea plus chest discomfort as a cause of awakening during late night or getting up earlier than expectation. It should be noteworthy that apnea and snoring are observed in this Moderate group as well as in Severe group. Among 5 groups of factors in Moderate OSA, the first rank group carries extremely high normalized weight vector (0.481) compared to the remaining 4 groups (0–0.291). Therefore, in addition to the severe group the first rank clinical risk factors of Moderate OSA may be included in the urgent PSG appointment.

For the remaining clinical risk factors and variables in Moderate, Mild, and No OSA groups, they should be indicated for queueing up in the usual PSG waiting list.

In No OSA group, the details of symptoms during sleep included the troubles at night or during sleep within the last month, that is, someone’s notice (witness) of having periodically grasping hands and profound sweating during sleep. The frequencies of these symptoms are positively related to RDI [10]. The second running up clinical risk factor is Epworth Sleepiness Scale (2/8) and related variables. Epworth Sleepiness Scale (2/8), which is negatively related to RDI, still has no excessive daytime sleepiness even in sleeping induced atmosphere, that is, being a passenger in a car for an hour without break and sitting inactive in a public place. This is obvious evidence to separate No OSA group from Mild OSA (with positively correlated with RDI). The related variables with positive correlation to RDI are age, hypertension, and routine medicine use. Therefore, the elderly with hypertension and medicine routine use having no excessive daytime sleepiness, though having symptoms during sleep, should be really taken into account in No OSA group (sensitivity = 96.49%, specificity = 81.23%).

However, in the prioritization of Mild and No OSA group, the symptom during sleep in No OSA is at the borderline adjacent to Mild OSA. Herein, we propose that the screening for Mild from No OSA patients in sleep center setting may be implemented with the questionnaire which covers first rank clinical risk factor of No OSA group, that is, symptoms during sleep within the last month with the sensitivity of 87.12% and specificity of 72.22% (Table 19).

Table 19: The clinical risk factor and variables in screening for Mild from No OSA in sleep center setting.

To date, there have been no known previous studies concerning sleep questionnaire and anthropometrics data as the clinical risk factors for the prioritization of PSG appointment based on OSA severity.

Our future work will be planning to create the formula of clinical information for urgent sleep study appointment and screening of Mild OSA using fuzzy binary logistic regression equation approach.

5. Conclusion

Using FAHP based on normalized weight vectors to prioritize 19 clinical factors revealed that in each severity group based on RDI as No, Mild, Moderate, and Severe OSA, their first clinical risk factors are nighttime symptoms. Then, the prioritized factors are selected to propose the criteria for sleep study appointment. Therefore, the urgent sleep study appointments based on clinical risk factors of Severe OSA have been presented. In addition, the screening for Mild from No OSA patients in sleep center setting using symptoms during sleep is recommended. Finally, the questionnaires for these purposes can be constructed to cover the concerning factors.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The authors wish to thank Faculty of Medicine Siriraj Hospital for the cooperation and Associate Professor Nitipatana Chierakul and Assistant Professor Wattanachai Chotinaiwattarakul for their decision-making and all the patients whose data were contributed.

References

  1. N. M. Punjabi, “The epidemiology of adult obstructive sleep apnea,” Proceedings of the American Thoracic Society, vol. 5, no. 2, pp. 136–143, 2008. View at Publisher · View at Google Scholar · View at Scopus
  2. E. O. Bixler, A. N. Vgontzas, H.-M. Lin et al., “Prevalence of sleep-disordered breathing in women: effects of gender,” The American Journal of Respiratory and Critical Care Medicine, vol. 163, no. 3, part 1, pp. 608–613, 2001. View at Publisher · View at Google Scholar · View at Scopus
  3. T. Young, P. E. Peppard, and D. J. Gottlieb, “Epidemiology of obstructive sleep apnea: a population health perspective,” The American Journal of Respiratory and Critical Care Medicine, vol. 165, no. 9, pp. 1217–1239, 2002. View at Publisher · View at Google Scholar · View at Scopus
  4. S. Sompradeekul, N. Maranetra, C. Naruman, and A. Nana, “The prevalence of obstructive sleep apnea among professional hospital drivers,” Siriraj Hospital Gazette, vol. 54, pp. 625–643, 2002. View at Google Scholar
  5. G. Pillar and N. Shehadeh, “Abdominal fat and sleep apnea: the chicken or the egg?” Diabetes Care, vol. 31, supplement 2, pp. S303–S309, 2008. View at Publisher · View at Google Scholar · View at Scopus
  6. A. Sanna, “Obstructive sleep apnoea, motor vehicle accidents, and work performance,” Chronic Respiratory Disease, vol. 10, no. 1, pp. 29–33, 2013. View at Publisher · View at Google Scholar · View at Scopus
  7. E. Shahar, C. W. Whitney, S. Redline et al., “Sleep-disordered breathing and cardiovascular disease: cross-sectional results of the sleep heart health study,” The American Journal of Respiratory and Critical Care Medicine, vol. 163, no. 1, pp. 19–25, 2001. View at Publisher · View at Google Scholar · View at Scopus
  8. P. Ruttanaumpawan, C. Nopmaneejumruslers, A. G. Logan, A. Lazarescu, I. Qian, and T. D. Bradley, “Association between refractory hypertension and obstructive sleep apnea,” Journal of Hypertension, vol. 27, no. 7, pp. 1439–1445, 2009. View at Publisher · View at Google Scholar · View at Scopus
  9. A. Tarasiuk and H. Reuveni, “The economic impact of obstructive sleep apnea,” Current Opinion in Pulmonary Medicine, vol. 19, no. 6, pp. 639–644, 2013. View at Publisher · View at Google Scholar · View at Scopus
  10. T. Maranate, A. Pongpullponsak, and P. Ruttanaumpawan, “Symptoms and anthropometric data related to normals and severity of obstructive sleep apnea patients by using factor analysis,” in Proceedings of the International Conference on Applied Statistics (ICAS '14), pp. 126–136, Khon Kaen,Thailand, 2014.
  11. T. L. Satty, Decision Making for Leaders, Lifetime Learning, New York, NY, USA, 1982.
  12. T. L. Saaty, The Analytical Hierarchy Process: Planning, Priority Setting, Resource Allocation, McGraw-Hill, New York, NY, USA, 1980. View at MathSciNet
  13. J. A. Alonso and M. T. Lamata, “Consistency in the analytic hierarchy process: a new approach,” International Journal of Uncertainty, Fuzziness and Knowlege-Based Systems, vol. 14, no. 4, pp. 445–459, 2006. View at Publisher · View at Google Scholar · View at Scopus
  14. C.-F. Chen, “Applying the analytical hierarchy process (AHP) approach to convention site selection,” Journal of Travel Research, vol. 45, no. 2, pp. 167–174, 2006. View at Publisher · View at Google Scholar · View at Scopus
  15. M. Akalın, G. Turhan, and A. Sahin, “The application of AHP approach for evaluating location selection elements for retail store: a case of clothing store,” International Journal of Research in Bussiness and Social Science, vol. 2, no. 4, pp. 1–19, 2013. View at Google Scholar
  16. C. Iber, S. Ancoli-Israel, A. Chesson, and S. F. Quan, AASM Manual for Scoring of Sleep and Associated Event, American Academy of Sleep Medicine, Westchester, Ill, USA, 2007.
  17. C.-W. Hsu, A. Hu, and W.-C. Wu, “Using FMEA and FAHP to risk evaluation of green components,” in Proceedings of the 16th IEEE International Symposium on Electronics and the Environment (ISEE '08), pp. 1–6, San Francisco, Calif, USA, May 2008. View at Publisher · View at Google Scholar · View at Scopus
  18. E. Bulut, O. Duru, T. Keçeci, and S. Yoshida, “Use of consistency index, expert prioritization and direct numerical inputs for generic fuzzy-AHP modeling: a process model for shipping asset management,” Expert Systems with Applications, vol. 39, no. 2, pp. 1911–1923, 2012. View at Publisher · View at Google Scholar · View at Scopus
  19. L. A. Zadeh, “Fuzzy sets,” Information and Computation, vol. 8, pp. 338–353, 1965. View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  20. D.-Y. Chang, “Applications of the extent analysis method on fuzzy AHP,” European Journal of Operational Research, vol. 95, no. 3, pp. 649–655, 1996. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus
  21. S. Lee, G. Mogi, S. Lee, and J. Kim, “Prioritizing the weights of hydrogen energy technologies in the sector of the hydrogen economy by using a fuzzy AHP approach,” International Journal of Hydrogen Energy, vol. 36, no. 2, pp. 1897–1902, 2011. View at Publisher · View at Google Scholar · View at Scopus
  22. M. Dağdeviren and I. Yüksel, “Developing a fuzzy analytic hierarchy process (AHP) model for behavior-based safety management,” Information Sciences, vol. 178, no. 6, pp. 1717–1733, 2008. View at Publisher · View at Google Scholar · View at Scopus
  23. Y. M. Wang, Y. Luo, and Z. Hua, “On the extent analysis method for fuzzy AHP and its applications,” European Journal of Operational Research, vol. 186, no. 2, pp. 735–747, 2008. View at Publisher · View at Google Scholar · View at Scopus
  24. C.-W. Chang, C.-R. Wu, and H.-C. Chen, “Using expert technology to select unstable slicing machine to control wafer slicing quality via fuzzy AHP,” Expert Systems with Applications, vol. 34, no. 3, pp. 2210–2220, 2008. View at Publisher · View at Google Scholar · View at Scopus
  25. O. Meixner, Fuzzy AHP Group Decision Analysis and Its Application for the Evaluation of Energy Sources, Institute of Marketing and Innovation, University of Natural Resources and Applied Life Sciences, Vienna, Austria, 2009.
  26. D. Chatterjee and B. Mukherjee, “Study of fuzzy-AHP model to search the criterion in the evaluation of the best technical institutions: a case study,” International Journal of Engineering Science and Technology, vol. 2, no. 7, pp. 2499–2510, 2010. View at Google Scholar
  27. G. Büyüközkan, G. Çifçi, and S. Güleryüz, “Strategic analysis of healthcare service quality using fuzzy AHP methodology,” Expert Systems with Applications, vol. 38, no. 8, pp. 9407–9424, 2011. View at Publisher · View at Google Scholar · View at Scopus
  28. R. Jaeshke, G. H. Guyatt, and D. L. Sackett, “Users' guide to an article about a diagnostic test. A. Are the results of the study valid?” The Journal of the American Medical Association, vol. 271, pp. 389–391, 1994. View at Publisher · View at Google Scholar
  29. R. H. Fletcher, S. W. Flether, and E. H. Wagner, Clinical Epidemiology. A Basic Science for Clinical Medicine, Williams & Wilkins, Baltimore, Md, USA, 2nd edition, 1988.