Application and Decision-Making Implications of Novel Optimization Models in Public HealthView this Special Issue
Capacity Evaluation of Diagnostic Tests For COVID-19 Using Multicriteria Decision-Making Techniques
In December 2019, cases of pneumonia were detected in Wuhan, China, which were caused by the highly contagious coronavirus. This study is aimed at comparing the confusion regarding the selection of effective diagnostic methods to make a mutual comparison among existing SARS-CoV-2 diagnostic tests and at determining the most effective one. Based on available published evidence and clinical practice, diagnostic tests of coronavirus disease (COVID-19) were evaluated by multi-criteria decision-making (MCDM) methods, namely, fuzzy preference ranking organization method for enrichment evaluation (fuzzy PROMETHEE) and fuzzy technique for order of preference by similarity to ideal solution (fuzzy TOPSIS). Computerized tomography of chest (chest CT), the detection of viral nucleic acid by polymerase chain reaction, cell culture, CoV-19 antigen detection, CoV-19 antibody IgM, CoV-19 antibody IgG, and chest X-ray were evaluated by linguistic fuzzy scale to compare among the diagnostic tests. This scale consists of selected parameters that possessed different weights which were determined by the experts’ opinions of the field. The results of our study with both proposed MCDM methods indicated that the most effective diagnosis method of COVID-19 was chest CT. It is interesting to note that the methods that are consistently used in the diagnosis of viral diseases were ranked in second place for the diagnosis of COVID-19. However, each country should use appropriate diagnostic solutions according to its own resources. Our findings also show which diagnostic systems can be used in combination.
After cases of pneumonia of unknown cause were detected in Wuhan, China, in December 2019, a new coronavirus was isolated from human airway epithelial cells and was named severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which is responsible for coronavirus disease (COVID-19) . SARS-CoV-2 is also a member of the coronavirus family that includes Middle East Respiratory Syndrome- (MERS-) CoV and SARS-CoV, which infect humans [1, 2]. Wild animals are the source of the infection.
According to phylogenetic analysis of full genome sequencing, the coronavirus that causes COVID-19 is a betacoronavirus in the same subgenus clade as SARS-CoV-2. The structure of the receptor binding site for cell entry is similar and uses the angiotensin-converting enzyme 2 receptor found in the epithelial cells of the alveoli used by SARS-CoV-2 . The International Committee on Taxonomy of Viruses has proposed that this virus be designated as SARS-CoV-2 [3, 4].
The main mode of transmission is via person-to-person spread. When an infected person coughs, sneezes, or speaks, the virus released in respiratory secretions can infect another person if it comes into direct contact with the mucous membranes through droplet delivery. Also, infection can occur if a person touches an infected surface and then their eyes, nose, or mouth . Infected but asymptomatic people can transmit the virus to others .
The most common serious sign of infection is pneumonia: it is characterized by fever, cough, shortness of breath, and bilateral infiltrates in chest imaging [7–9]. In severe cases, patients can quickly experience acute respiratory syndrome, septic shock, metabolic acidosis, and coagulopathy [10, 11]. The mortality rate is 1-2%, but this rate may increase up to 14%, especially in elderly patients with comorbidities such as hypertension, diabetes mellitus, or cardiovascular diseases . Due to the strong infectivity of SARS-CoV-2, it is necessary to identify, isolate, and treat patients as soon as possible, which can reduce mortality rates while reducing the risk of public contamination. In order to be able to treat patients, it is imperative that the disease is diagnosed quickly and accurately. The diagnosis is based on real-time reverse transcription-polymerase chain reaction (rRT-PCR) positivity for the presence of coronavirus . With nucleic acid isolation processing, rRT-PCR results usually require 5 to 6 hours. In addition, it remains unclear whether rRT-PCR is the gold standard and whether false-positive or false-negative results are common. The Centers for Disease Control and Prevention recommends that nasopharyngeal and oropharyngeal swab specimens should be collected to test for SARS-CoV-2 . Although false-positive tests are generally possible, a positive test for SARS-CoV-2 confirms the diagnosis of COVID-19. False-negative results can be obtained from the upper respiratory samples, so if the first test is negative and the patient continues to suspect COVID-19, it is recommended that the test be repeated . Serological tests, on the other hand, can be accessed and evaluated more easily and can identify patients with existing or previous infections but who have negative rRT-PCR tests . These tests can also be used because they are easier to access in places that do not have access to the rRT-PCR test. The technique of the test is quite simple. It gives results in a short time . Although samples taken from the patient are not evaluated for viral culture, they do have diagnostic value .
The other methods that are used in the diagnosis of COVID-19 are imaging methods. The diagnostic sensitivity of viral pneumonia by chest radiography is relatively low , whereas computerized tomography (CT) has high sensitivity for diagnosis of COVID-19 which makes it a primary tool for COVID-19 detection in epidemic areas . In COVID-19 patients, the sensitivity of chest CT scan is 97% but its specificity is 25% with rRT-PCR results as reference, which is why it is not used as a primary screening test .
Chest CT scan abnormalities have also been identified in patients prior to the development of symptoms and even prior to the detection of viral RNA from upper respiratory specimens [20, 21]. In the article published by Huang et al. in the Lancet, it was stated that it is necessary to have signs of infection and/or a positive nucleic acid test for chest CT scan to be used in diagnosis; otherwise, CT should not be recommended for screening or early diagnosis . They said that CT should not be recommended for screening or early diagnosis. However, when the rRT-PCR results of patients were considered to be clinically and epidemiologically negative for COVID-19, chest CT scan became more valuable than rRT-PCR in the early stage of the disease. Shi et al. showed that combining the evaluation of chest CT scan imaging features with clinical and laboratory findings may facilitate the early diagnosis of COVID-19 pneumonia [20, 23].
Thus far, seven different diagnostic tests have been used for COVID-19: rRT- PCR, cell culture, CoV-19 antigen detection, serological tests (CoV-19 antibody IgM, CoV-19 antibody IgG), chest X-ray, and chest CT. From the onset of the pandemic, real-life experience has shown that rRT-PCR and chest CT are the most preferred methods for the diagnosis of COVID-19. The suitability of SARS-CoV-2 diagnostic tests can be prioritized with the parameters of high sensitivity, high specificity, low false positivity, low false negativity, high usability, low cost, etc. In this study, we used the fuzzy preference ranking organization method for enrichment evaluation (fuzzy PROMETHEE) and fuzzy technique for order of preference by similarity to ideal solution (fuzzy TOPSIS) techniques to compare the confusion regarding the selection of effective diagnostic methods to make a mutual comparison between the aforementioned seven SARS-CoV-2 diagnostic tests and hence determine the best one. We have decided to use fuzzy PROMETHEE because fuzzy part can handle fuzzy data very well, and PROMETHEE part can handle the data when there are too many parameters to be set properly. One other advantage of using PROMETHEE is that it is a user-friendly outranking method and successfully adapted to real-life problems. Fuzzy TOPSIS, on the other hand, is simple, rational, and comprehensive, as well as able to measure relative performance of alternatives.
2. Materials and Methods
Fuzzy PROMETHEE and fuzzy TOPSIS techniques, which are widely used by researchers as MCDM methods, were applied for the evaluation of the SARS-CoV-2 diagnostic tests in order to provide the best alternatives. The fuzzy PROMETHEE technique is a hybrid model that provides decision analysis for nonnumerical data and also enables decision-makers to define vague conditions mathematically . The PROMETHEE technique was developed by Brans et al. [25, 26], and it is aimed at giving a comprehensive ranking for a finite set of alternatives corresponding to their criteria and the importance level of each criterion. By providing various preference functions, including V-shaped function, level function, Gaussian function, U-shaped function, and linear function, to the criteria for the comparison of the alternatives, the PROMETHEE technique differs from the other MCDM methods and can provide more sensitive results. The PROMETHEE and TOPSIS methods are unable to analyze fuzzy data (linguistic or vague data) in the actual decision-making environment. Thus, we have applied the fuzzy-based PROMETHEE and TOPSIS techniques, which are the hybrid methods that use a fuzzy scale. The fuzzy scale supports the decision-maker in decision analysis if the data is within a range, vague, or qualitative.
In this study, a linguistic fuzzy scale has been used to define the parameters of the SARS-CoV-2 diagnostic tests. These values were converted to triangular fuzzy numbers, and then the Yager index was applied for defuzzification of the data. Then, the PROMETHEE decision lab program with a Gaussian preference function was applied for the determination of the net ranking result.
The PROMETHEE method comprises 5 steps that are applied for the MCDM analysis and to rank the alternatives as follows [25–28]: (i)The preference function of each criterion should be defined(ii)Importance weights of each criterion , where should be defined as where is the number of criteria.(iii)For each of the alternative pairs , , the outranking relation should be determined by where denotes the value of the th criterion of the alternative and denotes the preference indices, which show the preference intensity for an alternative in comparison to an alternative while counting all criteria simultaneously.(iv)The positive outranking flow and negative outranking flow should be determined as follows: (a)A positive outranking flow of the alternative :(b)A negative outranking flow of the alternative : denotes the number of alternatives. The defines the strength of alternative , while the negative outranking flow defines the weakness of alternative .
PROMETHEE I determines a partial preorder of the alternatives based on the positive and negative outranking flows, and PROMETHEE II determines a complete preorder of the alternatives based on a net flow. The partial preorder of the options can be determined based on the following statements:
Via PROMETHEE I, alternative is selected to alternative if it satisfies either of the statements given below as
is indifferent to alternative if
And is incomparable to if
The net outranking flow can be calculated for each alternative by using the following:
Via PROMETHEE II, the complete order with net flow can be determined as
The higher value provides the better alternative.
In this study, triangular fuzzy scale was used for defining the linguistic data of the criteria of SARS-CoV-2 diagnostic tests as shown in Table 1. Zadeh  has defined the fuzzy sets in order to define the linguistic or vague data mathematically based on the membership degree. Triangular fuzzy sets can be represented with 3 specific points (, , and ) where is the initial point (left bound), is the peak point of the triangle which contain the maximum membership degree, and is the right bound of the triangle. The linguistic fuzzy scale indicates the linguistic terms of very high, high, medium, and low with their associated fuzzy sets. Furthermore, the importance level of each has been defined with the triangular fuzzy scale by the experts for analyzing the effectiveness criteria of SARS-CoV-2 diagnostic tests. The criteria selected for the analysis include cost, accessibility, sensitivity, and specificity.
For the validation of the fuzzy-based PROMETHEE output, fuzzy TOPSIS, a different type of MCDM technique, has been used for the same dataset. TOPSIS method was first proposed by Yoon and Hwang  and commonly applied for the decision-making problems of the conflicting criteria [31, 32] in wide disciplines. It compares a set of alternatives by identifying weights for each criterion, normalizing scores for each criterion and calculating the geometric distance between each alternative and the ideal alternative, which is the best score in each criterion. It based on the concept that the chosen alternative should have the shortest geometric distance from the positive ideal solution (PIS) and the longest geometric distance from the negative ideal solution (NIS).
There are 6 steps of the TOPSIS method to be applied for the MCDM analysis and rank the alternatives as follows . (1)The decision matrix which includes the parameters (criteria) of the alternatives and the importance weight of the criteria, , should be defined as where is the number of criteria.(2)The normalized decision matrix should be calculated by one of the equations below: where is the value of the th alternative and th criterion.(3)The values of the weighted normalized matrix should be calculated by applying the following: (4)And positive ideal solution and negative ideal solution should be determined by using Equation (13) and Equation (14), respectively where and are associated with benefit criteria and cost criteria, for Positive ideal solution is the set of the values that maximizes the benefit criteria while minimizing the cost criteria. Negative ideal solution can be considered an opposite of the positive ideal solution. In the negative ideal solution, the benefit criterion is minimized while the cost criterion is maximized. According to the TOPSIS method, the most suitable option is the one which is closer to the positive ideal solution and further to the negative ideal solution .(5)Then, the separation measures between alternatives and the positive ideal solution should be calculated by using Equation (15) and the separation measures between alternatives and the negative ideal solution should be calculated by using Equation (16) in order to obtain the distance between the th alternative and the positive ideal solution and the distance between the th alternative and the negative ideal solution as shown below: where .(6)Then, the relative closeness to the positive ideal solution should be calculated for every alternative by using Equation (17) as seen below: where , .
Positive and negative ideal solution sets of the SARS-CoV-2 diagnostic tests have been obtained as shown in Table 2 below.
In TOPSIS technique, relative closeness to the positive ideal solution determines the ranking results of the alternatives. The higher is a more preferred option. The weighted normalized dataset of the SARS-CoV-2 diagnostic tests can be seen in Table 3.
3. Results and Discussion
Table 4 indicates the results of the complete ranking for the SARS-CoV-2 diagnostic tests, where each alternative pair is compared numerically based on each criterion and their importance weights. The power of each alternative can be thought numerically as the positive outranking flow, while the weakness of the alternatives can be thought as the negative outranking flow. However, the net flow gives the net ranking results. The most effective alternative is the one with the higher net flow. With the maximum positive outranking flow and minimum negative outranking flow, the chest CT is the best available SARS-CoV-2 diagnostic test, followed by naso/oropharyngeal swab rRT-PCR. Even if naso/oropharyngeal swab rRT-PCR has a lower positive flow than cell culture, it also has lower negative flow than the cell culture. Therefore, the second-best option is naso/oropharyngeal swab rRT-PCR with 0.0258 net flow, and the third one is the cell culture with the 0.0098 net flow. The last two options are CoV-19 antigen detection and chest X-ray as seen in Table 4.
The strength and the weakness of the available SARS-CoV-2 diagnostic tests are shown in Figure 1. The criteria for each SARS-CoV-2 diagnostic tests are listed above or below the zero-threshold level based on their effect on the performance of the alternatives. If the criteria are in advantage of the technique, they are listed above, and if the criteria are in disadvantage of the technique, they are listed below the zero-threshold level. As represented, the most preferred technique is the chest CT, with almost all the criteria listed above the threshold level, while the last effective technique for the SARS-CoV-2 diagnosis is the chest X-ray, with most of the criteria listed above the threshold level. These results can be useful for hospitals, doctors, governments, and patients or anyone involved in the decision-making processes of the SARS-CoV-2 diagnosis.
The SARS-CoV-2 diagnostic tests ranking results obtained with fuzzy TOPSIS have been presented in Table 5. With the 0.6403 value of the closeness to the ideal solution , chest CT is the best alternative, naso/oropharyngeal swab rRT-PCR is the second best alternative with 0.5865 , and the cell culture with 0.5718 is the third best option between the alternatives. The CoV-19 antigen detection and chest X-ray were found as the least effective options.
These results are consistently similar with the fuzzy-based PROMETHEE outputs. Hence, the results from both techniques could give evidence to the decision-makers about the effectiveness of the available SARS-CoV-2 diagnostic tests.
According to the findings of a study conducted with more than 1,000 patients by Ai et al. , CT should be the primary diagnosis tool used to screen for COVID-19. However, their finding of 97% sensitivity for the chest CT was based on positive rRT-PCR test results. Chest CT scan for COVID-19 diagnosis can be a good complement to rRT-PCR . According to the study of Fang et al. , the sensitivity of CT for COVID-19 was higher compared to rRT-PCR (98% vs. 71%, respectively). Similarly, we also showed mathematically that CT should be the first preferred diagnostic test. Furthermore, differentiating COVID-19 from non-COVID-19 pneumonia on chest CT, which is a relatively harder task, was studied in , and the authors reported that the findings of CT are considered nonspecific despite its high sensitivity. In addition, one of the greatest issues faced is the need for an expert radiologist to interpret the radiography images. This procedure is even more difficult when attempting to differentiate COVID-19 from other types of pneumonia since the visual indicators can be subtle .
Therefore, the main limitation of our study is the lack of analysis of the combination of different diagnostic tests. Hence, in future studies, once the available techniques for diagnosing COVID-19 are more robust, we will apply the MCDM methods to the combination of different diagnostic tests. In addition, it is sensible to apply the MCDM methods for specific countries, regions, or even hospitals considering specific conditions such as their available diagnostic toolkits, scanners, and experts.
This study analyzed the seven selected available COVID-19 diagnostic tests using the fuzzy-based MCDM methods. The evaluations of the effectiveness of the alternatives were made based on the selected criteria, and their corresponding weights were determined by the experts. As a result of the evaluation, chest CT was found to be the most effective diagnostic test. Moreover, the results indicated that the top three most effective COVID-19 diagnostic tests are chest CT, rRT-PCR, and cell culture. On the other hand, chest X-ray is the last effective diagnostic test. It is interesting to note that the methods that are always used in the diagnosis of viral diseases were ranked second for the diagnosis of COVID-19.
In this study, during the process of evaluating the effectiveness of the given COVID-19 diagnostic tests, each technique was considered and compared based on their general properties collected from the latest available guidelines. The use of fuzzy-based MCDM methods can be also adapted based on the patient’s specific condition.
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
A. E. Gorbalenya, S. C. Baker, R. S. Baric et al., Severe acute respiratory syndrome-related coronavirus: the species and its viruses – a statement of the Coronavirus Study Group, bioRxiv, 2020.View at: Publisher Site
World Health Organization, Novel coronavirus situation reports -2, 2020, March 2020, https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200122-sitrep-2-2019-ncov.pdf.
A. Patel, D. B. Jernigan, 2019-nCoV CDC Response Team et al., “Initial Public Health Response and Interim Clinical Guidance for the 2019 Novel Coronavirus Outbreak — United States, December 31, 2019–February 4, 2020,” Morbidity and Mortality Weekly Report, vol. 69, no. 5, pp. 140–146, 2020.View at: Publisher Site | Google Scholar
World Health Organization, “Coronavirus disease (COVID-19) technical guidance: surveillance and case definitions,” March 2020, https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/surveillance-and-case-definitions.View at: Google Scholar
D. Uzun Ozsahin and I. Ozsahin, “A fuzzy PROMETHEE approach for breast Cancer treatment techniques,” International Journal of Medical Research & Health Sciences, vol. 7, no. 5, pp. 29–32, 2018.View at: Google Scholar
J. P. Brans, B. Mareschal, and P. Vincke, “PROMETHEE: a new family of outranking methods in MCDM,” Oper Res IFORS 84, pp. 477–490, 1984.View at: Google Scholar
C. L. Yoon and K. Hwang, Multiple Attribute Decision Making: Methods and Applications, Springer-Verlag, New York, NY, USA, 1981.