New Trends and Recent Advances in Operation Research on Supply ChainsView this Special Issue
Evaluating the Performance of Emergency Centers during Coronavirus Epidemic Using Multi-Criteria Decision-Making Methods (Case Study: Sari City)
In this study, due to the importance of emergency centers and patient transport vehicles in epidemic conditions, the performance of emergency centers has been evaluated based on health protocols. The criteria were first divided into preventive and operational sections by collecting opinions, health experts, standard criteria, and the Delphi method. Preventive criteria for evaluating emergency centers and operating criteria for assessing vehicles in these centers are considered. The weighting of the determined criteria was done by the triangular fuzzy aggregation method. According to the standard criteria, the emergency centers have been evaluated for a 30-day period. The results have been assessed as a qualitative and quantitative matrix using the PROMETHEE method. The results showed better performance of Center A (63%) due to proper performance and better compliance with protocols in both criteria (preventive and operational). The reason for the superiority of this center over Center B can be considered the better performance of this center in terms of prevention indicators and better performance of the center’s vehicles (Ambulance A-1 and Ambulance A-2) in the performance index by observing the standards.
In late 2019, a disease emerged in Wuhan, China, with symptoms similar to pneumonia and acute respiratory illness . After a few months, it became a global epidemic . This virus is known as the acute coronavirus syndrome (SARS-CoV-2) or COVID-19 because it belongs to the large family of coronavirus . The World Health Organization has issued guidelines to prevent the spread of COVID-19 as much as possible, such as banning unnecessary travel, closing public and busy centers, quarantine, social distance, and use of personal protective equipment (PPE) . The virus has challenged public health . So far, more than 1.6 million people have lost their lives, a significant number of whom are health workers . Due to the COVID-19 epidemic, much pressure was put on health workers, the first of which was high work pressure and psychological factors [7, 8]. The second was employees’ pressure to get the virus due to a lack of personal protective equipment (PPE). In some cases, asymptomatic people have been found to carry the COVID-19 . Therefore, it makes it harder to fight the disease, affecting the health of a community, health workers, and even the families of health workers . Hospitals are among the most sensitive centers against COVID-19; weaknesses such as lack of facilities in these centers cause more pressure on the whole community. As a result, the number of patients and deaths of this disease increases dramatically . The emergency department (ED) has delinquent equipment such as patient transport vehicles, various separate rooms such as isolated rooms, pre-rooms, equipment, and facilities for patients’ care and treatment, and having knowledgeable and sufficient staff . In the COVID-19 epidemic, it is essential to have well-equipped patient transport vehicles separate from other patients . Ambulance crews, such as isolation ward staff, should be in good mental condition and have adequate equipment (such as PPE) and equipment needed for patients . In the next step, this is the emergency department of each hospital, responsible for patient care and maintenance. The emergency department’s other important and main task is to save patients’ lives in other wards and protect staff’s lives . Due to this virus’s unpredictability, which causes delinquent symptoms and problems in delinquent individuals due to their age and immune system status, having adequate equipment and staff with sufficient information can be a good way to reduce COVID-19 be disease mortality . The COVID-19 pandemic occurred suddenly and unexpectedly in many countries . The evolution of COVID-19 and the unpredictability of the virus and the inconsistencies of health systems worldwide have made it difficult to combat the epidemic and achieve safety [17, 18]. This epidemic has challenged public health . The virus was initially thought to be transmitted only through people with certain symptoms. Still, with continuous transmission of the disease by asymptomatic people who carry the virus, the constant presence of health workers in the emergency departments is mandatory. Because this constant presence means that the emergency department staff is always close to the person with COVID-19, it increases the risk of staff contracting the disease, causing the disease to become more prevalent and even endangering staff families, so about 50% of emergency department staff have symptoms such as depression, fear, and anxiety about expressing themselves, and research shows that about 10%–20% of people with COVID-19 are health workers, especially in the emergency department (ICU) [20–23]. Given these points, the continued use of personal protective equipment (PPE) is essential for both health workers and the general public to reduce this disease’s incidence. Still, excessive use of PPE by the community has reduced the PPE standard available to health workers . Due to the importance of emergency centers and patient transport vehicles in epidemic conditions and due to the increase in virus transmission rate among emergency personnel and patients compared with normal conditions, in this study, the evaluation of the performance of emergency centers during the epidemic has been considered. Given the importance of the research topic, the answer to the question of whether the emergency centers (emergency centers considered in the study area) have adapted to the epidemic conditions? Do they function well during an epidemic? To answer this question, in this research, two emergency centers in Sari will be examined in terms of preventive and operational criteria. In this study, considering the essential role that emergency department equipment plays in the fight against this disease, to use multicriteria decision-making methods, evaluates the performance of emergencies and patient transport vehicles according to compliance with health protocol times. The rest of the study is organized as follows: the literature review will be explained in the second section. In the third section, the research method will be described. In this section, the weighting methods of the criteria and their results will be given. In the fourth and fifth sections, sampling methods and the results of these evaluations will be explained. The discussion section is presented in the sixth section. Finally, the conclusion will be stated in the seventh section.
2. Literature Review
With the introduction of COVID-19 in China and its spread to all continents of the world, much research has been done on various aspects of the virus. Zhu et al.  evaluated and prioritized patients whose treatment was delayed for any reason; this study is multicriteria decision-making (MCDM) problem. They used two methods DEMATEL and VIKOR because both quantitative and qualitative criteria are included in the study. Both ways showed their efficiency. Spoorthy et al.  examined the mental health problems of healthcare workers during the epidemic. Factors such as age, insomnia, occupation, and increased stress are related to employees’ mental health. Ng et al.  demonstrated the importance of health workers using personal protective equipment (PPE) during the COVID-19 epidemic. Employees who were two meters or less away from patients are considered at high risk of this disease. Nguyen et al.  who given the importance of using (PPE) by health workers and who are at the forefront of the fight against COVID-19 to the general public determined the needs. The results showed that the healthcare system and providing sufficient resources for PPE should also hire more staff. Amin  using 250 questionnaires distributed among health workers and physicians examined the psychological effects of COVID-19 in the community. According to the results, the more people are aware of COVID-19 and the criteria for its psychological effects, and the more its psychological effects will be reduced. Jahantigh and Ostovare  using the organizational preferential ranking method and PROMETHHE II examined 47 influential factors to evaluate teaching hospital sections’ performance; this evaluation was performed on teaching hospitals in Tehran, Iran. The results show that a significant number of teaching hospitals are inefficient. Liao et al.  by creating a framework of aggregation method based on normalization solved the screening problem, obtained critical screening factors from the fuzzy Delphi method, and analyzed the proposed method. Choukolaei et al.  by determining 25 quantitative-qualitative criteria by experts by the Delphi method evaluated isolation room and anteroom of coronavirus hospitals in Sari. They prioritized the options using the fuzzy method. SWARA-PROMETHEE is used. Yucesan and Gul  reviewed and prioritized hospitals in terms of services provided, using fuzzy AHP and fuzzy TOPSIS, and they presented this study. The hospitals in question are located in Turkey, and the results, while showing the effectiveness of these two methods, also identified the top hospitals. Kriksciuniene and Sakalauskas  ranked hospitals based on the number of beds, patients, and cost. Their ranking method was the regression method and compared with each other using an analytic hierarchy process.
In this study, due to the importance of emergency health personnel and ambulance personnel using PPE during the COVID-19 epidemic and reducing health workers’ mortality and maintaining their mental health, the emergency services are performed according to the use of PPE. To obtain the criteria and subcriteria, the fuzzy Delphi method is used in which experts have been used to evaluate these centers. For weighting and determining the questionnaire according to the criteria, the triangular fuzzy aggregation method was divided into preventive factors. Operating characteristics were determined using Morgan’s table sampling and field research. Finally, the emergencies were ranked through the PROMETHEE method. Lotfi et al.  have developed a robust regression-based optimization (RO) approach to effectively predict the number of patients with recently confirmed coronavirus infection (COVID-19). The mean and average of the uncertain parameters have been calculated using the conditional value at risk (CVaR). Sensitivity was examined. Finally, their proposed model had the lowest mean absolute deviation (MAD) and the highest correlation coefficient compared with other models.
Lotfi et al.  to improve the inventory management system and deal with uncertainty and disruption in COVID-19 conditions proposed three models of resilience and sustainable healthcare supply chain (RSHCSC) using strong fuzzy and data-driven optimization with a random planning approach. The mean and average of the uncertain parameters have been calculated using the conditional value at risk (CVaR). The results show that increasing the fuzzy shear, confidence level, robustness coefficient, flexibility coefficient, and CVaR confidence level increase the cost.
Modibbo et al.  by presenting a mixed-integer linear programming model using a multicriteria decision-making process, the best qualified suppliers in the pharmaceutical industry are examined and a model for supplier selection problems is proposed. They reduce the size of the data from the concept of analysis. They also used the principal component analysis (PCA). They also tested the concept of triangular fuzzy number (TFN) on the reliability of decision-makers (DMs).
Navaei et al.  by presenting a multi-cycle and multiproduct model for hospital evacuation and drug supply chain for critical patients and considering the Epsilon constraint method compared and evaluated the performance of solution methods.
Moreira et al.  to predict and identify infected patients in epidemic conditions analyzed machine learning algorithms. They analyzed and evaluated the options using the PROMETHEE-GAIA method. Finally, they made a clear analysis of the selection of pre-algorithms. Nose was introduced to fight COVID-19.
Martins et al.  using three multicriteria decision-making techniques evaluated the main preventive measures in COVID-19 administrative services. They used PROMETHEE I, PROMETHEE II, and ELECTRE III techniques to prioritize preventive measures. Analysis of the results shows that the use of masks, gel alcohol along with physio-social distance and training were the most effective ways to prevent the outbreak of COVID-19. The literature review table is as follows (Table 1):
The research contribution is as follows:(i)Using the Delphi method to collect and classify criteria and using the TFN-PROMETHEE technique to evaluate options during epidemic(ii)Applying preventive and operational criteria to assess emergencies during an epidemic(iii)Considering preventive and operational criteria simultaneously
This study aimed to evaluate the performance of emergency services in Sari, Iran, following health protocols’ observance.
3. Research Methodology
The information required for this study was collected through the library, documentary, previous studies, and field studies. It was identified and selected with health experts’ cooperation and the Delphi method subcriteria and the criteria. The experts then weighed these criteria using the triangular fuzzy aggregation method. A questionnaire including operational and preventive questions consisted of ten questions, and the importance of each criterion was prepared. Using Morgan’s table and considering each emergency’s average daily operations, the number of samples from the patient car’s missions was 52 samples per day. The vehicles carrying each emergency station (each containing two vehicles) were evaluated in a month by daily sampling. The average performance of each of these indicators and the quality evaluation questionnaire of these databases are considered as input information. Finally, the performance of each emergency center was evaluated using the PROMETHEE method. Figure 1 shows an overview of the research.
The advantages of the methods used in this research are as follows:(i)Delphi Method: The Delphi method uses a questionnaire to collect ideas by creating coordination between the expert opinions. Delphi is referred to as an integrated method, that is, a combination of qualitative and quantitative methods [41, 42]. The Delphi method is based on the logical assumption that several thoughts are better than one thought . In the Delphi method, the questionnaire is performed in two or more stages and the results obtained from the previous courses are used for the new period of compiling and modifying the questionnaire.(ii)Triangular Fuzzy Aggregation: the traditional process of quantifying people’s perspectives does not fully reflect the human thinking style. In other words, the use of fuzzy sets is more compatible with linguistic and sometimes ambiguous human explanations. Therefore, it is better to use long-term predictions and real-world decisions using fuzzy sets (using fuzzy numbers).(iii)PROMETHEE Method: in recent years, the PROMETHEE method has been used to prioritize options, especially in crisis situations [44, 45]. Among the advantages of PROMETHEE method are comprehensibility, ability to deal with uncertainty, value decision-makers, visual display power of data, high reliability, and flexibility. Also, in higher versions of PROMETHEE (PROMETHEE 5) it is possible to apply restrictions to determine the optimal options .
3.1. Delphi Method
The Delphi method is an iterative process for gathering expert opinions; these experts are known to the panel members, whose only comments are published [47–49]. In this study, the panel members are health experts. Preliminary criteria were sent to members, and their opinions were collected. According to their views, the criteria were modified and sent to the panel members for the second time in the next step. In this study, the criteria are divided into preventive and operational, and a questionnaire has been developed for evaluation and comment. Preventive criteria are considered for evaluating emergency bases and functional criteria for assessing these centers’ patient transport vehicles (ambulances). Table 2 shows the research criteria, which are divided into two parts: preventive and operational.
3.2. Weighting Criteria
Triangular fuzzy numbers (TFNs), which have a very high computational efficiency due to their simplicity and comprehensibility, were used in this study. Experts use this widespread weighting method, and it is more reliable. Each triangular fuzzy number consists of three values the upper bound u is the maximum value of fuzzy number F. The lower bound of is the minimum value that a fuzzy number can bring, and is the most probable value of a fuzzy number.
In this weighting step, the triangular fuzzy aggregation numbers are obtained according to the following formula :
3.3. Weighting Results
According to the set criteria and with experts’ opinions, each of the criteria has been weighted. This weighting was determined by the triangular fuzzy aggregation method, and the criteria were defined in Excel software. The calculated weight is shown in Table 3. In the preventive weight section, the criteria for preventing the activity of suspicious personnel (0.91) and holding training sessions with weight (0.75) and in the operational area the weight measure or observe personal hygiene by the driver (1.00) and the car disinfection after operation criterion with weight (0.91) are one of the highest importance of criteria.
4. Sampling through the Morgan Table and Field Evaluation
Table 3 lists the operational criteria for field evaluation used to evaluate the patient transport vehicles (ambulance) for each research emergency base.
One of the easiest and most conservative ways to determine sample size is to use the Morgan table. To use this table, the target population size of Morgan table and sample size must be found . Based on this, and using the information obtained, the number of patient transport vehicle operations in Sari, Iran, which were performed for one month, performs an average of 60 missions per day, and using the Morgan table, 52 sampling had to be done. The information of these evaluations is recorded daily and during the missions. For example, Ambulance B-1 (second emergency center) used a mask (N95) in 50 operations on the 30th day. After a field evaluation of the vehicles during a month, the average score obtained for each patient transport vehicle has been calculated. This average value indicates the level of compliance with the standard criteria of each vehicle during 30 days of sampling and field evaluation. Table 4 shows the results related to the first emergency and Table 5 shows the products related to the second emergency in full, which are given during a month.
4.1. Evaluation of the First and Second Emergency Centers
Based on the criteria of experts, a questionnaire was prepared to evaluate the prevention indicators of emergency centers. According to this questionnaire, emergency centers were evaluated for good performance and compliance with standards (Yes) and poor performance and noncompliance with standards (No). Table 5 shows the results of the performance evaluation of these centers in terms of compliance or noncompliance with health protocols.
5. Evaluation of the Results of Calculations
After determining the criteria and their weight by specialists, the evaluation of the transportation vehicles (Ambulance A-1, Ambulance A-2, Ambulance B-1, and Ambulance B-2) are performed using the PROMETHEE method. Quantitative results (Tables 4 and 6) and qualitative results (Table 5) of these evaluations are placed in a pairwise comparison matrix in Visual PROMETHEE software.
5.1. PROMETHEE Method
One of the main advantages of the PROMETHEE method is its simplicity and clarity. In this method, each of the criteria and weights can affect the answers, which shows the efficiency of this method. This method is used to make multicriteria decisions . PROMETHEE I method gives the accurate ranking, and the PROMETHEE II method provides a complete ranking . The PROMETHEE V method can also be used to consider problem constraints and evaluate the performance of options . According to the objectives of this study, PROMETHEE II method was used to prioritize and evaluate options.
Set of Options. The shows the criteria against which the options are evaluated. The options are compared in pairs. This comparison is performed through a predefined priority function with ranges [0, 1]. For function , there are options and criterion .so that shows the difference in sizes in the index. The final ranking of the options is obtained through the following formula:where is the standard weight determined by the decision maker and normalized by [55, 56]. For the option, the ranking stream is calculated by considering the other options.
The positive preference flow is as follows:
Positive flow indicates better performance of one option than other options.
The negative preference flow is as follows:
The input data for evaluating the options are the weight obtained from the criteria in Table 2, and the quantitative and qualitative values obtained from the field evaluation of the patient’s vehicles are in Tables 4–6. Quantitative and qualitative values are considered as a pairwise comparison matrix for options and criteria. Emergencies were evaluated based on the optimal performance of patient transport vehicles of each center in compliance with health protocols.
Table 7 is the output of PROMETHEE flow table. This table shows the positive (Phi +), negative (Phi−), and net Flow (Phi). Net is the balance of positive and negative flows. Each higher net indicates superior performance. In this way, the difference in performance between the two responses can be examined. As can be seen in Table 8, Ambulance B-1 (Phi = 0.1776) ranked first, Ambulance A-2 (Phi = 0.0846) ranked second, Ambulance A-1 (Phi = −0.0888) ranked third, and Ambulance B-2 (Phi = −0.1734) took the fourth place.
Figure 2 shows the GAIA chart. This chart has three different categories of information:
(1) Actions that are displayed with dots; (2) criteria in which the axes are displayed, and longer axis of the criteria is higher degree of importance; and (3) the weight of the criteria indicated by the decision axis. The length of each of them indicates the relative strength of the standard. The longer it is, the more critical that criterion is. On the other hand, the direction of an axis indicates where this criterion’s best possible actions are located. In the GAIA diagram, the options that are similar to each other are closer to each other. In displaying GAIA diagrams, the matching options are more relative to each other, and the conflicting options are farther apart. The criteria that have similar preferences are in the same direction, and the criteria that have inconsistent choices are in different directions. For example, Ambulance A-2 performs well in terms of car disinfection after operation and does not perform well in the standard of appropriate treatment equipment (due to being in the opposite direction of this standard).
Figure 3 is a rainbow diagram of a PROMETHEE output. This chart is a display of complete rankings. The sections at the top of the diagram show each option’s positive attribute, and the sections at the bottom of the diagram show the opposing point of each option. In this diagram, ranking is done from left to right, and using this diagram, each option’s performance and strengths and weaknesses can be determined. As shown in Figure 3, left-to-right Ambulance B-1 performed positively on seven criteria and negatively on three criteria. Also, the observed personal hygiene (1), car disinfection after operation (0.91), and separate personnel health supply (0.75) criteria had poor performance.
Figure 4 is a square corresponds to the page (Phi+, Phi−), where a dot indicates each option. Phi + scores increase from the left corner to the top corner, and phi- scores increase from the left corner to the bottom corner. For each option, a cone is drawn from the option position on the page. As can be seen, the Ambulance A-1 and Ambulance A-2 options have very similar functions, so that their performance axis overlaps. The reason for this can be considered the functional similarity in the criteria. This functional similarity has led to the convergence of the results of these two options. The high performance of the B-1 option in pure flow and positive flow has made it higher than other options in both the vertical axis and the left corner axis.
Finally, considering the amount of net flow () and each option’s performance score, each emergency center (Emergency A and Emergency B) is obtained. As shown in Figure 5, Emergency A (63%) performance was better. The reason for this center’s superiority over Center B can be considered the better performance of this center in terms of prevention indicators (Table 5) and better performance of the center’s vehicles (Ambulance A-1 and Ambulance A-2) in the performance index by observing the standards. Center B did not meet two of the five precautionary criteria. The center’s vehicles (Ambulance B-1 and Ambulance B-2) performed poorly on many of the standard criteria of high weight importance, ultimately leading to poor performance.
The purpose of this study was to evaluate emergency centers and vehicles carrying these centers at the time of coronavirus infection. First, with the cooperation of health experts and Delphi method, the criteria and subcriteria were identified and divided into operational and prevention. Then, the centers were evaluated based on preventive criteria, which included five criteria, and using the Morgan table and considering the average daily operations of each emergency, the number of samples of patient vehicle missions was determined. Each emergency station (each containing two vehicles) was evaluated daily by sampling. The results of these evaluations are evaluated and ranked using the PROMETHEE method as a binary comparison matrix. The results showed better performance of Center A (63%) due to proper performance and better compliance with protocols in both criteria (preventive and operational). The reason for the superiority of this center over Center B can be considered the better performance of this center in terms of prevention indicators and better performance of the center’s vehicles (Ambulance A-1 and Ambulance A-2) in the performance index by observing the standards. The center has complied with all standards of preventive standards. The vehicles of this Center A-1 and Ambulance A-2 also had a positive performance in observing the performance indicators. Given the importance of emergency bases during an epidemic to control the disease and prevent the spread of the disease from treatment personnel to the patient and vice versa, the risk of vulnerability during service should be minimized by observing the set standards. By eliminating weaknesses and increasing strengths, relief bases can prevent disruption in the treatment and transmission of patients in severe epidemic conditions and minimize the possibility of vulnerability and transmission of the disease to medical staff and patients. According to the results, it can be concluded that in general the performance of the considered emergencies has been good in terms of operational and preventive criteria, and it can be concluded that these centers have been adapted to the epidemic conditions. The Ambulance B-2 option, which is ranked last in the ranking out of ten evaluation criteria, has a positive performance in 6 criteria and only in the criteria of staff use N95 mask, proper ventilation system, car cleaning facilities and equipment, and car disinfection after operation had poor performance, which with more management and stricter monitoring can be seen better performance in these criteria. Also, in the evaluation of the criteria by experts, the criteria of prevents the activity of suspicious personnel and car disinfection after the operation were of higher weight (0.91) than other criteria, which were performed by regular virus identification test and timely identification of emergency personnel and car disinfection. The patient can lead to better performance of emergency centers during an epidemic. The Crisis Management Organization, the Red Crescent, and the emergency centers are among the organizations that can address structural and operational problems during an epidemic by considering the results of in-depth management research and studies. This research can help relevant organizations and centers minimize financial and human losses in the event of an epidemic. In general, the research results include the following:(i)The results showed better performance of Center A (63%) due to proper performance and better compliance with protocols in both criteria (preventive and operational).(ii)In the evaluation of the criteria by experts, the criteria for preventing the activity of suspicious personnel and postoperative car disinfection had a higher weight (0.91) than other criteria.(iii)According to the results, it can be concluded that in general the performance of the considered emergencies has been good in terms of operational and preventive criteria, and it can be concluded that these centers have been adapted to the epidemic conditions.(iv)Disinfection of staff and sick vehicles can lead to better performance of emergency centers during an epidemic.
Lack of access to experts for face-to-face evaluation and noncooperation of some emergency centers to record information due to the epidemic conditions have been the limitations of this study. Also, due to epidemic conditions, it has been tried to exchange information online as much as possible, and due to special restrictions, it was not possible to receive operational information at longer intervals. In addition to emergency centers, hospitals and relief centers are also recommended. It is also recommended to check the performance of emergency centers under normal conditions and compare it with epidemic conditions to identify the effective factors and take action to better manage the epidemic.
The data that support the findings of this study are available from the corresponding author, upon reasonable request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
World Health Organization (Who), WHO Statement Regarding Cluster of Pneumonia Cases in Wuhan, China. Beijing, WHO, Geneva, Switzerland, 2020, https://www.who.int/china/news/detail/09-01-2020-who-statement-regarding-cluster-of-pneumonia-cases-in-wuhan-china/.
E. Babaee Tirkolaee and N. S. Aydın, “A sustainable medical waste collection and transportation model for pandemics,” Waste Management & Research, vol. 39, no. 1, pp. 34–44, 2021.View at: Google Scholar
A. P. Chobar, M. A. Adibi, and A. Kazemi, “Multi-objective hub-spoke network design of perishable tourism products using combination machine learning and meta-heuristic algorithms,” Environment, Development and Sustainability, Springer, Berlin, Germany, pp. 1–28, 2022.View at: Publisher Site | Google Scholar
Centers for Disease Control and Prevention, 2021, https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html/.
M. B. McNamara, “Saving Lives; Living the Dream”: Gender and Emotional Labor Among Ambulance-Based 911 Paramedics,” University of California, Santa,Cruz, 2016, Doctoral dissertation.View at: Google Scholar
S. R. Pitts, W. Niska, and C. W. Burt, “National hospital ambulatory medical care survey: 2006 emergency department summary,” National health statistics reports, vol. 6, no. 7, pp. 1–3, 2008, doi: 10.1.1.218.855.View at: Google Scholar
S. Jahangiri, M. Abolghasemian, A. Pourghader Chobar, A. Nadaffard, and V. Mottaghi, “Ranking of key resources in the humanitarian supply chain in the emergency department of iranian hospital: a real case study in COVID-19 conditions,” Journal of applied research on industrial engineering, vol. 8, no. Special Issue, pp. 1–10, 2021.View at: Publisher Site | Google Scholar
M. Rezaei Kallaj, M. Abolghasemian, S. Moradi Pirbalouti, M. Sabk Ara, and A. Pourghader Chobar, “Vehicle routing problem in relief supply under a crisis condition considering blood types,” Mathematical Problems in Engineering, vol. 2021, Article ID 7217182, 10 pages, 2021.View at: Publisher Site | Google Scholar
H. Liao, Y. Long, M. Tang, D. Streimikiene, and B. Lev, “Early lung cancer screening using double normalization-based multi-aggregation (DNMA) and Delphi methods with hesitant fuzzy information,” Computers & Industrial Engineering, vol. 136, pp. 453–463, 2019.View at: Publisher Site | Google Scholar
H. A. Choukolaei, S. Shafaee Tilaki, and A. Abraham, “Evaluation of isolation room and Anteroom of hospitals in corona conditions (case study: Sari, Iran),” in Proceedings of the International Conference on Intelligent and Fuzzy Systems, pp. 775–786, Izmir, Turkey, August 2021.View at: Google Scholar
D. Kriksciuniene and V. Sakalauskas, “AHP model for quality evaluation of healthcare system 2017,” in Proceedings of the International Conference on Information and Software Technologies, vol. 756, pp. 129–141, Kaunas University of Technology (Lithuania), October, 2022.View at: Publisher Site | Google Scholar
R. Lotfi, B. Kargar, M. Rajabzadeh, F. Hesabi, and E. Özceylan, “Hybrid fuzzy and data-driven robust optimization for resilience and sustainable health care supply chain with vendor-managed inventory approach,” International Journal of Fuzzy Systems, vol. 24, no. 2, pp. 1216–1231, 2022.View at: Publisher Site | Google Scholar
A. Navaei, S. Gupta, D. D. Hammawa, and U. M. Modibbo, “A bi-objective model for hospital evacuation and reliable pharmaceutical supply chain for critical patients,” in Proceedings of the 2021 2nd International Conference on Intelligent Engineering and Management (ICIEM), pp. 587–592, IEEE, Amity University London, April 2021.View at: Publisher Site | Google Scholar
M. Turoff and H. A. Linstone, The Delphi Method-Techniques and Applications, Addison-Wesley, Boston, MA, USA, 2002.
L. Stone Fish and D. M. Busby, “The Delphi Method,” Research Methods in Family Therapy, Guilford, New York, pp. 238–253, 2005.View at: Google Scholar
D. Darvishi, H. Ahmadi Choukolaei, and S. Shafaee, “Assessing the location of relief centers using a combination of multi-criteria decision-making methods and gis (case study: district 18 of tehran),” International journal of research in industrial engineering, vol. 11, no. 1, pp. 16–29, 2022.View at: Google Scholar
R. C. Brownson, E. A. Baker, A. D. Deshpande, and K. N. Gillespie, Evidence-based Public Health, Oxford University Press, Oxford, United Kingdom, 2017.
A. Goli, E. B. Tirkolaee, and A. K. Sangaiah, “Hybrid neural network and improved cuckoo optimization algorithm for forecasting thermal comfort index at urban open spaces,” Advances in Edge Computing: Massive Parallel Processing and Applications, vol. 35, no. 264, pp. 264–280, 2020.View at: Publisher Site | Google Scholar
A. Ayough, “Adaptive aggregate production planning with fuzzy goal programming approach,” Journal of Industrial Engineering and Management Studies, vol. 5, no. 2, pp. 38–60, 2018, doi: 10.22116/JIEMS.2018.80684.View at: Google Scholar
P. Ghasemi and E. Talebi Brijani, “An integrated FAHP-PROMETHEE approach for Selecting the best Flexible Manufacturing system,” European Online Journal of Natural and Social Sciences, vol. 3, no. 4, p. 1137, 2014.View at: Google Scholar
H. Ahmadi Choukolaei, M. Jahangoshai Rezaee, P. Ghasemi, and M. Saberi, “Efficient crisis management by selection and analysis of relief centers in disaster integrating GIS and multicriteria decision methods: a case study of Tehran,” Mathematical Problems in Engineering, vol. 2021, Article ID 944828, 22 pages, 2021.View at: Publisher Site | Google Scholar
S. Greco, J. Figueira, and M. Ehrgott, Multiple Criteria Decision Analysis, Springer, New York, USA, 2016.