Gastroenterology Research and Practice

Gastroenterology Research and Practice / 2021 / Article

Research Article | Open Access

Volume 2021 |Article ID 8674367 | https://doi.org/10.1155/2021/8674367

Duc Trong Quach, Uyen Pham-Phuong Vo, Nguyet Thi-My Nguyen, Ly Thi-Kim Le, Minh-Cong Hong Vo, Phat Tan Ho, Tran Ngoc Nguyen, Phuong Kim Bo, Nam Hoai Nguyen, Khanh Truong Vu, Manh Van Dang, Minh Cao Dinh, Thai Quang Nguyen, Xung Van Nguyen, Suong Thi-Ngoc Le, Chi Pham Tran, "An External Validation Study of the Oakland and Glasgow-Blatchford Scores for Predicting Adverse Outcomes of Acute Lower Gastrointestinal Bleeding in an Asian Population", Gastroenterology Research and Practice, vol. 2021, Article ID 8674367, 7 pages, 2021. https://doi.org/10.1155/2021/8674367

An External Validation Study of the Oakland and Glasgow-Blatchford Scores for Predicting Adverse Outcomes of Acute Lower Gastrointestinal Bleeding in an Asian Population

Academic Editor: Konstantinos Triantafyllou
Received26 May 2020
Revised06 Dec 2020
Accepted24 Dec 2020
Published06 Jan 2021

Abstract

Aims. This study is aimed at (1) validating the performance of Oakland and Glasgow-Blatchford (GBS) scores and (2) comparing these scores with the SALGIB score in predicting adverse outcomes of acute lower gastrointestinal bleeding (ALGIB) in a Vietnamese population. Methods. A multicenter cohort study was conducted on ALGIB patients admitted to seven hospitals across Vietnam. The adverse outcomes of ALGIB consisted of blood transfusion; endoscopic, radiologic, or surgical interventions; severe bleeding; and in-hospital death. The Oakland and GBS scores were calculated, and their performance was compared with that of SALGIB, a locally developed prediction score for adverse outcomes of ALGIB in Vietnamese, based on the data at admission. The accuracy of these scores was measured using the area under the receiver operating characteristic curve (AUC) and compared by the chi-squared test. Results. There were 414 patients with a median age of 60 (48–71). The rates of blood transfusion, hemostatic intervention, severe bleeding, and in-hospital death were 26.8%, 15.2%, 16.4, and 1.4%, respectively. The SALGIB score had comparable performance with the Oakland score (AUC: 0.81 and 0.81, respectively; ) and outperformed the GBS score (AUC: 0.81 and 0.76, respectively; ) for predicting the presence of any adverse outcomes of ALGIB. All of the three scores had acceptable and comparable performance for in-hospital death but poor performance for hemostatic intervention. The Oakland score had the best performance for predicting severe bleeding. Conclusions. The Oakland and SALGIB scores had excellent and comparable performance and outperformed the GBS score for predicting adverse outcomes of ALGIB in Vietnamese.

1. Introduction

The prevalence of acute lower gastrointestinal bleeding (ALGIB) which leads to hospitalization has been increasing over the last twenty years [1]. Although there are several risk scores for predicting adverse outcomes of ALGIB, most of these scores fail to accurately predict outcomes when externally validated in other populations [2, 3]. The Oakland score was developed and validated for predicting the absence of adverse outcomes in ALGIB patients in the UK [4]. Although it has been recommended by the British guidelines for stratifying risk of patients presenting with ALGIB [5], there are concerns that the number of patients and hospitals in validation studies is short and the score needs further validation in other populations [68]. Besides, the Glasgow-Blatchford score (GBS), which is widely applied in upper gastrointestinal bleeding, has recently demonstrated good performance in patients with ALGIB [3, 9]. To date, there have been no validation studies of the two scores in Asian populations. In Vietnam, we have recently developed and validated a local scoring system, the severe acute lower gastrointestinal (SALGIB) score, to predict severe ALGIB (Table 1) [10]. This score is simpler and easier compared to the two above-mentioned scores and has demonstrated good performance. However, which score has the best performance needs to be investigated. This study is aimed at (1) validating the performance of Oakland and Glasgow-Blatchford (GBS) scores and (2) comparing these scores with the SALGIB score in predicting adverse outcomes of ALGIB in a Vietnamese population.


Oakland scoreSALGIB scoreBlatchford score

Age (years)Heart rate (bpm)Heart rate (bpm)
 <400<1000<1000
 40–691≥1001≥1001
 ≥702SBP (mmHg)SBP (mmHg)
Sex≥1000≥1100
 Female0<1003100–1091
 Male1Hematocrit (%)90–992
Previous ALGIB admission≥350<903
 No030–34.91Blood urea nitrogen (mg/dL)
 Yes125–29.93<190
DRE findings<255≥19 and <22.42
 No blood0Platelet count (103/μL)≥22.4 and <283
 Blood1>1500≥28 and <704
Heart rate (bpm)≤1501≥706
 <700Hemoglobin (male) (g/L)
 70–891 ≥1300
 90–1092≥120 and <1301
 ≥1103≥100 and <1203
SBP (mmHg)<1006
 <905Hemoglobin (female) (g/L)
 90–1194 ≥1200
 120–1293≥100 and <1201
 130–1592<1006
 ≥1600Chronic disease
Hemoglobin (g/L)Hepatic disease2
 <7022Cardiac failure2
 70–8917Melena
 90–10913No0
 110–1298Yes1
 130–1594Syncope
 ≥1600No0
Yes2

ALGIB: acute lower gastrointestinal bleeding; DRE: digital rectal examination; SBP: systolic blood pressure.

2. Materials and Methods

2.1. Setting and Study Design

A multicenter cohort study was conducted at seven secondary and tertiary hospitals across Vietnam (Gia-Dinh People’s Hospital, Cho-Ray Hospital, Dong-Nai General Hospital, Can-Tho Central Hospital, Bach-Mai Hospital, Hue Central Hospital, and Da-Nang General Hospital). This study was approved by the Board of Ethics in Biomedical Research of the University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam (numbered 146/DHYD-HDDD, signed on April 21, 2018). The study protocol conforms to the ethical guidelines of the 1975 Declaration of Helsinki. Written informed consent was obtained from all patients or their legal guardians.

2.2. Study Population

Patients aged ≥16 years with symptoms suggesting overt ALGIB (i.e., red or maroon colored stools, blood mixed in with the stools, clots per rectum, or the passage of melena without hematemesis) who were admitted and underwent lower gastrointestinal endoscopy at participating hospitals from October 2018 to November 2019 were recruited. Patients who developed ALGIB while admitted for other reasons who had proven findings of upper gastrointestinal bleeding on oesophagogastroduodenoscopy (OGD) were excluded.

2.3. Data Collection

Data at admission were recorded in every patient, which included demographic characteristics, symptoms, prior history of ALGIB, comorbidities, vital signs, finding of digital rectal examination, and results of complete blood count, renal function, and coagulation tests.

2.4. Treatment

This is an observational study. If patients presented with melena alone or hematochezia and concomitant hemodynamic instability, OGD was performed to exclude upper gastrointestinal bleeding. Lower gastrointestinal endoscopy (sigmoidoscopy or colonoscopy) was performed in all patients to investigate the sources of bleeding, but complete colonoscopy was not a requirement. Patients whose confirmative source of bleeding (i.e., lesions with stigmata of recent bleeding, friable tumors, or colitis) was identified by sigmoidoscopy or those with life-threatening bleeding or comorbidities who could not safely undergo complete colonoscopy were also recruited.

Regarding the hemostatic strategy, gastrointestinal endoscopy was the first choice. If it failed to control bleeding or was not able to identify the source of bleeding, a multidisciplinary team meeting was organized with interventional radiologists and general surgeons to decide the rescue management.

2.5. Outcomes

The adverse outcomes of ALGIB in this study consisted of blood transfusion, hemostatic intervention, severe bleeding, and in-hospital death. Blood transfusion was decided according to the recommendations of the National Institute for Health and Care Excellence [11]. Hemostatic intervention was a composite of endoscopic, radiologic, and surgery interventions. Endoscopic interventions were considered as appropriate if endoscopic lesions with stigmata of recent bleeding (i.e., active bleeding, a visible vessel, or an adherent clot) were detected and endoscopically treated. Severe ALGIB was defined as persistent bleeding within the first 24 hours and/or recurrent bleeding after 24 hours of stability accompanied by a further decrease in hematocrit of ≥20% and/or requirement of ≥2 units of packed red blood cells [12].

2.6. Statistical Analysis

Categorical data were presented numerically. Quantitative data were tested for normality using the Kolmogorov-Smirnov test, and those with no normal distribution were presented as median and interquartile range (IQR). The Oakland, SALGIB, and GBS scores were calculated basing on the data at admission (Table 1). The performance of each score was measured using the area under the receiver operating characteristic curve (AUC) and was considered as follows: no discrimination (), acceptable (AUC: 0.7–0.8), excellent (AUC: 0.8–0.9), and outstanding () [13]. The chi-squared test was used to compare the difference between AUC curves according to the method described by DeLong et al. [14]. All statistical analyses were performed using MedCalc version 19 (MedCalc Software Ltd., Ostend, Belgium).

3. Results

3.1. Patient Characteristics and Adverse Outcomes

The demographic and clinical characteristics of patients in this study are presented in Table 2. The causes of ALGIB in our study are presented in Table 3. And the adverse outcomes of ALGIB are summarized in Figure 1.


Characteristics

Age, median (interquartile range)60 (48-71)
Sex
 Male, (%)202 (48.8)
 Female, (%)212 (51.2)
Participating site
 Gia-Dinh People’s Hospital, (%)74 (17.9)
 Cho-Ray Hospital, (%)111 (26.8)
 Dong-Nai General Hospital, (%)43 (10.4)
 Can-Tho Central Hospital, (%)55 (13.4)
 Bạch-Mai Hospital, (%)51 (12.3)
 Hue Central Hospital, (%)40 (9.6)
 Da-Nang General Hospital, (%)40 (9.6)
Previous admission with ALGIB, (%)115 (27.8)
Comorbidities
 Congestive heart failure, (%)12 (2.9)
 Ischemic heart disease, (%)42 (10.1)
 Chronic obstructive pulmonary disease, (%)6 (1.4)
 Chronic liver disease, (%)37 (8.9)
 Chronic kidney disease, (%)26 (6.3)
 Stroke or transient ischemic attack, (%)15 (3.6)
 Cancer, (%)38 (9.2)
 Hypertension, (%)147 (35.5)
 Diabetes, (%)45 (10.9)
Preadmission medications
 Aspirin, (%)20 (4.8)
 Clopidogrel, (%)22 (5.3)
 Dual antiplatelet, (%)8 (1.9)
 Warfarin, (%)2 (0.5)
 NOAC, (%)5 (1.2)
 Corticosteroid, (%)9 (2.2)
 NSAIDs, (%)30 (7.2)
Presenting signs and symptoms
 Heart rate (beat per minute)86 (80-92)
 Systolic blood pressure (mmHg)120 (110-130)
 Melena, (%)64 (15.4)
 Syncope, (%)2 (0.5)
 Blood on DRE, (%)260 (62.8)
Laboratory data at admission
 Hematocrit (%)31.8 (24.5-38.4)
 Hemoglobin (g/L)103 (76-126)
 Platelet (×103/μL)242 (191-311)
 INR1.06 (1.00-1.15)
 Blood urea nitrogen (mg/dL)14.2 (11-21)
 Creatinine (μmol/L)0.93 (0.79-1.14)
Types of lower gastrointestinal endoscopy
 Colonoscopy, (%)264 (63.8)
 Sigmoidoscopy, (%)78 (18.8)
 Rectoscopy, (%)68 (16.4)
 Enteroscopy, (%)5 (1.2)
Abdominal CT scan, (%)80 (19.3)

Data are (%) or median (interquartile range). NOAC: novel oral anticoagulant; NSAIDs: nonsteroidal anti-inflammatory; DRE: digital rectal examination.

Source of bleeding (%)

Hemorrhoids120 (29.0)
Inflammatory bowel disease46 (11.0)
Colon adenocarcinoma31 (7.5)
Colorectal polyps31 (7.5)
Diverticulosis25 (6.0)
Angioectasia25 (6.0)
Colon ulcers25 (6.0)
Colitis23 (5.6)
Benign anorectal disorder (except hemorrhoids)15 (3.6)
Small intestinal ulcers7 (1.7)
Small bowel tumor2 (0.5)
Duodenal pseudoaneurysm rupture1 (0.2)
Postpolypectomy/posthemorrhoidectomy11 (2.7)
Unknown52 (12.6)

3.2. Performance of Scoring Systems in Predicting Adverse Outcomes of ALGIB

For predicting the presence of any adverse outcomes, the SALGIB score had comparable performance with the Oakland score (AUC: 0.81 and 0.81, respectively; ) and outperformed the GBS score (AUC: 0.81 and 0.76, respectively; ) (Table 4). All scores had acceptable and comparable performance for in-hospital death but no discrimination for hemostatic intervention. For predicting the need of blood transfusion, the SALGIB score also had comparable performance with the Oakland score (AUC: 0.91 and 0.93, respectively; ) and outperformed the GBS score (AUC: 0.91 and 0.87, respectively; ). The Oakland score was the best one for predicting severe ALGIB with an AUC of 0.90.


Red blood cell transfusionHemostatic interventionSevere ALGIBIn-hospital deathAny adverse outcome
(26.8%) (15.2%) (16.4%) (1.4%) (38.9%)

SALGIB0.91 (0.88-0.94)0.53 (0.48-0.58)0.87 (0.83-0.90)0.82 (0.78-0.86)0.81 (0.77-0.84)
Oakland0.93 (0.90-0.95)0.52 (0.47-0.56)0.90 (0.87-0.93)0.77 (0.72-0.81)0.81 (0.77-0.85)
Blatchford0.87 (0.84-0.90)0.50 (0.45-0.55)0.83 (0.79-0.87)0.76 (0.72-0.80)0.76 (0.72-0.80)

Data are presented as areas under the receiver operating characteristic curve and 95% confidence intervals. Combined outcome of red blood cell transfusion, hemostatic interventions, severe bleeding, and in-hospital death; values are from the DeLong et al. test.

The cut-off points of the Oakland score at ≤14, the SALGIB score at 0, and the GBS score at ≤2 had a sensitivity of about 85% for predicting an uneventful course (i.e., ALGIB without any in-hospital adverse outcomes) in one-third of ALGIB patients (Table 5). And the cut-off points of the Oakland score at 0 and GBS at 0 had the sensitivities of 96.9% and 92.4% for predicting an uneventful course of ALGIB in 10.4% and 17.5% of patients, respectively.


Oakland scoreSALGIB scoreBlatchford score
ScoreSensSpecCum%ScoreSensSpecCum%ScoreSensSpecCum%

>5100.000.400.2>085.7145.8533.6>092.4523.7217.5
>6100.001.581.0>181.3770.3650.2>188.6843.0830.8
>7100.002.371.4>277.0274.3154.3>285.5348.6235.4
>899.385.933.9>365.8487.7566.9>383.0254.9440.3
>999.387.514.8>460.2592.4972.0>479.2560.8745.4
>1096.8915.0210.4>530.4394.0784.5>578.6265.6148.5
>1193.7923.7216.9>613.6699.2194.2>659.7578.2663.6
>1290.0630.0422.2>79.94100.0096.1>745.2886.1774.0
>1389.4437.1526.8>88.70100.0096.6>838.9990.1278.9
>1486.3446.2533.6>94.35100.0098.3>926.4294.0786.2
>1584.4751.3837.4>100.00100.00100.0>1022.0196.8489.6
>1683.2358.5042.3>1115.0998.4293.2
>1782.6163.6445.7>1211.3299.2195.1
>1881.3766.8048.1>135.66100.0097.8
>1978.8869.1750.5>142.52100.0099.0
>2073.9173.1254.8>151.89100.0099.3
>2168.9482.2162.3>160.00100.00100.0
>2266.4683.7964.3
>2360.8787.3568.6
>2456.5292.4973.4
>2549.6994.4777.3
>2643.4896.4480.9
>2740.9997.2382.4
>2832.3098.4286.5
>2924.2298.4289.6
>3014.2999.6094.2
>316.8399.6097.1
>323.7399.6098.3
>330.00100.00100.0

Cum%: percentage of the total frequency of patient distribution according to score. Sens: sensitivity; spec: specificity. Cut-off points determined by the Youden index.

4. Discussion

The prevalence of severe ALGIB and in-hospital death in our study was comparable with that reported by Oakland et al. in a multicenter cohort study in the UK [4], but it was lower than that reported in many single-center cohorts [2, 3, 12, 15]. This difference could be partly explained by the fact that the participants in these single-center studies were recruited from tertiary hospitals, and only those performed with colonoscopy were included [3, 15]. As we aimed to evaluate the performance of prediction scores in real-life practice, ALGIB patients in our study were recruited from hospitals with different levels of care across Vietnam, and those who did not have complete colonoscopy were also included.

To the best of our knowledge, our study was the first validation study of the Oakland score in an Asian population. Previously, this score has been validated in the UK and showed good performance for predicting blood transfusion, rebleeding, and the presence of any adverse outcomes [4]. Another cohort study in the US also found that it was the best score for predicting severe ALGIB compared to the GBS, AIMS65, and Strate scores [3]. Our study showed that the SALGIB and the Oakland scores had excellent and comparable performance for predicting the presence of any adverse outcomes in Vietnamese. However, as the former has much fewer components, it would be more convenient for daily practice than the later score. In our study, although the GBS score was outperformed by the two other scores, it still had acceptable discrimination for predicting the presence of any adverse ALGIB outcomes. The GBS score has been well validated for use in upper gastrointestinal bleeding, but some recent studies also reported that it also performed well in patients with ALGIB [3, 9, 16]. In real-life practice, it is sometimes very difficult to locate the source of gastrointestinal bleeding at admission. Our study, therefore, suggested that the GBS score could be considered an alternative prediction tool in such challenging situations.

Regarding the cut-off point for predicting the absence of adverse ALGIB outcomes, an Oakland in our study or ≤8 in the previous cohort in the UK had a sensitivity of 96.9% and 95%, respectively [4]. These cut-offs, while having the merit of selecting patients with potentially safe discharge, are not clinically important as only less than one-tenth of patients in both above-mentioned cohorts had such points. In our study, we found that an Oakland or a SALGIB score at 0 or a GBS helped to predict an uneventful course of ALGIB with a sensitivity of about 85% in one-third of ALGIB patients. These cut-offs, therefore, could be more practical to stratify ALGIB patients for an appropriate level of care.

Our study has some limitations. First, there was no follow-up after hospital discharge in this study. Therefore, it only demonstrated the performance of the scoring systems in predicting in-hospital adverse outcomes. Identifying patients with low risk who are suitable to manage as outpatients needs to be further evaluated in a future study. Second, as only two-thirds of patients in this study underwent colonoscopy, the source of bleeding might have been missed in some patients. Third, the hemostatic strategies at participating sites were divergent depending on the local resources and expertise and could potentially affect the adverse outcomes of ALGIB in this study.

In conclusion, we found that the Oakland and SALGIB scores had excellent and comparable performance and outperformed the GBS score for predicting adverse outcomes of ALGIB in Vietnamese.

Data Availability

The data file (.sav) used to support the findings of this study are available from the corresponding author upon request.

Ethical Approval

This study was approved by the Board of Ethics in Biomedical Research of the University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam (numbered 146/DHYD-HDDD, signed on April 21, 2018). The study protocol conforms to the ethical guidelines of the 1975 Declaration of Helsinki.

Written informed consent was obtained from all patients or their legal guardians.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

Quach D designed and supervised the study. Nguyen N, Vo U, Le L, Vo C, Ho P, Nguyen T, Bo P, Nguyen N, Vu K, Dang M, Dinh M, Nguyen T, Nguyen X, Le S, and Pham C performed the research and collected data. Quach D and Vo U analyzed the data. Quach D made and critically revised the first draft. All authors approved the final version of the manuscript. Quach D submitted the manuscript.

Supplementary Materials

The supplementary file includes the TRIPOD Checklist: Prediction Model Development. (Supplementary Materials)

References

  1. A. Lanas, L. A. García-Rodríguez, M. Polo-Tomás et al., “Time trends and impact of upper and lower gastrointestinal bleeding and perforation in clinical practice,” The American Journal of Gastroenterology, vol. 104, no. 7, pp. 1633–1641, 2009. View at: Publisher Site | Google Scholar
  2. S. A. Xavier, F. J. Machado, J. T. Magalhães, and J. B. Cotter, “Acute lower gastrointestinal bleeding: areSTRATEandBLEEDscores valid in clinical practice?” Colorectal Disease, vol. 21, pp. 357–364, 2018. View at: Publisher Site | Google Scholar
  3. N. Tapaskar, B. Jones, S. Mei, and N. Sengupta, “Comparison of clinical prediction tools and identification of risk factors for adverse outcomes in acute lower GI bleeding,” Gastrointestinal Endoscopy, vol. 89, no. 5, pp. 1005–1013.e2, 2019. View at: Publisher Site | Google Scholar
  4. K. Oakland, V. Jairath, R. Uberoi et al., “Derivation and validation of a novel risk score for safe discharge after acute lower gastrointestinal bleeding: a modelling study,” The Lancet Gastroenterology & Hepatology, vol. 2, no. 9, pp. 635–643, 2017. View at: Publisher Site | Google Scholar
  5. K. Oakland, G. Chadwick, J. E. East et al., “Diagnosis and management of acute lower gastrointestinal bleeding: guidelines from the British Society of Gastroenterology,” Gut, vol. 68, no. 5, pp. 776–789, 2019. View at: Publisher Site | Google Scholar
  6. N. Sengupta and E. B. Tapper, “Embracing early discharge in patients with lower gastrointestinal bleeding,” The Lancet Gastroenterology & Hepatology, vol. 2, no. 9, pp. 620-621, 2017. View at: Publisher Site | Google Scholar
  7. R. L. Kosowicz and L. L. Strate, “Predicting outcomes in lower gastrointestinal bleeding: more work ahead,” Gastrointestinal Endoscopy, vol. 89, no. 5, pp. 1014–1016, 2019. View at: Publisher Site | Google Scholar
  8. P. Cañamares, E. Alfaro, and A. Lanas, “Safe hospital discharge based on lower GI bleeding scores: a long way to go,” AME Medical Journal, vol. 2, p. 147, 2017. View at: Publisher Site | Google Scholar
  9. A. Ur-Rahman, J. Guan, S. Khalid et al., “Both full Glasgow-Blatchford score and modified Glasgow-Blatchford score predict the need for intervention and mortality in patients with acute lower gastrointestinal bleeding,” Digestive Diseases and Sciences, vol. 63, no. 11, pp. 3020–3025, 2018. View at: Publisher Site | Google Scholar
  10. D. T. Quach, N. T.-M. Nguyen, U. P.-P. Vo et al., “Development and validation of a scoring system to predict severe acute lower gastrointestinal bleeding in Vietnamese,” Digestive Diseases and Sciences, 2020. View at: Publisher Site | Google Scholar
  11. S. Padhi, S. Kemmis-Betty, S. Rajesh, J. Hill, and M. F. Murphy, “Blood transfusion: summary of NICE guidance,” BMJ, vol. 351, article h5832, 2015. View at: Publisher Site | Google Scholar
  12. L. L. Strate, E. J. Orav, and S. Syngal, “Early predictors of severity in acute lower intestinal tract bleeding,” Archives of Internal Medicine, vol. 163, no. 7, pp. 838–843, 2003. View at: Publisher Site | Google Scholar
  13. J. N. Mandrekar, “Receiver operating characteristic curve in diagnostic test assessment,” Journal of Thoracic Oncology, vol. 5, no. 9, pp. 1315-1316, 2010. View at: Publisher Site | Google Scholar
  14. E. R. DeLong, D. M. DeLong, and D. L. Clarke-Pearson, “Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach,” Biometrics, vol. 44, no. 3, pp. 837–845, 1988. View at: Publisher Site | Google Scholar
  15. T. Aoki, N. Nagata, T. Shimbo et al., “Development and validation of a risk scoring system for severe acute lower gastrointestinal bleeding,” Clinical Gastroenterology and Hepatology, vol. 14, no. 11, pp. 1562–1570.e2, 2016. View at: Publisher Site | Google Scholar
  16. S. Khalid, E. A. Idrisov, X. He et al., “Sa1212 Glasgow-Blatchford score as a prognostic scoring system for lower gastrointestinal bleed: is it as good as for upper gastrointestinal Bleed?” Gastrointestinal Endoscopy, vol. 85, no. 5, p. AB202, 2017. View at: Publisher Site | Google Scholar

Copyright © 2021 Duc Trong Quach et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


More related articles

 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder
Views34
Downloads123
Citations

Related articles

Article of the Year Award: Outstanding research contributions of 2020, as selected by our Chief Editors. Read the winning articles.