Sepsis: Diagnosis, Pathophysiology, and Novel Therapeutics for Clinical ManagementView this Special Issue
Review Article | Open Access
Ying Wang, Jingyi Zhao, Yinhui Yao, Lan Yang, Dan Zhao, Shiquan Liu, "The Accuracy of 16S rRNA Polymerase Chain Reaction for the Diagnosis of Neonatal Sepsis: A Meta-Analysis", BioMed Research International, vol. 2021, Article ID 5550387, 11 pages, 2021. https://doi.org/10.1155/2021/5550387
The Accuracy of 16S rRNA Polymerase Chain Reaction for the Diagnosis of Neonatal Sepsis: A Meta-Analysis
Objective. To determine the accuracy of 16S rRNA polymerase chain reaction (PCR) for the diagnosis of neonatal sepsis through a systematic review and meta-analysis. Methods. Studies involving 16S rRNA PCR tests for the diagnosis of neonatal sepsis were searched in the PubMed, Medline, Embase, and Cochrane Library databases. The methodological quality of the identified studies was evaluated using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2), and the sensitivity, the specificity, the positive likelihood ratio (PLR), the negative likelihood ratio (NLR), the diagnostic odds ratio (DOR), and the area under the curve (AUC) of operator characteristic (SROC) curves were determined. Heterogeneity between studies was analyzed by metaregression. Stata 14.0 and Meta-disc 1.4 software were used for the analyses. Results. This meta-analysis included 19 related studies. The analysis found a sensitivity of 0.98 (95% CI: 0.85-1), specificity of 0.94 (95% CI: 0.87-0.97), PLR of 16.0 (95% CI: 7.6-33.9), NLR of 0.02 (95% CI: 0.00-0.18), DOR of 674 (95% CI: 89-5100), and AUC of 0.99 (95% CI: 0.97-0.99). Metaregression analysis identified Asian countries, arterial blood in blood samples, and as the main sources of heterogeneity. This meta-analysis did not uncover publication bias. Sensitivity analysis showed that the study was robust. Fagan’s nomogram results showed clinical usability. Conclusions. The results from this meta-analysis indicate that 16S rRNA PCR testing is effective for the rapid diagnosis of neonatal sepsis.
Neonatal sepsis, a systemic inflammatory reaction to the invasion of bacteria, viruses, fungi, and other pathogens in a newborn’s blood, produces toxins and is the most common form of infectious disease among newborns . Globally, neonatal sepsis has an intimate connection with a 2.2% morbidity rate and a mortality rate of 11-19%, with higher mortality in developing countries [2, 3]. Clinical signs of neonatal sepsis are often aspecific, which limits initial diagnosis. Late diagnosis leads to disease progression, resulting in multiple organ failure and even death. Thus, early diagnosis of sepsis and early treatment are key to successful outcomes.
The gold standard for the diagnosis of neonatal sepsis is through culture of the microorganisms from patient blood or other body fluids, such as urine, and cerebrospinal fluid. However, due to factors such as insufficient samples, maternal use of antibiotics, and antibiotic use before sampling, this method may give false negative results [4, 5]. Various biomarkers, including C-reactive protein (CRP), procalcitonin (PCT), neutrophil CD64, interleukin-8, and interleukin-27, are used for sepsis diagnosis [6–11]. However, these biomarkers may also be elevated in noninfectious conditions such as premature rupture of membranes, fetal distress, dystocia, and perinatal asphyxia, resulting in false positive results and low specificity for neonatal sepsis . Thus, there is an urgent need for faster, more sensitive tests for neonatal sepsis diagnosis.
Recently, the PCR technique has been universal deployed in clinical diagnoses, which makes it possible to diagnose infectious diseases caused by microorganisms quickly and accurately. Moreover, numerous studies have shown that 16S rRNA PCR can diagnose neonatal sepsis [12–17]. The 16S rRNA gene is 1500 nucleotides long and encodes the 30S ribosomal subunit in all prokaryotes. The 16S rRNA gene is highly conserved and does not change over time. Within a certain range, the 16S rRNA gene can accurately identify specific bacteria based on gene-specific signatures [18, 19]. Relative to culture techniques, 16S rRNA PCR is cost-effective and rapid . Here, we conducted a systematic review and meta-analysis of published studies to determine the utility of 16S rRNA PCR in neonatal sepsis diagnosis.
2.1. Search Strategy
Two authors independently performed literature searches of PubMed, Medline, Embase, and the Cochrane Library using the search terms 16S rRNA, 16S ribosomal ribonucleic acid, septic, septicemia, and neonatal sepsis. No restrictions were applied on the search, which included studies published until 13 January 2021. To ensure comprehensive literature identification, we manually searched relevant references in the identified studies.
2.2. Study Selection
The following inclusion criteria were used: (1) samples were neonatal blood; (2) true positive (TP), true negative (TN), false positive (FP), and false negative (FN) values could be directly or indirectly obtained; (3) all data were derived from 16S rRNA PCR tests for neonatal sepsis diagnosis. The exclusion criteria were as follows: (1) literature in the form of review, meta-analysis, case report, or letter; (2) studies that were not clearly defined as involving neonates; (3) studies in with insufficient data for meta-analysis; and (4) studies involving nonblood samples.
2.3. Data Extraction
The following data were extracted by 2 authors: first author name, publication year, region, TN, FN, TP, FP, test method, and test sample. Disagreements were resolved by a 3rd author.
2.4. Quality Assessment
The inclusion criteria and methodological quality of selected articles were evaluated by 2 independent authors using QUADAS-2 , and disagreements were resolved by a 3rd author.
2.5. Statistical Analysis
Study heterogeneity was evaluated by test. Heterogeneity due to the threshold effect was evaluated by the Spearman model. When study heterogeneity was statistically significant (% or ), the random effect model was used; otherwise, a fixed effects model was used [22, 23]. To evaluate 16S rRNA PCR potential and accuracy in neonatal sepsis diagnosis, sensitivity, specificity, PLR, NLR, DOR, and AUC of SROC curve analyses were used. Metaregression analysis was used to determine heterogeneity sources. Deeks’ funnel plot asymmetry test was used to evaluate publication bias . Sensitivity analysis was used to evaluate the robustness of this study. Statistical analyses were performed by Stata 14.0 and Meta-disc 1.4.
3.1. Study Characteristics
A total of 2545 studies were identified, and 1058 duplicate studies were eliminated. Upon title and abstract review, 1368 studies were excluded, and 119 were subjected to full-text review. Of these, 100 were excluded because valid data could not be extracted, and the remaining 19 articles were included in our study [12, 13, 25–41] (Figure 1).
The19 articles incorporated into in our study involved a comparison between the diagnostic value of 16S rRNA PCR and blood culture for pathogenic microorganism identification in neonatal sepsis patients. The important features of the 19 articles are displayed in Table 1. They involved a total of 4740 neonatal blood samples, of which 553 were positive and 4187 were negative. The included studies were from Turkey (1), China (6), Egypt (3), India (4), Israel (1), the US (1), Japan (1), Italy (1), and Sweden (1). 16S rRNA amplification was achieved by PCR.
3.2. Quality Assessment
Methodological quality and risk of bias in the included studies were assessed using QUADAS-2. All studies used a prospective study design to avoid inappropriate exclusion (Figure 2). Five studies did not specify whether patients were continuously enrolled or not [25, 26, 28–30]. The remaining studies specified continuous enrollment. The reference standard in all studies was pathogenic microorganism blood culture. Some studies did not report sufficient data on indicator tests and/or reference criteria, so these items were used as ambiguous risk of bias scores. QUADAS-2 did not include an overall bias score, but the overall quality of the studies included in the analysis was moderate to high.
3.3. Heterogeneity Analysis and Diagnostic Accuracy
The sensitivity was 99.95 (95% CI: 99.94-99.55), . And the specificity was 99.32 (95% CI: 99.23-99.42), . Because the results point out heterogeneity among the studies, the random effect model was adopted. Analysis results were displayed in Figure 3. The overall sensitivity and specificity of the 19 studies were 0.98 (95% CI: 0.85-1.00) and 0.94 (95% CI: 0.87-0.97), respectively. The PLR was 16.0 (95% CI: 7.6-33.9), the NLR was 0.02 (95% CI: 0.00-0.18), and the DOR was 674 (95% CI: 89-5100). The SROC curve analysis of the 16S rRNA gene PCR test accuracy in neonatal sepsis diagnosis revealed an AUC of 0.99 (95% CI: 0.97-0.99; Figure 4).
3.4. Subgroup Analysis and Metaregression
In order to investigate the potential sources of heterogeneity, we conducted threshold effect analysis using Meta-disc 1.4 and obtained a Spearman correlation coefficient of 0.262 (), indicating that the threshold effect was not the source of the heterogeneity. Next, metaregression analysis was used to divide the subgroups into location (Asian vs. non-Asian), specimen (arterial vs. nonarterial blood), center (single center vs. multicenter), and sample size (≥200 vs. <200) (Figure 5). The main sources of sensitivity heterogeneity were location, specimen, center, and sample size. Specificity heterogeneity was mainly due to sample size.
3.5. Sensitivity Analysis
Sensitivity analysis was used to evaluate the reliability and robustness of the analysis results. The validity and robustness of the models involved in the statistical analyses were verified by goodness-of-fit and bivariate normality analysis (Figures 6(a) and 6(b)). Influence analysis (Figure 6(c)) showed 3 influence studies, and outlier detection (Figure 6(d)) found 1 outlier study. Sequential exclusion of influencing factors and outliers did not significantly alter the overall results (Table 2).
3.6. Clinical Utility of the Index Test
To evaluate posttest probabilities, Fagan’s nomogram could be used to calculate the posttest probability of 16S rRNA PCR for neonatal sepsis diagnosis. When the pretest probability was set at 11%, it was found that the probability of neonatal sepsis was 0.66 if the results were positive and 0 if the results were negative (Figure 7). A likelihood ratio scatter plot showed that 16S rRNA PCR was effective for neonatal sepsis diagnosis (positive) and exclusion (negative), with the summary point of the probability ratio in the upper left quadrant (Figure 8).
3.7. Publication Bias
Deeks’ funnel plot asymmetry test on the 19 included studies found no publication bias (Figure 9, ).
Past studies demonstrated the high potential of 16S rRNA PCR tests for diagnosing bloodstream infections. This strategy has been suggested to be effective and fast for screening sepsis . In this meta-analysis, we evaluated the performance of 16S rRNA PCR for neonatal sepsis diagnosis relative to blood culture techniques. 16S rRNA PCR tests have the potential to accelerate neonatal sepsis diagnosis, thereby ensuring timely and effective treatment.
Our analysis found that the sensitivity and specificity of 16S rRNA PCR tests for neonatal sepsis diagnosis were 0.98 (95% CI: 0.85-1.00) and 0.94 (95% CI: 0.87-0.97), respectively, indicating high diagnostic effectiveness. The SROC curve showed the trade-off between the sensitivity and specificity of diagnostic research, and the AUC of the SROC curve is a measure of the integrity of the diagnostic testing ability, providing a precise basis for the overall study . Our analysis revealed an AUC of 0.99 (95% CI: 0.97-0.99), indicating that 16S rRNA PCR is highly accurate for neonatal sepsis diagnosis. The DOR is a way of the usefulness of a diagnostic test and is given as a value between 0 and ∞, and a higher value means better performance. Conversely, a value of <1 indicates that the test lacks the ability to distinguish between outcomes . Our analysis revealed a DOR value of 674 (95% CI: 89-5100), indicating high accuracy. The likelihood ratio fully reflects the diagnostic value of a screening test and is very stable. The DOR comprises the PLR (ratio of true positive rate to false positive rate, where the larger the ratio is, the greater the likelihood of a true positive test result) and the NLR (ratio of false negative rate to true negative rate, where the smaller the ratio is, the greater the chance of a true negative test result). Our pooled results revealed a PLR of 16.0 (95% CI: 7.6-33.9) and NLR of 0.02 (95% CI: 0.00-0.18), indicating that 16S rRNA PCR has good diagnostic ability.
Although the 16S rRNA PCR test was effective, there was significant heterogeneity in this meta-analysis. Different regions were also major sources of heterogeneity in this study. In the Asian population relative to the non-Asian populations, the sensitivity, specificity, PLR, NLR, DOR, and AUC were 1.00 (0.89-1.00) vs. 0.71 (0.41-0.89), 0.91 (0.79-0.97) vs. 0.96 (0.89-0.98), 11.5 (4.4-30.3) vs. 16.7 (7.0-39.5), 0 (0-0.14) vs. 0.31 (0.13-0.72), 4297 (64-289895) vs. 55 (17-180), and 1.00 (0.99-1.00) vs. 0.95 (0.93-0.97), respectively. Blood sample sources, sample sizes, and single- or multicenter studies were also sources of heterogeneity. However, because there were few multicenter studies, more standardized multicenter studies are needed to better understand the value of 16S rRNA PCR tests in neonatal sepsis diagnosis .
Our meta-analysis has some limitations. First, the included studies defined sepsis using different criteria, which may be reflected in different clinical symptoms and routine blood tests for the included patients. Although blood culture, the gold standard for sepsis diagnosis, was used in all studies, the associated false positive rate was high . Second, the kits and testing tools used for blood cultures and testing were manufactured by different companies, and there are no studies on whether the results vary by kit manufacturer. Third, most studies included in this meta-analysis did not distinguish between early-onset sepsis and late-onset sepsis. Thus, we could not carry out subgroup analysis between early-onset sepsis and late-onset sepsis.
In summary, this meta-analysis shows that 16S rRNA PCR tests are effective for rapid neonatal sepsis diagnosis. However, PCR amplification methods are not fully defined, and future prospective studies should carry out subgroup analysis of PCR methods.
The data of Table 1 used to support the findings of this study are included within the article (see References).
Conflicts of Interest
No conflicts of interest are declared.
This project was sponsored by the Science and Technology Planning Project of Chengde (grant nos. 202006A049 and 202006A088).
- F. Kim, R. A. Polin, and T. A. Hooven, “Neonatal sepsis,” BMJ, vol. 371, article m3672, 2020.
- Y. Dong, R. Basmaci, L. Titomanlio, B. Sun, and J. C. Mercier, “Neonatal sepsis: within and beyond China,” Chinese Medical Journal, vol. 133, no. 18, pp. 2219–2228, 2020.
- C. Fleischmann-Struzek, D. M. Goldfarb, P. Schlattmann et al., “The global burden of paediatric and neonatal sepsis: a systematic review,” The Lancet Respiratory Medicine, vol. 6, no. 3, pp. 223–230, 2018.
- I. O. Odabasi and A. Bulbul, “Neonatal Sepsis,” Sisli Etfal Hastan Tip Bul, vol. 54, no. 2, pp. 142–158, 2020.
- M. Satar, A. E. Arısoy, and I. H. Celik, “Turkish Neonatal Society guideline on neonatal infections-diagnosis and treatment,” Turk Pediatri Ars, vol. 53, no. 1, pp. S88–S100, 2018.
- M. Tsokos, U. Reichelt, A. Nierhaus, and K. Püschel, “Serum procalcitonin (PCT): a valuable biochemical parameter for the post-mortem diagnosis of sepsis,” International Journal of Legal Medicine, vol. 114, no. 4-5, pp. 237–243, 2001.
- A. V. Raveendran, A. Kumar, and S. Gangadharan, “Biomarkers and newer laboratory investigations in the diagnosis of sepsis,” The Journal of the Royal College of Physicians of Edinburgh, vol. 49, no. 3, pp. 207–216, 2019.
- J. Zhao, S. Zhang, L. Zhang et al., “Serum procalcitonin levels as a diagnostic marker for septic arthritis: A meta-analysis,” The American Journal of Emergency Medicine, vol. 35, no. 8, pp. 1166–1171, 2017.
- Y. Song, Y. Chen, X. Dong, and X. Jiang, “Diagnostic value of neutrophil CD64 combined with CRP for neonatal sepsis: A meta-analysis,” The American Journal of Emergency Medicine, vol. 37, no. 8, pp. 1571–1576, 2019.
- M. Zhou, S. Cheng, J. Yu, and Q. Lu, “Interleukin-8 for diagnosis of neonatal sepsis: a meta-analysis,” PLoS One, vol. 10, no. 5, article e0127170, 2015.
- Y. Wang, J. Zhao, Y. Yao, D. Zhao, and S. Liu, “Interleukin-27 as a Diagnostic Biomarker for Patients with Sepsis: A Meta- Analysis,” BioMed Research International, vol. 2021, Article ID 5516940, 7 pages, 2021.
- R. Yu, Q. Zhou, S. Jiang, Y. Mei, and M. Wang, “Combination of 16S rRNA and procalcitonin in diagnosis of neonatal clinically suspected sepsis,” Journal of International Medical Research, vol. 48, no. 3, article 030006051989241, 2020.
- A. K. Yadav, C. G. Wilson, P. L. Prasad, and P. K. Menon, “Polymerase chain reaction in rapid diagnosis of neonatal sepsis,” Indian Pediatrics, vol. 42, no. 7, pp. 681–685, 2005.
- J. A. Jordan and M. B. Durso, “Real-time polymerase chain reaction for detecting bacterial DNA directly from blood of neonates being evaluated for sepsis,” The Journal of Molecular Diagnostics, vol. 7, no. 5, pp. 575–581, 2005.
- C. Oeser, M. Pond, P. Butcher et al., “PCR for the detection of pathogens in neonatal early onset sepsis,” PLoS One, vol. 15, no. 1, article e0226817, 2020.
- A. Ohlin, A. Bäckman, U. Ewald, J. Schollin, and M. Björkqvist, “Diagnosis of neonatal sepsis by broad-range 16S real-time polymerase chain reaction,” Neonatology, vol. 101, no. 4, pp. 241–246, 2012.
- C. S. Buhimschi, V. Bhandari, Y. W. Han et al., “Using proteomics in perinatal and neonatal sepsis: hopes and challenges for the future,” Current Opinion in Infectious Diseases, vol. 22, no. 3, pp. 235–243, 2009.
- D. L. Church, L. Cerutti, A. Gürtler, T. Griener, A. Zelazny, and S. Emler, “Performance and Application of 16S rRNA Gene Cycle Sequencing for Routine Identification of Bacteria in the Clinical Microbiology Laboratory,” Clinical Microbiology Reviews, vol. 33, no. 4, 2020.
- K. Fukuda, M. Ogawa, H. Taniguchi, and M. Saito, “Molecular Approaches to Studying Microbial Communities: Targeting the 16S Ribosomal RNA Gene,” Journal of UOEH, vol. 38, no. 3, pp. 223–232, 2016.
- R. M. Hassan, M. G. El Enany, and H. H. Rizk, “Evaluation of broad-range 16S rRNA PCR for the diagnosis of bloodstream infections: two years of experience,” Journal of Infection in Developing Countries, vol. 8, no. 10, pp. 1252–1258, 2014.
- P. F. Whiting, A. W. Rutjes, M. E. Westwood et al., “QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies,” Annals of Internal Medicine, vol. 155, no. 8, pp. 529–536, 2011.
- T. B. Huedo-Medina, J. Sánchez-Meca, F. Marín-Martínez, and J. Botella, “Assessing heterogeneity in meta-analysis: Q statistic or I2 index?” Psychological Methods, vol. 11, no. 2, pp. 193–206, 2006.
- F. L. Schmidt, I. S. Oh, and T. L. Hayes, “Fixed- versus random-effects models in meta-analysis: model properties and an empirical comparison of differences in results,” The British Journal of Mathematical and Statistical Psychology, vol. 62, no. 1, pp. 97–128, 2009.
- J. J. Deeks, P. Macaskill, and L. Irwig, “The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed,” Journal of Clinical Epidemiology, vol. 58, no. 9, pp. 882–893, 2005.
- K. İstanbullu, N. Köksal, M. Çetinkaya et al., “The potential utility of real-time PCR of the 16S-rRNA gene in the diagnosis of neonatal sepsis,” The Turkish Journal of Pediatrics, vol. 61, no. 4, pp. 493–499, 2019.
- M. I. El-Amir, M. A. El-Feky, D. A. Abo Elwafa, and E. A. Abd-Elmawgood, “<p>Rapid diagnosis of neonatal sepsis by PCR for detection of 16S rRNA gene, while blood culture and PCR results were similar in <em>E.coli</em>-predominant EOS cases</p>,” Infect Drug Resist, vol. 12, pp. 2703–2710, 2019.
- H. Punia, G. Gathwala, D. B. Dhaulakhandi, and M. Aamir, “Diagnosis of neonatal sepsis using 16S rRNA polymerase chain reaction,” Tropical Doctor, vol. 47, no. 4, pp. 336–339, 2017.
- D. A. Midan, W. M. M. Abo El Fotoh, and A. H. El Shalakany, “The potential role of incorporating real-time PCR and DNA sequencing for amplification and detection of 16S rRNA gene signatures in neonatal sepsis,” The Journal of Maternal-Fetal & Neonatal Medicine, vol. 30, no. 12, pp. 1476–1483, 2017.
- A. Rohit, B. Maiti, S. Shenoy, and I. Karunasagar, “Polymerase chain reaction-restriction fragment length polymorphism (PCR-RFLP) for rapid diagnosis of neonatal sepsis,” The Indian Journal of Medical Research, vol. 143, no. 1, pp. 72–78, 2016.
- S. el Gawhary, M. el-Anany, R. Hassan, D. Ali, and E. Q. el Gameel, “The Role of 16S rRNA Gene Sequencing in Confirmation of Suspected Neonatal Sepsis,” Journal of Tropical Pediatrics, vol. 62, no. 1, pp. 75–80, 2016.
- S. Dutta, A. Narang, A. Chakraborty, and P. Ray, “Diagnosis of neonatal sepsis using universal primer polymerase chain reaction before and after starting antibiotic drug therapy,” Archives of Pediatrics & Adolescent Medicine, vol. 163, no. 1, pp. 6–11, 2009.
- C. L. Liu, H. W. Ai, W. P. Wang et al., “Comparaison entre l'ARNr 16S et l'hemoculture pour le diagnostic de septicemie neonatale,” Archives de Pédiatrie, vol. 21, no. 2, pp. 162–169, 2014.
- M. Fujimori, K. Hisata, S. Nagata et al., “Efficacy of bacterial ribosomal RNA-targeted reverse transcription-quantitative PCR for detecting neonatal sepsis: a case control study,” BMC Pediatrics, vol. 10, no. 1, p. 53, 2010.
- A. Ohlin, A. Bäckman, M. Björkqvist, P. Mölling, M. Jurstrand, and J. Schollin, “Real-time PCR of the 16S-rRNA gene in the diagnosis of neonatal bacteraemia,” Acta Paediatrica, vol. 97, no. 10, pp. 1376–1380, 2008.
- Y. D. Wu, S. Q. Shang, J. P. Li et al., “A broad-range 16S rRNA gene real-time PCR assay for the diagnosis of neonatal septicemia,” Zhonghua Er Ke Za Zhi, vol. 45, no. 6, pp. 446–449, 2007.
- J. A. Jordan, M. B. Durso, A. R. Butchko, J. G. Jones, and B. S. Brozanski, “Evaluating the near-term infant for early onset sepsis: progress and challenges to consider with 16S rDNA polymerase chain reaction testing,” The Journal of Molecular Diagnostics, vol. 8, no. 3, pp. 357–363, 2006.
- I. R. Makhoul, T. Smolkin, P. Sujov et al., “PCR-based diagnosis of neonatal staphylococcal bacteremias,” Journal of Clinical Microbiology, vol. 43, no. 9, pp. 4823–4825, 2005.
- S. Shang, G. Chen, Y. Wu, L. Du, and Z. Zhao, “Rapid diagnosis of bacterial sepsis with PCR amplification and microarray hybridization in 16S rRNA gene,” Pediatric Research, vol. 58, no. 1, pp. 143–148, 2005.
- M. Q. Tong, S. Q. Shang, Y. D. Wu, and Z. Y. Zhao, “Rapid diagnosis of neonatal sepsis by 16SrRNA genes PCR amplification and genechip hybridization,” Zhonghua Er Ke Za Zhi, vol. 42, no. 9, pp. 663–667, 2004.
- S. Shang, Z. Chen, and X. Yu, “Detection of bacterial DNA by PCR and reverse hybridization in the 16S rRNA gene with particular reference to neonatal septicemia,” Acta Paediatrica, vol. 90, no. 2, pp. 179–183, 2001.
- N. Laforgia, B. Coppola, R. Carbone, A. Grassi, A. Mautone, and A. Iolascon, “Rapid detection of neonatal sepsis using polymerase chain reaction,” Acta Paediatrica, vol. 86, no. 10, pp. 1097–1099, 1997.
- G. Su, Z. Fu, L. Hu, Y. Wang, Z. Zhao, and W. Yang, “16S Ribosomal Ribonucleic Acid Gene Polymerase Chain Reaction in the Diagnosis of Bloodstream Infections: A Systematic Review and Meta-Analysis,” PLoS One, vol. 10, no. 5, article e0127195, 2015.
- S. D. Walter, “Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data,” Statistics in Medicine, vol. 21, no. 9, pp. 1237–1256, 2002.
- F. Tuzun, H. Ozkan, M. Cetinkaya et al., “Is European Medicines Agency (EMA) sepsis criteria accurate for neonatal sepsis diagnosis or do we need new criteria?” PLoS One, vol. 14, no. 6, article e0218002, 2019.
Copyright © 2021 Ying Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.