BioMed Research International

BioMed Research International / 2020 / Article

Research Article | Open Access

Volume 2020 |Article ID 7434737 | 9 pages | https://doi.org/10.1155/2020/7434737

Nomogram Based on microRNA Signature Contributes to Improve Survival Prediction of Clear Cell Renal Cell Carcinoma

Academic Editor: Maria Stangou
Received18 Sep 2019
Revised03 Mar 2020
Accepted11 Mar 2020
Published24 Mar 2020

Abstract

Objective. Numerous microRNAs (miRNAs) have been identified in ccRCC and recommended to be used for predicting clear cell renal cell carcinoma (ccRCC) prognosis. However, it is not clear whether a miRNA-based nomogram results in improved survival prediction in patients with ccRCC. Methods. miRNA profiles from tumors and normal tissues were downloaded from The Cancer Genome Atlas (TCGA) database and analyzed using the “limma” package. The association between differentially expressed miRNAs and patient prognosis was identified using univariate, least absolute shrinkage and selection operator (LASSO), and multivariate Cox regression analyses. Next, all patients were randomly divided into development and validation cohorts at a ratio of 1 : 1. A nomogram was established based on independent prognostic factors in the development cohort. The prognostic performance of the nomogram was validated in both cohorts using the concordance index (C-index) and calibration plots. Results. Multivariate Cox analysis identified the 13-miRNA signature, as well as AJCC stage and age, as independent prognostic factors after adjusting for other clinical covariates. The nomogram was built based on the independent variables. In the development cohort, the C-index for the constructed nomogram to predict overall survival (OS) was 0.792, which was higher than the C-index (0.731) of the AJCC staging system and C-index (0.778) of the miRNA signature. The nomogram demonstrated good discriminative ability in the validation cohort in predicting OS, with a C-index of 0.762. The calibration plots indicated an excellent agreement between the nomogram predicted survival probability and the actual observed outcomes. Furthermore, decision curve analysis (DCA) indicated that the nomogram was superior to the AJCC staging system in increasing the net clinical benefit. Conclusions. The novel proposed nomogram based on a miRNA signature is a more reliable and robust tool for predicting the OS of patients with ccRCC compared to AJCC staging system, thus, improving clinical decision-making.

1. Introduction

Renal cell carcinoma (RCC) is a common genitourinary malignant tumor, accounting for 2–3% of all adult cancers, resulting in over 100,000 annual deaths worldwide [1, 2]. More than 66,800 newly diagnosed renal cancer patients and 13,860 deaths were reported in China in 2014 [3]. In the USA, approximately 63,920 newly diagnosed individuals and 13,860 renal cancer-related deaths were reported in 2014 [4]. The most common subtype of renal cancer is ccRCC, which accounts for 70–80% of all RCC cases [5].

microRNAs (miRNAs) are small noncoding RNAs consisting of 19–25 nucleotides that function in the regulation of protein-coding or noncoding gene expression, mainly by binding to the 3untranslated region of target mRNAs [6]. Recently, many studies have revealed that abnormal miRNA expression can be utilized to evaluate the clinical prognosis of patients with various cancers, including ccRCC [711]. Currently, the prognosis of renal cancer mainly relies on the American Joint Committee on Cancer (AJCC) TNM staging system [12, 13]. However, the efficiency of its clinical application in predicting prognosis remains controversial [14, 15]. Moreover, the system does not incorporate other vital prognostic variables, such as age, sex, tumor size, primary site, and differentiation. Therefore, the development of a novel prognostic nomogram incorporating miRNA would be advantageous. To our knowledge, the availability of prognostic nomogram incorporating a miRNA signature is limited for patients with ccRCC. This study aimed to identify a miRNA signature and formulate a predictive nomogram which incorporates independent prognostic factors. The performance of the nomogram was compared with the presently used AJCC staging system.

2. Materials and Methods

2.1. Patients and Differentially Expressed miRNAs

In this study, miRNA expression data and clinical information from TCGA data portal (http://tcga.cancer.gov) were used. After excluding patients with incomplete clinical pathological information, a total of 504 patients were included. The ccRCC miRNA expression data were derived from 616 specimens, including 545 renal cancer specimens and 71 nontumor tissues. The “edgeR” package in the R software was used to screen differentially expressed miRNAs between cancer and normal tissues with a threshold of and adjusted value <0.05 [16].

2.2. Identification of a Potential Prognostic miRNAs Signature

Univariate Cox analyses were performed to identify the link between the expression level of miRNAs and patient survival, with statistical significance set at . LASSO analysis was then conducted on the miRNAs founded to be statistically significant in the univariate analysis. It conducted a subselection of miRNAs via shrinkage of the regression coefficient by way of forcing a penalty proportional to their size [17]. The miRNAs with a value <0.05 were further put through a multivariate Cox regression analysis to determinate the predictive miRNAs. Subsequently, a prognostic miRNA signature was established according to a liner combination of the expression levels of prognostic miRNAs weighted by the evaluated regression coefficient in the multivariate Cox regression model.

2.3. Identification of Independent Prognostic Variables and Nomogram Construction

The miRNA signature and other clinical variables were merged. The 504 patients were randomly divided into development () and validation () cohorts. To identify independent prognostic variables related to patient survival in the development cohort, univariate and multivariate Cox regression analysis were performed using prognosis as the dependent variable and miRNA signature and other related clinical factors including age, gender, stage, grade, and laterality as independent variables. Nomogram for the predictions of 1-, 3-, and 5-year survival was conducted according to the independent variables in the development cohort. The nomogram and calibration plots were performed using the R software (version 3.5.1, http://www.r-project.org/) with “rms” package.

2.4. Calibration and Validation of the Nomogram

The performance of our nomogram was validated by measuring the concordance index (C-index) and calibration curves in both cohorts [18]. Calibration plots were applied to examine the performance of the nomogram. A calibration plot along the 45-degree line in two cohorts would present an excellent calibration model between the bootstrap-predicted probabilities and the actual outcomes. The nomogram and AJCC staging system were compared using the rcorrp.cens package in R. The decision curve analysis (DCA) is a novel method that evaluates predictive models from the perspective of clinical consequences. DCA was performed in the entire cohort to test the clinical usefulness of the nomogram in comparison with the present AJCC staging system [19]. All statistical analyses were performed using the R software (v3.5.1, http://www.r-project.org). A two-tailed value <0.05 was considered statistically significant.

3. Results

3.1. Identification of the miRNA Signature

A total of 173 differentially expressed miRNAs were screened out. Among these miRNAs, 17 prognostic miRNAs were retained by the LASSO method. To build the optimal predictive signature, these potential miRNAs were further selected with a multivariate Cox regression analysis. Finally, 13 miRNAs were identified (Figure 1, Table 1), and a miRNA signature was constructed according to the expression levels of these miRNAs weighted by their relative coefficient derived from multivariate Cox regression. As a result, the prognostic risk score (miRNA signature) was determined for each of the 504 patients.


microRNAβHRHR 95% CI value

hsa-mir-1269a0.049371.050611.0111-1.09170.01162
hsa-mir-130b0.209871.233520.9537-1.59540.10983
hsa-mir-144-0.139270.869990.7529-1.00530.05904
hsa-mir-1490.017491.017640.8731-1.18610.82294
hsa-mir-18a0.367381.443951.1497-1.81350.00158
hsa-mir-210.205221.22780.9671-1.55870.09191
hsa-mir-2210.118941.126310.967-1.31190.12636
hsa-mir-2230.243471.275671.096-1.48480.00167
hsa-mir-365b0.259021.295661.052-1.59580.01482
hsa-mir-376a-10.04351.044460.8591-1.26970.66246
hsa-mir-376b0.07021.072720.8799-1.30770.48733
hsa-mir-4732-0.223160.799990.6466-0.98980.03995
hsa-mir-6160.130381.139260.9463-1.37160.16851

Note: β indicates the regression coefficient; HR: hazard ratio; CI: confidence interval.
3.2. Nomogram Construction

The demographic and clinical characteristics of the development and validation cohorts are shown in Table 2. We used the univariate and multivariate Cox regression analysis to select independent predictors for patients with ccRCC in the development cohort. Risk score, age, grade, and AJCC stage were found to be related to survival in the development cohort (Figure 2(a)). A multivariate adjustment for other factors indicated that the risk score, age, and AJCC stage are independent prognostic factors () in the development cohort (Figure 2(b)). A prognostic nomogram derived from these risk variables was established enabling the determination of an individual patient’s score of each variable and the estimation of the probability of survival (Figure 3). Furthermore, in the validation cohort, all patients were classified into the high-risk group and low-risk group. Kaplan-Meier plots, tested by log-rank method, were used to establish the potential relationship between patients’ survival and high-risk and low-risk groups. The Kaplan-Meier survival curve revealed that median survival time of the high-risk group is associated with shorter survival time than low-risk group with value of <0.0001, as showed by the log-rank test (Figure 4). Moreover, the reproducibility of the miRNA signature in predicting the RFS of ccRCC patients in two cohorts was also determined. Patients were divided into a high-risk group and a low-risk group as previously determined. It was revealed by the log-rank test that the survival analyses of patients with low-risk group had higher RFS than those with high-risk group in the development cohort (Figure 5(a), ) and in the validation cohort (Figure 5(b), ).


Development cohort ()Validation cohort () value

Age ()0.5099
Sex (, %)0.02493
 Male152 (60.32)176 (69.84)
 Female100 (39.68)76 (30.16)
AJCC stage (, %)0.7451
 I123 (48.81)125 (49.60)
 II23 (9.13)29 (11.51)
 III65 (25.79)57 (22.62)
 IV41 (16.27)41 (16.27)
Grade (, %)0.1368
 I3 (1.19)10 (3.97)
 II102 (40.48)113 (44.84)
 III107 (42.46)95 (37.70)
 IV40 (15.87)34 (13.49)
Laterality (, %)1
 Left117 (46.43)117 (46.43)
 Right135 (53.57)135 (53.57)
Risk score ()0.8244
Survival months ()0.9696
Survival status (, %)0.57161
 Dead82 (32.54)88 (34.92)
 Alive170 (67.46)164 (65.08)

SD: Standard Deviation; AJCC: American Joint Committee on Cancer.
3.3. Calibration and Validation of the Nomogram

We validated the performance of the nomogram in both cohorts. The nomogram exhibited favorable accuracy for OS prediction with a C-index of 0.792 (95% CI: 0.744-0.840). The C-index of the AJCC stages and miRNA signature alone for predicting survival in the development cohort was 0.731 (95% CI: 0.675-0.787), 0.778 (95% CI: 0.731-0.825), respectively, which were significantly lower than the nomogram. As for the validation cohort, the nomogram to predict OS also yielded favorable accuracy with a C-index of 0.762 (95% CI: 0.713-0.811). The C-index of the AJCC stages and miRNA signature alone for predicting survival was also lower than the nomogram at 0.717 (95% CI: 0.663-0.771), 0.685 (95% CI: 0.624-0.746), respectively. The calibration curves for the probabilities of 1-, 3-, and 5-year survivals indicated an excellent agreement between the nomogram prediction and observed outcomes in both the development and validation cohorts (Figure 6). These results indicate that the nomogram is better at predicting survival than the AJCC stage system and miRNA signature alone.

3.4. Decision Curve Analysis

In DCA, the established nomogram presented a higher net benefit together with a broader range of threshold probability than the AJCC staging system and the miRNA signature (Figure 7), which demonstrates that our nomogram showed powerful predictive ability for survival.

4. Discussion

CcRCC is one of the most prevalent kidney cancers with high mortality and morbidity rates [20]. Nevertheless, current models for predicting patient survival only utilize conventional clinical parameters. For this reason, effective identification of other predictive variables is clinically challenging. The development of ccRCC is closely associated with the activation of proto-oncogenes, as well as the inactivation of tumor suppressor genes controlled by numerous miRNAs [21]. However, a single miRNA does not play a complete role in determining the prognosis of ccRCC.

Researches have confirmed that miRNA is closely associated with the occurrence of many cancers, and numerous miRNA have been used as novel biomarkers in cancers prognosis [22, 23]. Various miRNAs have been revealed to play an important role in tumor carcinogenesis. A study reported that three independent prognostic miRNAs (hsa-mir-144, hsa-mir-21, and hsa-mir-155) participating in the competing endogenous RNA network in ccRCC [24]. A study based on TCGA database identified four miRNAs, miR-149-5p, miRNA-21-5p, miRNA-9-5p, and miRNA-30b-5p, as independent prognostic indicators in patients with ccRCC. These miRNA target genes were mainly related to many pathways associated with tumor [25]. However, it failed to build a nomogram and compared with the presently used AJCC staging system. Nomograms provide individual predicts of future clinical outcomes by combining the effects of various variables associated with these events. The nomogram has proven to be a reliable tool for predicting the clinical survival of many types of cancers [2630]. In this study, we screened 13 miRNAs, and a nomogram was then built based on the independent prognostic variables associated with patient survival. The nomogram exhibited favorable accuracy for OS prediction in both development and validation cohorts. The C-indexes of the AJCC stages and miRNA signature alone for predicting survival were also lower than the nomogram in both cohorts. The reproducibility of the miRNA signature in predicting the RFS of ccRCC patients in two cohorts was also determined. It is generally accepted that a C-index of >0.75 exhibits clearly useful discrimination [31]. These findings indicate that this novel prognostic model exhibits perfect performance.

In current clinical practice, the AJCC staging system has long been broadly used for prognostic evaluation of renal cancer [32]. However, many studies have suggested that this method has gradually lost its advantage in predicting patient survival time since other vital variables are neglected [14, 33]. A recent study established two prognostic nomograms that incorporated the log odds of positive lymph nodes for patients with RCC which found to be superior to the AJCC staging system in predicting cancer-specific survival and OS [34]. In our study, we corroborated this finding using a miRNA signature-based nomogram. However, perfect predictive ability does not indicate superior practicality in clinical practice [35]. Therefore, we used DCA to demonstrate that the novel nomogram has better clinical validity and applicability.

Although the novel nomogram demonstrated favorable accuracy, several limitations should be noted. Firstly, this nomogram was built based on the TCGA database, and external validation from other databases is needed. Secondly, several potential predictive factors were still not identified, including tumor size, chemotherapy, and radiotherapy, as they were unavailable in most patients.

5. Conclusion

In summary, miRNA signature is an independent prognostic factor for patients with ccRCC. The novel nomogram based on a miRNA signature is a powerful model for predicting the OS of patients with ccRCC compared to AJCC staging system, thus, improving clinical decision-making.

Data Availability

The raw data of this study are derived from the TCGA data portal (http://tcga.cancer.gov), which is a publicly available database.

Ethical Approval

This study does not contain any work with human participants conducted by any of the authors.

Conflicts of Interest

The authors declare that they have no competing interests.

Authors’ Contributions

Enfa Zhao is the principle investigator. Enfa Zhao and Xiaofang Bai conducted the statistical analysis and drafted the manuscript. Enfa Zhao performed data management and bioinformatics analysis. Enfa Zhao and Xiaofang Bai edited and revised the manuscript. All authors read and approved the final manuscript.

Acknowledgments

The authors gratefully acknowledge The Cancer Genome Atlas (TCGA) database, which made the data of ccRCC available.

References

  1. J. Brugarolas, “Molecular genetics of clear-cell renal cell carcinoma,” Journal of Clinical Oncology, vol. 32, no. 18, pp. 1968–1976, 2014. View at: Publisher Site | Google Scholar
  2. E. Calvo, M. Schmidinger, D. Y. C. Heng, V. Grünwald, and B. Escudier, “Improvement in survival end points of patients with metastatic renal cell carcinoma through sequential targeted therapy,” Cancer Treatment Reviews, vol. 50, pp. 109–117, 2016. View at: Publisher Site | Google Scholar
  3. W. Chen, R. Zheng, P. D. Baade et al., “Cancer statistics in China, 2015,” CA: a Cancer Journal for Clinicians, vol. 66, no. 2, pp. 115–132, 2016. View at: Publisher Site | Google Scholar
  4. R. Siegel, J. Ma, Z. Zou, and A. Jemal, “Cancer statistics, 2014,” CA: a Cancer Journal for Clinicians, vol. 64, no. 1, pp. 9–29, 2014. View at: Publisher Site | Google Scholar
  5. A. Lopez-Beltran, M. Scarpelli, R. Montironi, and Z. Kirkali, “2004 WHO classification of the renal tumors of the adults,” European Urology, vol. 49, no. 5, pp. 798–805, 2006. View at: Publisher Site | Google Scholar
  6. L. A. Yates, C. J. Norbury, and R. J. Gilbert, “The long and short of microRNA,” Cell, vol. 153, no. 3, pp. 516–519, 2013. View at: Publisher Site | Google Scholar
  7. F. C. Zeng, M. Q. Zeng, L. Huang et al., “Downregulation of VEGFA inhibits proliferation, promotes apoptosis, and suppresses migration and invasion of renal clear cell carcinoma,” OncoTargets and Therapy, vol. 9, pp. 2131–2141, 2016. View at: Publisher Site | Google Scholar
  8. B. Liang, J. Zhao, and X. Wang, “A three-microRNA signature as a diagnostic and prognostic marker in clear cell renal cancer: an in silico analysis,” PLoS One, vol. 12, no. 6, article e0180660, 2017. View at: Publisher Site | Google Scholar
  9. X. Chen, A. Ruan, X. Wang et al., “miR-129-3p, as a diagnostic and prognostic biomarker for renal cell carcinoma, attenuates cell migration and invasion via downregulating multiple metastasis-related genes,” Journal of Cancer Research and Clinical Oncology, vol. 140, no. 8, pp. 1295–1304, 2014. View at: Publisher Site | Google Scholar
  10. E. R. Maher, “Genomics and epigenomics of renal cell carcinoma,” Seminars in Cancer Biology, vol. 23, no. 1, pp. 10–17, 2013. View at: Publisher Site | Google Scholar
  11. M. D. Jansson and A. H. Lund, “MicroRNA and cancer,” Molecular Oncology, vol. 6, no. 6, pp. 590–610, 2012. View at: Publisher Site | Google Scholar
  12. M. Sun, S. F. Shariat, C. Cheng et al., “Prognostic factors and predictive models in renal cell carcinoma: a contemporary review,” European Urology, vol. 60, no. 4, pp. 644–661, 2011. View at: Publisher Site | Google Scholar
  13. B. Ljungberg, K. Bensalah, S. Canfield et al., “EAU guidelines on renal cell carcinoma: 2014 update,” European Urology, vol. 67, no. 5, pp. 913–924, 2015. View at: Publisher Site | Google Scholar
  14. V. Ficarra, G. Novara, A. Galfano, and W. Artibani, “Neoplasm staging and organ-confined renal cell carcinoma: a systematic review,” European Urology, vol. 46, no. 5, pp. 559–564, 2004. View at: Publisher Site | Google Scholar
  15. C. Terrone, P. Gontero, A. Volpe et al., “Proposal of an improved prognostic classification for pT3 renal cell carcinoma,” The Journal of Urology, vol. 180, no. 1, pp. 72–78, 2008. View at: Publisher Site | Google Scholar
  16. S. Anders and W. Huber, “Differential expression analysis for sequence count data,” Genome Biology, vol. 11, no. 10, p. R106, 2010. View at: Publisher Site | Google Scholar
  17. R. Tibshirani, “Regression shrinkage and selection via the lasso,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 58, no. 1, pp. 267–288, 1996. View at: Publisher Site | Google Scholar
  18. M. Wolbers, M. T. Koller, J. C. Witteman, and E. W. Steyerberg, “Prognostic models with competing risks: methods and application to coronary risk prediction,” Epidemiology, vol. 20, no. 4, pp. 555–561, 2009. View at: Publisher Site | Google Scholar
  19. A. J. Vickers and E. B. Elkin, “Decision curve analysis: a novel method for evaluating prediction models,” Medical Decision Making, vol. 26, no. 6, pp. 565–574, 2006. View at: Publisher Site | Google Scholar
  20. R. L. Siegel, K. D. Miller, and A. Jemal, “Cancer statistics, 2018,” CA: a Cancer Journal for Clinicians, vol. 68, no. 1, pp. 7–30, 2018. View at: Publisher Site | Google Scholar
  21. G. Aguiari, “MicroRNAs in clear cell renal cell carcinoma: biological functions and applications,” Journal of Kidney Cancer and VHL, vol. 2, no. 4, pp. 140–152, 2015. View at: Publisher Site | Google Scholar
  22. L. Fabris, Y. Ceder, A. M. Chinnaiyan et al., “The potential of microRNAs as prostate cancer biomarkers,” European Urology, vol. 70, no. 2, pp. 312–322, 2016. View at: Publisher Site | Google Scholar
  23. A. McGuire, J. A. Brown, and M. J. Kerin, “Metastatic breast cancer: the potential of miRNA for diagnosis and treatment monitoring,” Cancer Metastasis Reviews, vol. 34, no. 1, pp. 145–155, 2015. View at: Publisher Site | Google Scholar
  24. Y. Luo, L. Chen, G. Wang, Y. Xiao, L. Ju, and X. Wang, “Identification of a three-miRNA signature as a novel potential prognostic biomarker in patients with clear cell renal cell carcinoma,” Journal of Cellular Biochemistry, vol. 120, no. 8, pp. 13751–13764, 2019. View at: Publisher Site | Google Scholar
  25. M. Xie, Y. Lv, Z. Liu et al., “Identification and validation of a four-miRNA (miRNA-21-5p, miRNA-9-5p, miR-149-5p, and miRNA-30b-5p) prognosis signature in clear cell renal cell carcinoma,” Cancer Management and Research, vol. Volume 10, pp. 5759–5766, 2018. View at: Publisher Site | Google Scholar
  26. W. Liang, L. Zhang, G. Jiang et al., “Development and validation of a nomogram for predicting survival in patients with resected non-small-cell lung cancer,” Journal of Clinical Oncology, vol. 33, no. 8, pp. 861–869, 2015. View at: Publisher Site | Google Scholar
  27. S. Y. Ho, C. Y. Hsu, P. H. Liu et al., “Albumin-bilirubin grade-based nomogram of the BCLC system for personalized prognostic prediction in hepatocellular carcinoma,” Liver International, vol. 40, no. 1, pp. 205–214, 2019. View at: Publisher Site | Google Scholar
  28. Y. Liu, Z. Q. Zou, J. Xiao, M. Zhang, L. Yuan, and X. G. Zhao, “A nomogram prediction model for recurrent laryngeal nerve lymph node metastasis in thoracic oesophageal squamous cell carcinoma,” Journal of Thoracic Disease, vol. 11, no. 7, pp. 2868–2877, 2019. View at: Publisher Site | Google Scholar
  29. P. Nie, G. Yang, Z. Wang et al., “A CT-based radiomics nomogram for differentiation of renal angiomyolipoma without visible fat from homogeneous clear cell renal cell carcinoma,” European Radiology, vol. 30, no. 2, pp. 1274–1284, 2019. View at: Publisher Site | Google Scholar
  30. Y. Wang, W. Liu, Y. Yu et al., “CT radiomics nomogram for the preoperative prediction of lymph node metastasis in gastric cancer,” European Radiology, vol. 30, no. 2, pp. 976–986, 2019. View at: Publisher Site | Google Scholar
  31. J. Cai, J. Cheng, H. Li et al., “A nomogram for the prediction of cerebrovascular disease among patients with brain necrosis after radiotherapy for nasopharyngeal carcinoma,” Radiotherapy and Oncology, vol. 132, pp. 34–41, 2019. View at: Publisher Site | Google Scholar
  32. V. Ficarra, A. Galfano, M. Mancini, G. Martignoni, and W. Artibani, “TNM staging system for renal-cell carcinoma: current status and future perspectives,” The Lancet Oncology, vol. 8, no. 6, pp. 554–558, 2007. View at: Publisher Site | Google Scholar
  33. M. K. Gospodarowicz, D. Miller, P. A. Groome, F. L. Greene, P. A. Logan, and L. H. Sobin, “The process for continuous improvement of the TNM classification,” Cancer, vol. 100, no. 1, pp. 1–5, 2004. View at: Publisher Site | Google Scholar
  34. W. Zhou, C. Huang, and N. Yuan, “Prognostic nomograms based on log odds of positive lymph nodes for patients with renal cell carcinoma: a retrospective cohort study,” International Journal of Surgery, vol. 60, pp. 28–40, 2018. View at: Publisher Site | Google Scholar
  35. A. J. Vickers and A. M. Cronin, “Everything you always wanted to know about evaluating prediction models (but were too afraid to ask),” Urology, vol. 76, no. 6, pp. 1298–1301, 2010. View at: Publisher Site | Google Scholar

Copyright © 2020 Enfa Zhao and Xiaofang Bai. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

24 Views | 49 Downloads | 0 Citations
 PDF  Download Citation  Citation
 Download other formatsMore
 Order printed copiesOrder

We are committed to sharing findings related to COVID-19 as quickly and safely as possible. Any author submitting a COVID-19 paper should notify us at help@hindawi.com to ensure their research is fast-tracked and made available on a preprint server as soon as possible. We will be providing unlimited waivers of publication charges for accepted articles related to COVID-19.