Disease Markers

Disease Markers / 2018 / Article

Research Article | Open Access

Volume 2018 |Article ID 3784589 | 11 pages | https://doi.org/10.1155/2018/3784589

Amino Acid-Based Metabolic Panel Provides Robust Prognostic Value Additive to B-Natriuretic Peptide and Traditional Risk Factors in Heart Failure

Academic Editor: Giuseppe Biondi-Zoccai
Received13 Apr 2018
Revised23 Aug 2018
Accepted29 Aug 2018
Published10 Oct 2018


Metabolic disturbances represent functional perturbation in peripheral tissues and predict outcomes in patients with heart failure (HF). This study developed an amino acid-based metabolic panel and sought to see whether this panel could add diagnostic and prognostic value to currently used B-type natriuretic peptide (BNP) measurements. Mass spectrometry and ultra-performance liquid chromatography were performed on 1288 participants, including 129 normal controls and 712 patients at HF stages A to D in the initial cohort and 447 stage C patients in the validation cohort. Patients were followed up for composite events (death/HF-related rehospitalization). Histidine, ornithine, and phenylalanine were 3 metabolites found strongly significant to identify patients at stage C and were adopted to develop the HOP panel. Compared to BNP, HOP had better value in discriminating the patients at different stages, especially in elderly patients and those with atrial fibrillation, high body mass index, or kidney dysfunction. HOP was correlated with the distance of 6 min walking distance better than BNP. For prognosis, HOP predicted composite events in patients at stages C and D, independent of log (BNP), age, sex, left ventricular ejection fraction, New York Heart Association functional class, HF stage, diabetes mellitus, chronic kidney disease, hypertension, hemoglobin, and albumin. Higher BNP (≥750 pg/mL) along with higher HOP (≥14) robustly predicted lower event-free survival compared to all others [hazard (2.23–4.46), ]. The prognostic value of HOP was confirmed in the validation cohort. In conclusion, aiming for clinical applications, this study proved that the HOP panel provides diagnostic and prognostic value additive to BNP and traditional risk factors.

1. Introduction

Metabolomics is a rapidly growing field with the potential to provide a comprehensive molecular landscape for describing metabolic health, tracking response to treatment, and monitoring disease recurrence [1]. A number of metabolomics studies of patients with HF have provided novel metabolic panels for diagnosis and prognosis [25]. Metabolic disturbances identified by metabolomics in plasma represent functional perturbation in peripheral tissues and potentially compensate for the value of B-type natriuretic peptide (BNP), which majorly indicates cardiac wall stress [6, 7].

Metabolites applied for clinical diagnosis and prognosis included amino acids and a variety of lipids [15]. Although metabolomics provides complimentary information on metabolism, it is not applicable for clinical use at current status. Considerable practical issues include the availability of mass spectrometry, stability of measuring low concentration metabolites, and interpretation of sophisticated metabolite profiles. Amino acids are metabolites that are feasible to quantify by ultra-performance liquid chromatography (UPLC) available worldwide. Targeting amino acids with relatively high plasma concentrations and measurable by UPLC offers a promising solution. To establish a clinically applicable metabolite panel, the aims of this study included the following: (1) based on the dataset of normal controls and patients at stage C HF (according to the American College of Cardiology and the American Heart Association HF classification system) [8], to identify the potential diagnostic metabolites from high concentration amino acids and develop a UPLC-based amino acid panel; (2) to test the ability of this panel in discriminating different stages of HF; and (3) based on the dataset from HF patients with potential of events (at HF stages C and D), to see whether this panel could improve prognostic value. The prognostic value was further tested in an independent validation cohort.

2. Methods

2.1. Patients and Study Design

For the initial study, we enrolled patients at HF stages A to D and normal controls from January 2011 to May 2014. HF stages A to D were classified according to the American College of Cardiology and the American Heart Association HF classification system [8]. Patients at stage C were those hospitalized due to acute or decompensated chronic HF and aged 20–85 years, with a left ventricular ejection fraction . Patients at stage D were those with marked symptoms after receiving optimal medical therapy. Patients at stage B were asymptomatic and included postacute myocardial infarction patients regardless of their LVEF, those with any severe structural abnormalities, and those with an LVEF of <50%. Patients at stage A included (i) asymptomatic patients with risk factors, with or without angiogram-documented coronary disease, and with an LVEF of >50% and (ii) those without structural heart disease. Meanwhile, normal controls were aged 20–85 years and had no significant systemic disease, such as hypertension, diabetes mellitus, or coronary artery disease. They were not on any medications and had an LVEF of >60%.

Exclusion criteria included (i) the presence of systemic diseases such as hypothyroidism, decompensated liver cirrhosis, and systemic lupus erythematosus; (ii) the presence of disorders other than HF that might compromise survival within 6 months; (iii) patients being bedridden for >3 months and/or unable to stand alone; (iv) patients with a serum creatinine of >3 mg/dL; and (v) patients with severe coronary artery disease without complete revascularization therapy. Informed consent was obtained from all patients. The study was designed and carried out in accordance with the principles of the Declaration of Helsinki and with approval from the Ethics Review Board of Chang Gung Memorial Hospital.

To validate the prognostic value of the metabolic panel, a second independent cohort, including 447 patients at stage C as defined above, was recruited from July 2013 to January 2016. (The study flow diagram is provided in Supplementary Figure I.)

2.2. Blood Sampling and Examination

For patients at stages C and D, blood samples for metabolomics were collected in the early morning after fasting for 8 hours before discharge from the hospital in EDTA-containing tubes. Plasma was analyzed by metabolomics workflow. BNP was measured with the Triage BNP Test (Biosite, San Diego, CA), which was a fluorescence immunoassay for quantitative determination of plasma BNP. Precision, analytical sensitivity, and stability characteristics of the assay were previously described [6]. For normal controls and patients at stages A and B, blood samples for metabolomics analysis and BNP were collected at enrollment. Measurement of other parameters, including kidney function, hemoglobin, γ-glutamyltransferase (γGT), aspartate aminotransferase (AST), alanine aminotransferase (ALT), estimated glomerular filtration rate (eGFR), and albumin, was conducted in the central core laboratory.

2.3. Follow-Up Program

For patients at stages C and D, follow-up data were prospectively obtained every month from hospital records, personal communication with the patients’ physicians, telephone interviews, and patients’ regular visits to staff physician outpatient clinics for 3 years. “Rehospitalization” was defined as HF-related rehospitalizations. A committee of 3 cardiologists adjudicated all hospitalizations without knowledge of patients’ clinical variables to determine whether the events are related to worsening HF. “All-cause death” was chosen as an endpoint because of the interrelationship of HF with other comorbidities in the patient cohort. Only was the composite event of HF-related rehospitalization and all-cause death analyzed for the prognostic purpose. In the second independent population for validation, patients were followed up for one year.

2.4. Stable Isotope Dilution-Multiple Reaction Monitoring Mass Spectrometry

The metabolomic analyses were carried out with the AbsoluteIDQ® p180 Kit (Biocrates Life Sciences AG, Innsbruck, Austria). The kit enables us to identify and quantify 21 amino acids. All reagents used in this analysis were of LC-MS grade. Ten μL aliquot of each plasma sample was mixed with isotopically labeled internal standards in a multititer plate and dried under nitrogen. Amino acids were derivatized with 5% phenyl isothiocyanate for 20 min and subsequently dried under nitrogen. Three hundred μL of extraction solvent (5 mM ammonium acetate in methanol) was added, and after 30 min incubation, it was centrifuged for 2 min at 100 × g. Subsequently, 150 μL of filtrate was transferred to a microtiter plate and diluted with 150 μL of water for analysis of amino acids by LC-MS/MS. The remaining filtrate was mixed with 400 μL of running solvent for flow injection analysis coupled with tandem mass spectrometric analysis (FIA-MS/MS). The analysis was performed in positive and negative electrospray ionization mode. Identification and quantification were achieved by multiple reaction monitoring. It was standardized by spiking in of isotopically labeled standards. LC-MS analysis was performed with Waters Xevo TQ coupled to a UPLC (Waters Corp., Milford, USA). Metabolites were separated on a reversed-phase column (2.1 mm × 50 mm, BEH C18, Waters Corp., Milford, USA) using a mobile phase, which was composed of a gradient mixture of solvent A (formic acid 0.2% in water) and solvent B (formic acid 0.2% in acetonitrile) (0 min 0% B, 3.5 min 60% B, 3.8 min 0% B, and 3.9 min 0% B). Elution was performed at a flow rate of 900 μL/min. The column temperature was maintained at 50°C. For FIA, an isocratic method was used (100% organic running solvent) with varying flow conditions (0 min, 30 μL/min; 1.6 min, 30 μL/min; 2.4 min, 200 μL/min; 2.8 min, 200 μL/min; and 3 min 30 μL/min). The corresponding MS settings were as follows: dwell time of 0.019–0.025 sec, capillary voltage at 3.92 KV for positive mode, capillary voltage at 1.5 kV for negative mode, nitrogen as collision gas medium, and source temperature at 150°C.

2.5. Ultra-Performance Liquid Chromatography- (UPLC-) Based Measurement

The plasma samples (100 μL) were precipitated by 10% sulfosalicylic acid. After protein precipitation and centrifugation, derivatization was initiated by AQC in acetonitrile. Then, amino acids were analyzed using the ACQUITY UPLC System consisted of a Binary Solvent Manager, a Sample Manager, and a Tunable UV detector. The system was controlled and data collected using Empower™ 2 Software. Separations were performed on a 2.1 × 100 mm ACQUITY BEH C18 column at a flow rate of 0.70 mL/min. The average intra-assay coefficient of variation was 4.3% for histidine, 4.6% for ornithine, and 4.6% for phenylalanine. A total coefficient of variation was 3.1% for histidine, 3.6% for ornithine, and 3.7% for phenylalanine. The detection limit was 0.5 μM for histidine, 2.0 μM for ornithine, and 3.3 μM for phenylalanine. The linear range was 25–500 μM for these four amino acids. Spermidine level was low with high coefficient of variation. Lipid metabolites were not measurable by UPLC.

2.6. Measuring Skeletal Muscle Mass

Segmental multifrequency bioelectrical impedance analysis (SMBIA) was used to measure skeletal muscle mass. Eight-polar bioimpedance analyses of each participant were obtained using an InBody 720 multifrequency analyzer (at 1, 5, 50, 250, 500, and 1000 kHz) (Model Biospace InBody 720, Seoul, Korea). The validity of SMBIA has been documented in previous studies [911], showing no statistical difference between magnetic resonance imaging-measured and SMBIA-derived values for skeletal muscle mass. Skeletal muscle mass was measured for patients at stage C on the day we collected blood samples and 2 weeks later. The difference between the two measurements was correlated to the metabolic data. Skeletal muscle growth and loss were defined as skeletal muscle mass at two weeks minus skeletal muscle mass at baseline, with >0 kg signaling growth and <0 kg signaling loss.

2.7. Six-Minute Walking Distance

To estimate the patients’ functional condition, we measured six-minute walking distance (6MWD) for patients at stages ranging from B to D on the day metabolites were analyzed. Exclusion criteria were (1) extremity problems (polio, arthralgia, and amputation, ), (2) deafness (), (3) uncontrolled chronic pulmonary diseases (), (4) age > 85 years old (), (5) severe obesity (), (6) post-surgery with wound pain (), (7) uncontrolled psychiatric problems (), (8) uncorrected congenital heart disease (), (9) not able to walk (), and (10) stroke with hemiparesis (). Finally, 6MWD was performed in 453 patients.

2.8. Statistical Analysis

Results are expressed as for continuous variables and as the number (percentage) for categorical variables. Data were compared by two-sample -tests, ANOVA (subgroup analysis was conducted by Bonferroni), and chi-square (multiple comparison with Bonferroni adjusted value), when appropriate. The linear trend of the distribution of demographic and laboratory data across study groups was tested using Cochran-Armitage chi-square analysis for categorical variables and linear contrast in the general linear model for continuous variables. For accuracy measurement, a multinomial logistic regression model was used. The classification accuracy was calculated by comparing the predicted probability of HOP score and log (BNP) based on the logistic model to the dependent variable defined by different HF stages. For subgroup analysis, the cutoffs of BMI and eGFR were set at 24 kg/m2 (mild obesity) and 60 mL/min/1.73 m2 (≥stage 3 chronic kidney disease), respectively. Spearman’s correlation analysis was used to assess the correlation between variables. The integrated MetIDQ software (Biocrates, Innsbruck, Austria) was applied to streamline data analysis by automated calculation of metabolite concentrations. To maximize identification of differences in metabolic profiles between groups, the OPLS-DA model was applied and performed using the SIMCA-P software (version 13.0, Umetrics AB, Umea, Sweden). The variable importance in the projection (VIP) value of each variable in the model was calculated to indicate its contribution to the classification. A higher VIP value represents a stronger contribution to discrimination between groups. The VIP values of those variables greater than 1.0 are considered significantly different. For diagnosis, stepwise logistic regression analysis was used. Variables with and value of <0.05 in the univariate analysis were selected for the multivariable analysis. Odds ratios (ORs) and 95% confidence intervals (CIs) were calculated. The HOP score was calculated by a combination of histidine, ornithine, and phenylalanine according to the formula , with denoting the standardized value for the metabolites and denoting the regression coefficient from the regression model containing the indicated metabolites. Receiver operating characteristic (ROC) curves and area under curves (AUCs) were also estimated, and the cutoff value of variables was identified based on Youden’s index. Follow-up data were collected as scheduled or at the last available visit. Cox proportional hazards models were used to determine independent predictors of the first defined events (death, or HF-related rehospitalization). Variables with value <0.05 in the univariate analysis were selected for the multivariable analysis. Hazard ratios (HRs) and 95% CIs were also calculated. All statistical analyses were 2-sided and performed using SPSS software (version 22.0, SPSS, Chicago, IL, USA). A value of <0.05 was considered significant.

3. Results

3.1. Baseline Characteristics

A total of 841 participants, including 129 normal controls and 712 patients at HF stages A (), B (), C (), and D (), were enrolled in the initiation cohort. Baseline characteristics and laboratory data are shown in Table 1. From stage A to stage D, patients had a trend to be older and have lower LVEF, blood pressure, cholesterol, triglyceride, hemoglobin, albumin, and eGFR, but have higher heart rate, BNP levels, and incidence of diabetes mellitus, chronic kidney disease, and atrial fibrillation. The main etiology of HF in the study cohort was coronary artery disease.

NormalStage AStage BStage CStage D
for trend

Age (years)53.7 ± 8.058.6 ± 11.460.9 ± 12.360.7 ± 14.261.5 ± 13.0<0.001
Male (%)46 (35.7)64 (68.8)83 (69.2)263 (66.1)66 (65.3)0.465
LVEF (%)71.1 ± 11.964.8 ± 15.946.2 ± 15.433.1 ± 11.328.7 ± 10.4<0.001
Blood pressure (mm Hg)
 Systolic123.2 ± 15.5127.4 ± 17.7124.9 ± 20.8123.4 ± 18.5117.2 ± 17.4<0.001
 Diastolic74.3 ± 11.477.6 ± 11.076.1 ± 14.174.5 ± 12.573.1 ± 13.10.008
Heart rate, beats/min73.1 ± 11.575.6 ± 13.574.6 ± 14.177.3 ± 11.879.4 ± 13.10.016
 Diabetes mellitus (%)0 (0)30 (32.3)45 (37.5)156 (39.2)46 (45.5)0.046
 Chronic kidney disease (%)0 (0)16 (17.2)20 (16.7)97 (24.4)29 (28.7)0.015
 Hypertension (%)0 (0)51 (54.8)87 (72.5)255 (61.1)68 (67.3)0.287
 Atrial fibrillation (%)0 (0)3 (3.2)31 (25.8)135 (33.9)34 (33.7)<0.001
 COPD (%)0 (0)6 (6.5)12 (10.0)45 (11.3)11 (10.9)0.239
Ischemia (%)0 (0)45 (48.4)80 (66.7)175 (44.0)48 (47.5)0.059
Body mass index (kg/m2)24.1 ± 3.925.1 ± 4.025.8 ± 4.425.2 ± 5.325.9 ± 7.90.368
 ACEI or ARB (%)0 (0)28 (30.1)94 (78.3)358 (89.9)92 (91.1)<0.001
β-Blocker (%)0 (0)31 (33.3)90 (75.0)301 (75.6)67 (66.3)<0.001
 Diuretic (%)0 (0)11 (11.81)45 (37.5)236 (59.3)71 (70.3)<0.001
Laboratory data
 BNP (log)0.94 ± 0.291.39 ± 0.722.13 ± 0.652.58 ± 0.542.92 ± 0.41<0.001
 Cholesterol (mg/dL)211.1 ± 34.8192.7 ± 53.2189.4 ± 53.0177.2 ± 45.8157.3 ± 36.0<0.001
 Triglyceride (mg/dL)102.2 ± 58.2153.7 ± 102.4142.1 ± 96.8126.8 ± 80.2114.5 ± 93.10.001
 Serum sodium (mEq/L)140.2 ± 1.1138.6 ± 4.2139.2 ± 2.4139.1 ± 3.3138.3 ± 3.30.468
 Hemoglobin (g/dL)13.87 ± 1.4013.98 ± 1.6813.55 ± 1.9713.35 ± 2.1212.95 ± 2.06<0.001
 Albumin (g/dL)4.29 ± 0.724.05 ± 0.593.80 ± 0.493.61 ± 0.473.32 ± 0.51<0.001
 ALT (U/L)26.6 ± 14.029.0 ± 20.224.2 ± 12.924.9 ± 15.728.6 ± 22.10.951
γGT (U/L)26.5 ± 19.642.4 ± 56.747.5 ± 48.268.5 ± 137.1143.3 ± 117.10.001
 eGFR (mL/min/1.73 m2)99.8 ± 18.475.7 ± 24.079.1 ± 29.572.1 ± 28.461.9 ± 21.5<0.001
 QRS complex, msec90.3 ± 10.493.7 ± 18.196.9 ± 19.1103.3 ± 25.9110.7 ± 25.6<0.001
 HOP score−4.50 ± 4.881.82 ± 4.424.10 ± 5.1110.77 ± 4.1318.00 ± 5.02<0.001
  Histidine (μM)95.2 ± 17.288.8 ± 14.383.2 ± 16.974.1 ± 14.771.6 ± 20.5<0.001
  Ornithine (μM)59.7 ± 19.568.7 ± 27.977.1 ± 30.5101.4 ± 36.5114.8 ± 49.5<0.001
  Phenylalanine (μM)57.0 ± 8.562.3 ± 10.566.6 ± 16.775.6 ± 18.889.0 ± 30.6<0.001

Comparison among patients from stages A to D. ACC/AHA: American College of Cardiology/American Heart Association; ACEI: angiotensin-converting enzyme inhibitor; ALT: alanine aminotransferase; ARB: angiotensin receptor blocker; COPD: chronic obstructive pulmonary disease; chronic kidney disease, estimated glomerular filtration rate ; HF: heart failure; HOP: histidine, ornithine, and phenylalanine; LVEF: left ventricular ejection fraction; γGT: γ-glutamyltransferase.
3.2. Development of Amino Acid-Based Metabolic Panel

Comparisons between normal controls and stage C patients in all essential and nonessential amino acids measure by mass spectrometry are shown in Table 2. To identify a potential amino acid panel for diagnosing HF, we performed multivariable analyses only for the amino acids with a VIP . Histidine, ornithine, and phenylalanine were three strongly and independently significant amino acids for the integrated metabolic panel and were finally adopted to develop the HOP panel based on the logistic regression model. For unified quantification, all HOP measurements in subsequent studies were performed by UPLC. Histidine, ornithine, and phenylalanine changed in a significant trend from normal patients to stages A through D (Table 1). The diagnostic value of the HOP and log (BNP) for discriminating stage C patients from normal controls are shown by the ROC curves and AUCs (Figure 1(a)) and by univariate analysis in Table 3. The HOP gave rise to an AUC of 0.99, similar to BNP (). In multivariable analysis, the diagnostic value of the HOP and BNP was independent of age, sex, left ventricular ejection fraction, diabetes mellitus, kidney disease, and hypertension (Table 3). In stage C patients, HOP scores were not significantly different when comparing those taking angiotensin-converting enzyme inhibitors/angiotensin receptor blockers and those who did not (10.7 ± 4.13 vs. 11.3 ± 4.14, respectively, ). The difference between patients taking β-blockers and those who did not also did not reach the level of significance (10.6 ± 4.01 vs. 11.3 ± 4.46, respectively, ).

Normal controlsHeart failure stage CUnivariateMultivariable
Amino acid (μM)VIP score()()Odds ratio (95% CI)Odds ratio (95% CI) value

Histidine2.2095.2 ± 17.274.1 ± 14.70.92 (0.90–0.94)0.71 (0.64–0.79)<0.001
Ornithine2.0959.7 ± 19.5101.4 ± 36.51.06 (1.05–1.07)1.07 (1.04–1.11)<0.001
Phenylalanine1.8957.0 ± 8.575.6 ± 18.81.13 (1.10–1.16)1.48 (1.31–1.68)<0.001
Aspartate1.284.2 ± 2.66.7 ± 8.61.32 (1.23–1.42)
Glutamate1.2468.3 ± 68.1137.7 ± 110.51.01 (1.009–1.02)
Glutamine1.21679.6 ± 142.5569.0 ± 173.80.995 (0.994–0.997)
Tyrosine0.9165.4 ± 13.176.2 ± 24.61.03 (1.02–1.04)
Isoleucine0.8072.1 ± 18.681.5 ± 23.01.02 (1.01–1.03)
Citrulline0.7847.3 ± 27.936.3 ± 25.70.99 (0.979–0.993)
Tryptophan0.6342.9 ± 12.447.6 ± 14.51.03 (1.01–1.04)
Lysine0.62216.9 ± 41.0234.3 ± 58.11.06 (1.05–1.07)
Serine0.47126.2 ± 31.1118.2 ± 31.60.99 (0.98–0.99)
Arginine0.4162.5 ± 21.257.4 ± 24.40.99 (0.97–0.99)
Methionine0.3325.3 ± 6.123.9 ± 8.5
Proline0.33170.9 ± 66.4182.5 ± 66.9
Asparagine0.2549.1 ± 27.945.9 ± 21.4
Valine0.19243.9 ± 54.5238.1 ± 58.8
Leucine0.11146.9 ± 41.0144.5 ± 40.9
Alanine0.10347.7 ± 112.2340.8 ± 131.1
Threonine0.10116.3 ± 28.6114.6 ± 33.6
Glycine0.09241.4 ± 72.1245.3 ± 85.2

Data are presented as . VIP: variable importance in the projection. and , compared to normal controls. Multivariable analysis was performed for the amino acids with VIP by stepwise logistic regression model (forward selection).

Odds ratio (95% CI) valueOdds ratio (95% CI) value

HOP score1.53 (1.42~1.66)<0.0011.64 (1.45~1.86)<0.001
Log (BNP) (×10−2)1.05 (1.04~1.06)<0.0011.06 (1.04~1.08)<0.001

The range of HOP score, value calculated by the combination of histidine, ornithine, and phenylalanine, is from −17.7 to 35.2 (on the basis of logistic regression model). Multivariable analysis adjusting for age, sex, left ventricular ejection fraction, diabetes mellitus, estimated glomerular filtration rate, and hypertension.
3.3. HOP and BNP in Discriminating HF Stages

HOP and log (BNP) significantly separated patients at different HF stages (Figures 1(b) and 1(c)). The accuracy of HOP in discrimination was better than that of log (BNP) (66.2% and 59.4%, respectively). In patients at stages B, C, and D, the HOP discriminated stages B and C better than log (BNP) in elderly patients aged > 70 years (79.9% and 66.8% accuracy, respectively) and those with atrial fibrillation (78.2% and 69.8% accuracy, respectively) (Figure 1(d)). The HOP also demonstrated better discrimination between stages C and D than log (BNP) in patients with a body mass (73.7% and 62.3% accuracy, respectively) and eGFR <60 mL/min/1.73 m2 (79.9% and 65.0% accuracy, respectively).

3.4. Correlating the HOP Panel to Clinical Variables

HOP showed modest correlation with BNP levels (Supplementary Figure IIA). Ornithine levels were weakly associated with the levels of rGT (, ), but not with ALT or AST. Although both rGT and ornithine levels changed in a significant trend from normal patients to stages A through D, the areas under ROC curves in discriminating stage C patients from normal controls were 0.71 and 0.85, respectively. Skeletal muscle growth and loss were significantly related to baseline phenylalanine levels (Supplementary Figure IIB) and weakly related to BNP (, ). In patients who lost skeletal muscle, skeletal muscle mass changed from 29.9 ± 6.11 kg at baseline to 28.6 ± 5.69 kg () two weeks later. Patients who experienced skeletal muscle growth progressed from 28.1 ± 4.17 kg to 28.9 ± 4.13 kg two weeks later (). Baseline phenylalanine levels were significantly higher in patients who had skeletal muscle loss compared to those who had skeletal muscle growth (86.5 ± 22.8 μM vs. 68.6 ± 12.5 μM, respectively, ). To estimate the correlation between changes in phenylalanine and changes in skeletal muscle mass, we had 101 patients with blood samples also collected two weeks after baseline. To discriminate patients who experienced muscle growth from those who experienced muscle loss, we used ROC curves to determine the optimal baseline phenylalanine cutoff value (69.5 μM), with high phenylalanine levels defined as ≥69.5 μM and low phenylalanine levels as <69.5 μM. Using baseline phenylalanine and phenylalanine measurements taken two weeks later, we classified patients according to the following patterns: “high-to-high,” “high-to-low,” “low-to-low,” and “low-to-high.” The skeletal muscle loss was significantly greater in high-to-high patients, compared to the others (Supplementary Table I). For functional assessment, 6MWD demonstrated a significant correlation with HOP that was stronger than with BNP (Supplementary Figures IIC and IID), but not with albumin. Higher HOP was associated with shorter 6MWD.

3.5. Prognostic Value of HOP

The prognostic value of HOP was further tested in patients with potential risk of events. During a follow-up period of three years, we observed 172 (34.5%) composite events (death/HF-related rehospitalization) in patients at stages C and D (). In univariate analysis, associates of composite events included HOP, log (BNP), age, sex, left ventricular ejection fraction, functional class, HF stage, diabetes mellitus, chronic kidney disease, hypertension, hemoglobin, and albumin (Table 4). In multivariable analysis, HOP remained significant after adjusting for log (BNP), age, sex, left ventricular ejection fraction, functional class, HF stage, diabetes mellitus, chronic kidney disease, hypertension, hemoglobin, and albumin. However, log (BNP) became insignificant (Table 4).

ItemHazard ratio (95% CI) valueHazard ratio (95% CI) value

HOP score1.072 (1.044~1.100)<0.0011.057 (1.024~1.091)0.001
Log (BNP) × 10−11.078 (1.045~1.112)<0.0011.028 (0.995~1.063)0.102
Age (years)1.033 (1.020~1.046)<0.0011.029 (1.015~1.043)<0.001
Sex0.651 (0.478~0.886)0.0060.747 (0.519~1.075)0.116
LVEF (%)0.990 (0.980 ~1.000)0.050.989 (0.977~1.001)0.072
NYHA functional class ≥III4.196 (1.720~10.233)0.0021.455 (0.565~3.748)0.437
Stage D1.839 (1.416~2.388)<0.0011.417 (1.021~1.088)0.024
Systolic blood pressure (mmHg)0.996 (0.988~1.005)0.402
Diabetes mellitus1.685 (1.241~2.288)0.0011.360 (0.972~1.902)0.073
Chronic kidney disease2.058 (1.503~2.818)<0.0011.516 (1.078~2.132)0.017
Hypertension1.458 (1.035~2.054)0.0311.039 (0.712~1.516)0.843
Atrial fibrillation1.274 (0.926~1.751)0.136
Body mass index (kg/m2)0.971 (0.935~1.008)0.119
Serum sodium (mEq/L)0.976 (0.936~1.018)0.189
Hemoglobin (g/dL)0.873 (0.814~0.935)<0.0010.970 (0.887~1.059)0.495
Albumin (g/dL)0.531 (0.409~0.690)<0.0010.755 (0.546~1.044)0.089

BNP: B-type natriuretic peptide; chronic kidney disease, estimated glomerular filtration rate ; HOP: histidine, ornithine and phenylalanine; LVEF: left ventricular ejection fraction; NYHA: New York Heart Association.

The cutoff values of BNP and HOP were set at 750 pg/mL and 14 based on ROC curves. Kaplan-Meier curves show that patients with had lower 3-year event-free survival compared to those with (Figure 2(a)). Patients with had lower event-free survival compared to those with (Figure 2(b)). Based on the cutoffs, patients were separated into 4 subgroups, including high BNP and HOP (HBHH), high BNP and low HOP (HBLH), low BNP and high HOP (LBHH), and low BNP and low HOP (LBLH). Kaplan-Meier survival curves were further analyzed according to the 4 subgroups (Figure 2(c)). Patients with HBHH had significantly lower 3-year event-free survival than all others (, 95% , ). The prognostic value of the HOP panel was further validated in 447 stage C patients. During the one-year follow-up period, 85 (19%) composite events were noted. Patients with (Figure 2(d)) and those with (Figure 2(e)) were associated with a higher event rate over one year from enrollment. Those with HBHH also had the lowest one-year event-free survival rate than all others (Figure 2(f)). The demographic data for the validation cohort are shown in Supplementary Table II.

3.6. Differences between Subgroups Defined by BNP and HOP

Table 5 shows the differences in demography and laboratory data between four subgroups. Compared to patients with LBLH, those with HBHH had higher incidence of ischemic etiology, and higher levels of total bilirubin, but lower left ventricular ejection fraction, lower high-density lipoprotein cholesterol levels, serum sodium, hemoglobin, albumin, and eGFR. Patients with LBHH, compared to those with LBLH, had higher levels of total bilirubin, but lower levels of total cholesterol and high and low density of lipoprotein-cholesterol and albumin. Patients with HBHH, compared to those with HBLH, had lower levels of sodium and albumin. Compared to patients with LBHH, patients with HBHH had higher incidence of ischemic etiology and higher levels of low-density lipoprotein cholesterol, but they also had lower incidence of atrial fibrillation as well as lower left ventricular ejection fraction, body mass index, hemoglobin, albumin, and eGFR.


Age (years)59.8 ± 14.563.4 ± 13.758.5 ± 13.963.1 ± 12.20.26
Male (%)151 (66.8)56 (56.6)66 (75.9)56 (64.4)0.049
LVEF (%)36.9 ± 15.230.2 ± 12.936.5 ± 16.428.5 ± 11.9†ϕ<0.001
NYHA classification ≥III216 (95.6)99 (100)86 (98.9)85 (97.7)0.06
Blood pressure (mm Hg)
 Systolic123.3 ± 16.9121.6 ± 19.7120.2 ± 19.7122.1 ± 19.30.58
 Diastolic74.3 ± 13.173.4 ± 12.374.3 ± 12.274.8 ± 12.50.89
 Heart rate, beats/min77.0 ± 11.578.1 ± 13.577.9 ± 11.978.9 ± 11.90.63
 Diabetes mellitus (%)98 (43.4)40 (40.4)32 (36.8)32 (36.8)0.62
 Hypertension (%)145 (64.2)69 (69.7)44 (50.6)59 (67.8)0.034
 Atrial fibrillation (%)80 (35.4)27 (27.3)41 (47.1)21 (24.1)ϕ0.005
 COPD (%)27 (11.9)8 (8.1)7 (8.0)14 (16.1)0.26
 Ischemic (%)95 (42.0)51 (51.5)25 (28.7)52 (59.8)ϕ<0.001
Body mass index (kg/m2)26.2 ± 5.523.2 ± 3.727.9 ± 9.724.1 ± 3.9<0.001
Laboratory data
 Cholesterol (mg/dL)178.6 ± 45.8169.5 ± 48.3160.3 ± 36.7175.3 ± 42.40.009
 Triglyceride (mg/dL)122.9 ± 68.1115.5 ± 66.1136.6.9 ± 129.7124.9 ± 76.40.38
 LDL-C (mg/dL)113.6 ± 40.9106.2 ± 41.7101.0 ± 31.8117.0 ± 31.8§0.016
 HDL-C (mg/dL)41.4 ± 22.640.0 ± 15.135.2 ± 11.333.3 ± 11.50.001
 Serum sodium (mEq/L)139.4 ± 3.2139.2 ± 2.8138.5 ± 3.6137.7 ± 3.8†‡<0.001
 Hemoglobin (g/dL)13.4 ± 2.013.0 ± 2.113.7 ± 2.312.6 ± 2.2ϕ0.002
 Total bilirubin (mg/dL)1.0 ± 0.61.2 ± 0.91.3 ± 0.81.5 ± 1.0<0.001
 Albumin (g/dL)3.7 ± 0.53.5 ± 0.43.5 ± 0.53.2 ± 05†#ϕ<0.001
 eGFR (mL/min/1.73 m2)77.4 ± 29.263.7 ± 25.269.5 ± 21.458.6 ± 24.9†§<0.001

HB and LB indicate high and low BNP (≥750 and <750 pg/mL, respectively). HH and LH indicate high and low HOP score (≥14 and <14, respectively). COPD: chronic obstructive pulmonary disease; eGFR: estimated glomerular filtration rate; HDL-C: high-density lipoprotein-cholesterol; LDL-C: low density lipoprotein-cholesterol; LVEF: left ventricular ejection fraction; NYHA: New York Heart Association. and , compared to LBLH; and #, compared to HBLH; § and ϕ, compared to LBHH.

4. Discussion

The UPLC-based amino acid panel provided a metabolism spectrum for patients with HF. In identifying stage C patients, HOP was not inferior to BNP. However, HOP discriminates HF stages better than BNP especially in the elderly and those with atrial fibrillation, high body mass index, or kidney dysfunction. HOP was significantly associated with losses of both skeletal muscle mass and functional capacity. HOP added diagnostic and prognostic value to currently used BNP and a variety of traditional risk factors. Combining HOP and BNP synergistically and robustly identified patients at high risk of events.

4.1. Issues Relating to Current Biomarkers

BNP is a powerful diagnostic biomarker for HF, with ST2 and galectin 3 providing additional prognostic information [6, 12, 13]. However, all these biomarkers offer limited clues for aspects other than wall stress and activation of global fibrosis. The clinical benefit of natriuretic peptide-guided therapy has not been uniform across the trials, with some populations not shown benefit [7]. Although to define more precise and clinically attainable natriuretic peptide targets is suggested, this is complicated in the presence of the common confounders such as atrial fibrillation, ischemia, aging, and renal dysfunction [7]. Our study showed that HOP added diagnostic value to BNP. HOP offers a quantifiable readout of the biochemical state of metabolism in HF-related end organs that is often not obvious from BNP analysis. It seems that BNP represents the wall stress of the heart, which is central, while HOP represents the global metabolism status of peripheral tissues.

The relationship between BNP and HOP is only modest. Notably, the correlation between HOP score and BNP in the zone of or > 5000 pg/mL was better than that in the zone of BNP from 500 to 5000 pg/mL (Supplementary Figure IIA). Level of stress on cardiac chambers is apparently different from the peripheral metabolism status associated with the imbalance between demand and supply. Patients with high BNP were not always at high risk of events. High BNP with low HOP represented increased ventricular wall stress, but without significant impact of perturbed HF-related hemodynamics on peripheral tissues. Patients with both high BNP and high HOP were at very high risk of both central and peripheral disturbances. Compared to other subgroups, these patients tended to have lower ejection fractions, as well as lower levels of albumin, hemoglobin, sodium, high-density lipoprotein cholesterol, and eGFR. They also had higher incidence of ischemic etiology and higher total bilirubin levels. This subgroup had an event-free survival rate of only 51% one year after discharge. On the other hand, although patients with high HOP but low BNP had event risk similar to those with low BNP and low HOP, this subgroup represented a population with metabolism dysfunction and malnutrition, indicated by lower lipid profile and albumin levels, and demanded interventions designed specifically.

4.2. HOP Provides Insight into Metabolism

HOP is developed from statistical integration of molecular-level metabolites with no prior comprehension of the exact relevance of each component. Nitric oxide synthesis dysregulation and tetrahydrobiopterin depletion were probably related to impaired conversion of phenylalanine to tyrosine and the accumulation of phenylalanine [14, 15]. On the other hand, the increased phenylalanine levels may also be associated with increased muscular protein breakdown [16] and impaired liver function. Although our data revealed that high phenylalanine levels at baseline or persistently high phenylalanine levels are modestly correlated with skeletal muscle mass loss, the formal evaluation of muscle protein dynamics involves sophisticated calculation based on isotope infusion designs [17, 18] and is impractical for HF patients. The kinetics of circulating phenylalanine waits for further investigation and clarification in the future.

A large amount of histidine pooled in hemoglobin and carnosine in the muscle, delaying indications of histidine deficiency [19]. Consequently, by the time lower blood histidine concentration was noted, the histidine deficiency was probably advanced. For the decreased concentration of histidine, there are a few speculative mechanisms. First, in normal physiology, the protein turnover rate is much higher than that of carbohydrates and fat [18]. However, a majority of the broken proteins are efficiently resynthesized to form new proteins. Once histidine is used to synthesize muscle, it turns into another form, namely, methylhistidine [17, 18]. Based on the high turnover rate in muscle of HF patients, a huge amount of histidine is consumed without adequate refilling of the histidine pool. Second, histidine can be converted to glutamate and enter the glutamate-ornithine-proline pathway or Krebs cycle to produce energy for cardiac tissues. Finally, glucose is able to be converted to phosphoribosyl pyrophosphate for histidine biosynthesis. Overuse of glucose to produce energy in HF impairs the production of histidine. In sum, there is a strong possibility of unbalanced histidine supply and metabolism. However, again, identifying the real mechanisms requires further investigation. At the present stage for clinical application, the changes in histidine and phenylalanine levels may serve as feedback indicators for evaluating patients’ response to medications, rehabilitation, and nutritional interventions.

Some of the amino acids included in our study are not metabolized within skeletal muscle, such as ornithine and citrulline. The correlation between ornithine and traditional liver biochemistry is weak. So far, clinically available biochemical tests for liver function are neither functional nor specific. Elevated AST and ALT represent hepatocyte injury or necrosis, but are likewise insensitive for assessing HF-related liver dysfunction. rGT relates to liver fibrosis, but is also increased in biliary diseases and alcoholism. Ornithine is an important component of the urea cycle, which functions entirely in the liver. Ornithine level also represents the imbalance between the ornithine load derived from metabolism of amino acids and the function of urea cycle to clean ornithine. A significant portion of patients with normal rGT had increased ornithine levels, probably indicating acute liver congestion without established liver fibrosis. Increased rGT with normal ornithine may suggest a biliary or alcoholic etiology and may not necessarily be associated with HF. Although a significant trend of increase in ornithine levels from normal patients to stages A through D was noted, the relationship between ornithine and liver function indeed waits for future experiments and clinical studies. On the other hand, decreased citrulline levels have been observed in stage C patients compared to normal controls. This may be caused by impaired mitochondrial function which limits metabolism of ornithine to form citrulline in the urea cycle, or it may be caused by impaired nitric oxide synthase function [20].

4.3. Study Limitations

The findings in this study are limited to HF patients with . A 2-week time period to estimate the correlation between phenylalanine levels and changes in skeletal muscle mass was short. Although a longer observation time period would be more convincing, the design was substantially limited by multiple factors with potential interference, such as exercise. The effects of renin-angiotensin-aldosterone system and β-blockade on tissue metabolism have been investigated and remain controversial [2123]. Although our study did not produce statistically significant findings, our results could be confounded by the presence of preexisting contraindications to using these medications. Future studies could elaborate on these findings using a randomized design or by measuring metabolites before and after receiving medication. Moreover, this study is substantially limited by the unclear relationship between amino acids and their clinical value. Majority of the findings were based on statistical observations. Future works on the mechanism exploration by cell and animal models, and clinical studies are mandatory.

5. Conclusion

Aiming for clinical applications, this study developed a three amino acid-based metabolic panel for assessing HF patients. The metabolic insight of HOP compensates for the value of BNP. HOP adds diagnostic and prognostic value to currently used BNP measurement and other traditional risk factors. Whether tailored multidisciplinary intervention guided by the HOP and natriuretic peptides can modify disease outcomes warrants future interventional studies.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare no conflict of interest.


This work was supported by the Ministry of Science and Technology of Taiwan (MOST105-2314-B-182-046-MY2 and 107-2314-B-182-071-MY2), Chang Gung Memorial Hospital (CMRPG2C0313, G2E0351, G2G0601, and G2G0581), and the Ministry of Education of Taiwan (EMRPD1G0251 and EMRPD1H0401). The authors thank Cardiology Section, Department of Internal Medicine, Chang Gung Memorial Hospital, Keelung, Taiwan, for providing samples from patients and normal controls. We also thank Healthy Aging Research Center, Chang Gung University, from the Featured Areas Research Center Program within the Framework of the Higher Education Sprout Project by the Ministry of Education in Taiwan.

Supplementary Materials

Supplementary Figure I: study flow diagram. HOP: a metabolic panel composed of histidine, ornithine, and phenylalanine. Supplementary Figure II: the correlation of HOP and B-natriuretic peptide (BNP) to clinical parameters. (A) The relationship between HOP and BNP; (B) a higher blood phenylalanine level correlates with more skeletal muscle loss (∆ skeletal muscle indicates the changes in 14 days after measurement); (C, D) the relationship of HOP and BNP to 6 min walk distance (6MWD). HOP: a metabolic panel composed of histidine, ornithine, and phenylalanine. Supplementary Table I: changes in skeletal muscle mass from baseline to 2 weeks later in patients with different patterns of changes in phenylalanine levels from baseline to 2 weeks later. Supplementary Table II: demographic and laboratory data for the heart failure patients in the validation cohort. (Supplementary Materials)


  1. R. D. Beger, W. Dunn, M. A. Schmidt et al., “Metabolomics enables precision medicine: “A White Paper, Community Perspective”,” Metabolomics, vol. 12, no. 9, p. 149, 2016. View at: Publisher Site | Google Scholar
  2. W. G. Hunter, J. P. Kelly, R. W. McGarrah 3rd, W. E. Kraus, and S. H. Shah, “Metabolic dysfunction in heart failure: diagnostic, prognostic, and pathophysiologic insights from metabolomic profiling,” Current Heart Failure Reports, vol. 13, no. 3, pp. 119–131, 2016. View at: Publisher Site | Google Scholar
  3. D. Alexander, R. Lombardi, G. Rodriguez, M. M. Mitchell, and A. J. Marian, “Metabolomic distinction and insights into the pathogenesis of human primary dilated cardiomyopathy,” European Journal of Clinical Investigation, vol. 41, no. 5, pp. 527–538, 2011. View at: Publisher Site | Google Scholar
  4. L. Tenori, X. Hu, P. Pantaleo et al., “Metabolomic fingerprint of heart failure in humans: a nuclear magnetic resonance spectroscopy analysis,” International Journal of Cardiology, vol. 168, no. 4, pp. e113–e115, 2013. View at: Publisher Site | Google Scholar
  5. P. Wurtz, A. S. Havulinna, P. Soininen et al., “Metabolite profiling and cardiovascular event risk: a prospective study of 3 population-based cohorts,” Circulation, vol. 131, no. 9, pp. 774–785, 2015. View at: Publisher Site | Google Scholar
  6. A. S. Maisel, P. Clopton, P. Krishnaswamy et al., “Impact of age, race, and sex on the ability of B-type natriuretic peptide to aid in the emergency diagnosis of heart failure: results from the Breathing Not Properly (BNP) multinational study,” American Heart Journal, vol. 147, no. 6, pp. 1078–1084, 2004. View at: Publisher Site | Google Scholar
  7. O. Chioncel, S. P. Collins, S. J. Greene et al., “Natriuretic peptide-guided management in heart failure,” Journal of Cardiovascular Medicine, vol. 17, no. 8, pp. 556–568, 2016. View at: Publisher Site | Google Scholar
  8. S. A. Hunt, W. T. Abraham, M. H. Chin et al., “ACC/AHA 2005 Guideline Update for the Diagnosis and Management of Chronic Heart Failure in the Adult: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Update the 2001 Guidelines for the Evaluation and Management of Heart Failure): developed in collaboration with the American College of Chest Physicians and the International Society for Heart and Lung Transplantation: endorsed by the Heart Rhythm Society,” Circulation, vol. 112, no. 12, pp. e154–e235, 2005. View at: Publisher Site | Google Scholar
  9. H. Moreira, B. Passos, J. Rocha, V. Reis, A. Carneiro, and R. Gabriel, “Cardiorespiratory fitness and body composition in postmenopausal women,” Journal of Human Kinetics, vol. 43, no. 1, pp. 139–148, 2014. View at: Publisher Site | Google Scholar
  10. H. Ogawa, K. Fujitani, T. Tsujinaka et al., “Inbody 720 as a new method of evaluating visceral obesity,” Hepato-Gastroenterology, vol. 58, no. 105, pp. 42–44, 2011. View at: Google Scholar
  11. G. Medici, C. Mussi, A. L. Fantuzzi, M. Malavolti, A. Albertazzi, and G. Bedogni, “Accuracy of eight-polar bioelectrical impedance analysis for the assessment of total and appendicular body composition in peritoneal dialysis patients,” European Journal of Clinical Nutrition, vol. 59, no. 8, pp. 932–937, 2005. View at: Publisher Site | Google Scholar
  12. C. H. Wang, N. I. Yang, M. H. Liu, K. H. Hsu, and L. T. Kuo, “Estimating systemic fibrosis by combining galectin-3 and ST2 provides powerful risk stratification value for patients after acute decompensated heart failure,” Cardiology Journal, vol. 23, no. 5, pp. 563–572, 2016. View at: Publisher Site | Google Scholar
  13. S. U. Rehman, T. Mueller, and J. L. Januzzi Jr., “Characteristics of the novel interleukin family biomarker ST2 in patients with acute heart failure,” Journal of the American College of Cardiology, vol. 52, no. 18, pp. 1458–1465, 2008. View at: Publisher Site | Google Scholar
  14. Y. Nishijima, A. Sridhar, I. Bonilla et al., “Tetrahydrobiopterin depletion and NOS2 uncoupling contribute to heart failure-induced alterations in atrial electrophysiology,” Cardiovascular Research, vol. 91, no. 1, pp. 71–79, 2011. View at: Publisher Site | Google Scholar
  15. M. T. Ziolo, L. S. Maier, V. Piacentino, J. Bossuyt, S. R. Houser, and D. M. Bers, “Myocyte nitric oxide synthase 2 contributes to blunted β-adrenergic response in failing human hearts by decreasing Ca2+ transients,” Circulation, vol. 109, no. 15, pp. 1886–1891, 2004. View at: Publisher Site | Google Scholar
  16. R. Aquilani, M. T. la Rovere, O. Febo et al., “Preserved muscle protein metabolism in obese patients with chronic heart failure,” International Journal of Cardiology, vol. 160, no. 2, pp. 102–108, 2012. View at: Publisher Site | Google Scholar
  17. V. S. Nagabhushan and B. S. Narasinga Rao, “Studies on 3-methylhistidine metabolism in children with protein-energy malnutrition,” The American Journal of Clinical Nutrition, vol. 31, no. 8, pp. 1322–1327, 1978. View at: Publisher Site | Google Scholar
  18. Z. Liu and E. J. Barrett, “Human protein metabolism: its measurement and regulation,” American Journal of Physiology- Endocrinology and Metabolism, vol. 283, no. 6, pp. E1105–E1112, 2002. View at: Publisher Site | Google Scholar
  19. W. Kriengsinyos, M. Rafii, L. J. Wykes, R. O. Ball, and P. B. Pencharz, “Long-term effects of histidine depletion on whole-body protein metabolism in healthy adults,” The Journal of Nutrition, vol. 132, no. 11, pp. 3340–3348, 2002. View at: Publisher Site | Google Scholar
  20. R. R. Giraldez, A. Panda, Y. Xia, S. P. Sanders, and J. L. Zweier, “Decreased nitric-oxide synthase activity causes impaired endothelium-dependent relaxation in the postischemic heart,” The Journal of Biological Chemistry, vol. 272, no. 34, pp. 21420–21426, 1997. View at: Publisher Site | Google Scholar
  21. N. E. Boutagy, E. L. Marinik, R. P. McMillan et al., “Angiotensin II receptor blockade and skeletal muscle metabolism in overweight and obese adults with elevated blood pressure,” Therapeutic Advances in Cardiovascular Disease, vol. 9, no. 2, pp. 45–50, 2015. View at: Publisher Site | Google Scholar
  22. K. Kackstein, A. Teren, Y. Matsumoto et al., “Impact of angiotensin II on skeletal muscle metabolism and function in mice: contribution of IGF-1, Sirtuin-1 and PGC-1α,” Acta Histochemica, vol. 115, no. 4, pp. 363–370, 2013. View at: Publisher Site | Google Scholar
  23. L. Poirier and S. W. Tobe, “Contemporary use of β-blockers: clinical relevance of subclassification,” The Canadian Journal of Cardiology, vol. 30, no. 5, pp. S9–S15, 2014. View at: Publisher Site | Google Scholar

Copyright © 2018 Chao-Hung Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

More related articles

788 Views | 346 Downloads | 5 Citations
 PDF  Download Citation  Citation
 Download other formatsMore
 Order printed copiesOrder

Related articles

We are committed to sharing findings related to COVID-19 as quickly and safely as possible. Any author submitting a COVID-19 paper should notify us at help@hindawi.com to ensure their research is fast-tracked and made available on a preprint server as soon as possible. We will be providing unlimited waivers of publication charges for accepted articles related to COVID-19. Sign up here as a reviewer to help fast-track new submissions.