Oxidative Stress and Heart Failure: Mechanisms, Signalling Pathways, and TherapeuticsView this Special Issue
Serum Sulfhydryl Groups, Malondialdehyde, Uric Acid, and Bilirubin as Predictors of Adverse Outcome in Heart Failure Patients due to Ischemic or Nonischemic Cardiomyopathy
Oxidative stress plays a significant role in the pathogenesis of heart failure (HF). The aim of the study was to investigate the prognostic value of oxidation-reduction (redox) markers in patients with HF due to ischemic and nonischemic cardiomyopathy. The study included 707 patients of HF allocated into two groups depending on ethology: ischemic cardiomyopathy (ICM) () and nonischemic cardiomyopathy (nICM) (), who were followed up for one year. The endpoint occurrence (mortality or heart transplantation) in a 1-year follow-up was similar in the ICM and nICM group. The predictive value of endpoint occurrence of oxidative stress biomarkers such as the serum protein sulfhydryl groups (PSH), malondialdehyde (MDA), uric acid (UA), bilirubin, and MDA/PSH ratio and other clinical and laboratory data were assessed in both groups (ICM and nICM) separately using univariate and multivariate Cox regression analyses. In multivariate analysis, the higher concentrations of UA (, , 95% CI (1.005-1.044)) and MDA (, , 95% CI (1.296-3.741)) were significantly associated with adverse prognosis in patients with ICM. Contrastingly, in patients with nICM, we observed that higher bilirubin concentration (, , 95% CI (1.004-1.064)) and MDA/PSH ratio (, , 95% CI (1.096-10.302)) were significantly associated with increased risk of death or HT. The results showed the association of different oxidative biomarkers on the unfavorable course of heart failure depending on etiology.
The prognosis of patients with heart failure with reduced left ventricular ejection fraction is particularly poor. Dysregulated reduction-oxidation status, along with neurohormonal abnormalities and inflammation, is one of the common drivers of disease progression . Increased formation of reactive oxygen species, a by-product of reduction-oxidation reactions, can lead to lipid peroxidation, protein carboxylation, cytoskeletal disruption, and DNA damage. Many clinical and laboratory parameters reflecting disturbed pathophysiological pathways in heart failure are used to identify patients at higher risk of unfavorable prognosis. Due to the extremely short half-life of ROS, it is not feasible to assess it directly. The compromised choice may be the assessment of stable by-products, modified under oxidative conditions associated with elevated ROS formation, which have been released into the circulation. Malondialdehyde (MDA) is a low molecular weight aldehyde that is formed by free radical attacks on polyunsaturated fatty acids. MDA testing as a biomarker of oxidative damage was useful as a favorable prognostic tool, e.g., sepsis or nephrotic syndrome [2, 3]. Protein sulfhydryl groups (PSH) predominate in serum in contrast to the intracellular space, which mainly consists of low molecular weight thiols . Since reduced thiols are readily oxidized by reactive oxygen species, once oxidized, the thiols are less readily reduced in serum compared to their intracellular counterparts. Thus, free thiol depletion in serum reflects relatively stable systemic redox status. Recently published data revealed a prognostic role of serum free thiols in the general population . The risk assessment in the outpatient population, despite stability of symptoms, is important to escalate the therapy in the appropriate time. The data analyzing the dependence prognosis on ischemic or nonischemic cardiomyopathy causes of heart failure come from many years ago . In the Studies of Left Ventricular Dysfunction (SOLVD) including patients with heart failure, the etiology of the disease (ischemic or nonischemic) did not influence mortality in a one-year follow-up . However, in most studies, the ischemic group had poorer prognosis [8, 9].
The mechanisms of oxidative stress and antioxidant defense are partially different in the ICM and the nICM patients . Therefore, the aim of this study was to examine the prognostic value of clinical factors, oxidation-reduction (redox) biomarkers with special consideration of MDA and PSH, in two large cohorts of HF patients depending on ischemic or/and nonischemic etiology in a 1-year follow-up.
2. Study Group and Methods
2.1. The Inpatient
A clinic cohort with reduced ejection fraction heart failure (HFrEF) and chronic heart failure symptoms was considered potential candidates for heart transplantation and follow-up in the Prospective Registry of Heart Failure since 2003. Patients were stable and received optimal medical pharmacotherapy according to contemporary guidelines for at least 3 months before inclusion. Exclusion criteria were as follows: inability or unwillingness to provide informed consent, alcohol abuse or known antioxidant supplementation, and noncardiac diseases that affect life expectancy as judged by the treating physician. The data of 707 participants who had completed clinical laboratory assessment were included into the final analysis.
2.2. Endpoint of the Study
The endpoint of the study was all-cause mortality or urgent heart transplantation.
2.3. Clinical Assessments
Patients were characterized as an ischemic or nonischemic cardiomyopathy group according to the definition proposed by Felker et al. . Besides any history of myocardial infarction or coronary revascularization, the basis for the qualification was the results of angiography performed within the last six months. At the time of study entry, detailed clinical data were obtained using a questionnaire. History of smoking was defined as current or previous use of tobacco products. Comorbidities such as hypertension, diabetes mellitus, or hypercholesterolemia were recognized based on actual measurements of respective variables, current medication, and clinical history. The body mass index (BMI) was calculated from mass and height measured on the day of inclusion visit. The functional capacity was assessed by NYHA classification and exercise maximum O2 uptake in the cardiopulmonary testing (MaxVO2) . Two-dimensional transthoracic echocardiography was performed in all patients, and echocardiographic parameters were acquired in standard views as in the recommendation of the American Society of Echocardiography Committee . A follow-up on patients was obtained via direct or phone contact with patients or their family every 6 months by a research personnel. For some patients not contacted through this mechanism, the exact data of death were obtained from the national identification number database. Prior to enrolment in the study, all participants provided written informed consent.
2.4. Biochemical Methods
Venous blood samples were obtained from each patient at the enrollment process. Each serum sample was separated by centrifugation at 1500 g for 10 minutes at 4°C. UA, bilirubin, lipid parameters, blood hemoglobin, and serum iron, sodium, creatinine, glucose, and albumin concentrations were measured with the use of the colorimetric method (Roche, Cobas 6000 e 501). The chemiluminescence method was used to determine NT-proBNP concentration (Roche, Cobas 6000 e 501). The serum for determining the oxidant parameters was frozen at -70°C until assayed. Total oxidant status (TOS) was measured by a spectrophotometric method by Erel. This method is based on the oxidation of Fe2+ ions to form Fe3+ in acidic environment and consists of measuring the color intensity of Fe3+ ion complexes with xylenol orange. The measurements were performed using EM280 biochemical analyzer. TOS was expressed in mmol/L. . TAC was measured by Erel’s colorimetric methods with the use of 2,2-azino-bis (3-ethylbenzothiazoline-6-sulfonate) (ABTS+) . In this reaction, reduced ABTS, a colorless molecule, is oxidized to blue-green ABTS+. The oxidized form of ABTS is reduced to the original colorless reduced form as a result of reactions with oxidizable substances. TAC was expressed in mmol/L. The sulfhydryl groups (PSH) were determined with the use of 5,5-dithiobis (2-nitrobenzoic acid) (DTNB) in the method described by Koster. In this method, DTNB, after reduction by the sulfhydryl group-containing compounds, forms a yellow-colored anionic 5-thio-2-nitrobenzoic acid. The absorbance was measured with a Shimadzu 1700 UVVIS spectrophotometer at a wavelength of 412 nm . PSH concentration was expressed in μmol/g protein. Malondialdehyde (MDA) was measured by Ohkawa et al.’s spectrofluorimetric method. In this method, lipid peroxides react with thiobarbituric acid at the excitation wavelength 515 nm and emission wavelength 552 nm. The standard curve, prepared for 1,1,3,3-tetraethoxypropane, was used to calculate MDA concentration in μmol/L .
2.5. Statistical Analysis
The subjects were allocated into groups according to the etiology of cardiomyopathy: ischemic (ICM) and nonischemic (nICM). Then, for the purposes of the analysis, in each group, the following patients were distinguished: the patient who achieved the endpoint (EP+) or without the endpoint (EP-) during one year of observation. Categorical data were displayed as proportions and were compared using the chi-square test with the Yates correction. Distribution of all continuous variables was evaluated by the Shapiro-Wilk test. Because of abnormal distribution of most continuous variables, the continuous data were presented as median with the first and third quartile. Data were tested with the Kruskal-Wallis ANOVA test; next, we assessed the differences between the selected groups using the nonparametric Mann Whitney test and the chi2 test.
The estimation of risk of death and urgent transplantation was performed using a Cox proportional hazards model for the ICM and nICM group. All demographic, clinical, echocardiography, and laboratory variables and also medication data presented in Tables 1 and 2 were included in a univariate Cox analysis. A multivariate model was constructed separately for both groups based on the variables significantly associated with EP occurrence in each group (univariate analysis, ). The results of the Cox analysis were reported as relative risks with corresponding 95% confidence intervals (CI). Cumulative survival over one year of follow-up was displayed using the Kaplan-Meier method, with comparison between groups depending on the cutoff value of concentrations of MDA, UA, and bilirubin or the MDA/PSH ratio calculated in Receiver Operation Characteristic (ROC) analysis, and the differences were tested for significance by the log-rank test. Results were considered statistically significant if . Statistical analysis was performed using Statistica 13.1 PL (TIBCO, Cracow, Poland).
3.1. Baseline Characteristics of Subgroups in Relation to Etiology and Endpoint
In general, ischemic cardiomyopathy was more common than nonischemic cardiomyopathy (61.5% versus 38.5%). In the group of 435 patients of ICM etiology, 78 (17.9%) deaths and HT occurred over a one-year follow-up. 44 (16.2%) patients achieved an endpoint in 272 patients of the nICM group (chi2, ). Baseline characteristics for patients enrolled in the study are shown in Table 1. Patients of the ICM group were older than those of the nICM group, but there were no differences between EP+ and EP-.
The percentage of patients in NYHA class III-IV was greater, and patients with implanted ICD were lower in EP+ groups in the ICM and nICM population. In both ICM and nICM, EP+ groups characterized a more enlarged left ventricle in echocardiography and maximum VO2 consumption in cardiopulmonary exercise testing and higher concentration of creatinine and NT-proBNP. In patients with ICM (EP+), lower ejection fraction in echocardiography, iron concentration, and higher HDL-cholesterol were observed.
The differences in redooxidative parameters are presented in Table 2. In patients with ICM (EP+), higher uric acid, bilirubin, and MDA concentrations and MDA/PSH ratio than those in patients with ICM (EP-) were indicated. Similarly, bilirubin and MDA concentrations and the MDA/PSH ratio but not uric acid were higher in the nICM (EP+) than in the nICM (EP-) group. Additionally, TAC was the most increased in nICM (EP+).
3.2. Association between Redox Reaction Parameters and Risk of Endpoint
3.2.1. Uni- and Multivariate Cox Regression Analyses
Demographic and clinical data, laboratory results, comorbidities, pharmacotherapy, and oxidative stress-related parameters were assessed as risk factors for death or urgent heart transplantation in a 1-year follow-up in uni- and multivariate Cox regression analyses as presented in Tables 3 and 4. In univariate Cox regression analysis, higher levels of uric acid, bilirubin, MDA, and MDA/PSA were associated with endpoint occurrence in ICM patients. All assessed oxidative stress biomarkers without PSH in univariate Cox regression analysis were associated with the risk of death and HT in nICM.
In order to evaluate oxidative stress parameters in the context of confounders, a final multivariate model was calculated including all significant clinical, laboratory, and medication predictors significant in univariate analysis. After adjusting for significant predictors, only maxVO2 uptake in the cardiopulmonary testing, left ventricle ejection fraction, ICD presence, higher uric acid concentration (increase of risk (IoR) by 2.4% per 10 μmol/L, ), and higher serum MDA concentration (IoR by 220% per mmol/L, ) were independent risk factors for death or heart transplantation in ICM. On the other hand, NT-proBNP, left ventricle ejection fraction, ICD presence, higher bilirubin concentration (increase of risk (IoR) by 3.4% per 1 μmol/L, ), and higher MDA/PSH ratio (IoR by 336%, ) were independently related to endpoint occurrence in nICM.
The Kaplan-Meier plot with a log-rank test for the outcome in ICM is shown in Figures 1 and 2. MDA and UA concentrations above the cutoff value obtained in ROC analysis were associated with poor outcome in ICM ( and , respectively). The Kaplan-Meier plot with a log-rank test for the endpoint occurrence in nICM is shown in Figures 3 and 4. The MDA/PSH ratio () and bilirubin concentration () above the cutoff value were related to risk of death and HT.
While many clinical and biochemical variables have not only diagnostic but also prognostic value in heart failure, the question is whether these variables are of similar value in patients with ischemic and nonischemic heart failure. The observational studies, randomized clinical trials, and retrospective analyses of hospital records revealed inconsistent data on the impact of the etiology of heart failure on prognosis [7–9, 18].
In most of reports, a higher risk of death in patients with ICM compared with patients with nICM was indicated. In our study, the endpoint occurrence (all-cause mortality and HT) in a one-year follow-up was similar in ICM and nICM groups. A previous study has shown that oxidative stress is related to the severity of heart failure, although the results have been different in patients with ischemic and nonischemic cardiomyopathy . According to our knowledge, this is the first study analyzing the influence of the reduction-oxidative balance parameters on the prognosis in patients with systolic heart failure, taking into account ischemic and nonischemic etiology. Independent redox biomarkers related to increased all-cause mortality and heart transplantation in a one-year follow-up were higher concentrations of uric acid and MDA in ICM. In contrast, higher levels of bilirubin and MDA/PSH were predictors of the endpoint in the nICM group. Surprisingly, NT-proBNP was a predictor only in the nICM and the maxVO2 and left ventricle ejection fraction only in the ICM group.
The results of our study showed that the sex and age of the patients did not affect the achievement of the one-year endpoint in both groups ICM and nICM. Anker et al. indicated that uric acid, age, and left ventricle ejection fraction, but not peak VO2, were predictive of impaired survival in European heart failure patients (60% of ischemic etiology) . Similarly, UA was a prognostic marker of all-cause mortality after adjustment to NYHA class and creatinine clearance in the population of heart failure patients (70% of ischemic etiology) as described by Jankowska et al. . Sakai et al. confirmed the important prognostic role of UA in a Japanese population. Both increased UA and BNP were associated with mortality independently from etiology, sex, age, NYHA class, and LVEF . Hyperuricemia may reflect raised xanthine oxidase activity in heart failure; this enzyme system is an important source of oxygen free radicals [22, 23]. Uric acid concentration is affected by using a xanthine oxidase inhibitor (allopurinol). Allopurinol was used more often in the nICM groups (EP+ and EP-) than, respectively, in the ICM groups (EP+ and EP-). The median uric acid concentration did not differ statistically between ICM and nICM; however, uric acid concentration was statistically significantly higher in the ICM EP+ than in ICM EP- (). Higher uric acid level was a marker associated with increased mortality in this group;
Our results indicating malondialdehyde, a by-product of polyunsaturated fatty acid lipid peroxidation, as a predictor of death and heart transplantation only in ICM, are consistent with observation published by Radovanovic et al. that malondialdehyde predicted mortality in patients with chronic ischemic heart failure during 13-month observation . The previous reports demonstrated increase in MDA concentration in patients with HF of different etiologies, but the relationship between MDA and the severity of heart failure assessed as NYHA class, exercise intolerance LVEF, and invasive hemodynamic measurements is inconsistent [25–30]. The last papers do not confirm such a relationship perhaps due to the use of standardized treatment with inhibitors of the renin-angiotensin system and beta-blockers [29, 30]. MDA, the highest quartile concentration, was an independent risk factor for a cardiovascular event (myocardial infarction—fatal, nonfatal, stroke, and unstable angina) as compared to the lowest quartile. The prognostic value was independent of interleukin-6, C-reactive protein, and classical risk factors for atherosclerosis . The above observations draw our attention to the lipid profile of patients with ICM. In lipid parameters, it was surprising that only the high concentration of HDL cholesterol in univariable analysis had a protective effect on survival in the ischemic group. However, the ICM EP+ group had a significantly higher LDL than the nICM EP+ group despite more frequent use of statins. It should be emphasized that patients were enrolled in the study before the current very strict LDL target recommendations . This fact may confirm the large role of LDL in the pathogenesis of coronary diseases in general. LDL cholesterol is oxidized by MDA to MDA-LDL molecules. These combinations have been shown to be particularly atherogenic [33, 34]. Tani et al. proved that the concentration of MDA-LDL is independent of the overall LDL. It may depend on the oxidative state of a patient individually. Their study showed that patients with coronary artery disease have increased levels of MDA-LDL. It has been suggested that this may be a promising indicator in assessing the risk of coronary artery disease .
In the ICM group, the PSH, MDA/PSH index, and NT-proBNP in multivariable Cox regression analysis indices showed no prognostic significance in predicting the endpoint. The situation is different in the case of patients with nICM, in whom the MDA/PSH ratio and NT-proBNP were indicated as risk factors for death or OHT. Increasing attention, for the last years, has been paid to the role of oxidative stress in CVD, since it may lead to endothelial dysfunction . Frenay et al. show that high serum concentrations of PSH are associated with a favorable cardiovascular risk profile and better survival for patients with the transplanted kidney . In acute myocardial infarction, it was shown that thiols are an oxidative stress marker  and that plasma thiols are significantly lower in subjects with congestive heart failure, as compared to healthy subjects . Recently, Abdulle et al. found that protein-adjusted serum free thiol concentrations were able to predict the risk of all-cause mortality and cardiovascular events after adjustment for confounders in the general population. In the subgroup of subjects with a known history of cardiovascular disease (), regression analyses revealed age- and sex-adjusted associations between protein-adjusted serum free thiol concentrations and all-cause mortality when comparing the upper two tertiles with the lowest tertile and lack of significant associations with the risk of cardiovascular events . In the study by Koning et al., they found in a group of 101 patients with stable heart failure that free serum thiols are associated independently with age, cholesterol, and parathyroid hormone. Free thiols and age were inversely related. In multivariable analysis, an association of PSH with NT-proBNP was not identified. Both serum free thiols per gram of protein below the mean and NT-proBNP above the median were predictors of a composite endpoint: HF-related rehospitalisation and all-cause mortality. However, further analysis adjusting PSH to establish prognostic factors in HF (age, eGFR, and NT-proBNP) revealed no longer significant association with the endpoint . 72% of patients in this study characterized ischemic etiology. The participants were older compared to our nICM group and had less reduced ejection fraction. Despite free thiols (PSH, sulfhydryl groups) are more accurately reflecting the systemic in vivo redox status as compared to many other individual oxidant or antioxidant factors and their derivatives , in our study, the MDA/PSH ratio but not PSH was a valuable predictor of adverse outcome in nICM.
The positive association of reduced thiol concentration with increased circulating bilirubin was indicated in unconjugated hyperbilirubinemia (Gilbert syndrome). The modulation by bilirubin lipid status may be the explanation of protection from ischemic heart disease in Gilbert syndrome . The epidemiological studies indicate that mildly elevated bilirubin concentrations were protective against cardiovascular disease and all-cause mortality [43–45]. However, we did not observe significance of bilirubin concentration with endpoint prediction in heart failure of ischemic etiology. Moreover, higher bilirubin was associated with adverse outcome in the nICM group.
It is disappointing to find that using allopurinol did not improve patients’ prognosis in any group of our study, although previous studies indicated that allopurinol may have the ability to decrease mortality in HF. [46, 47]. The explanation may be that the dose was too low, because higher doses of allopurinol have a greater impact on cardiovascular outcomes than lower doses [47, 48]. Oxidation-reduction systems play an important role in the progression of heart failure. It has been proven that some of the commonly used drugs in HF, such as ACE inhibitors or beta-blockers, affect this system [49, 50]. Pharmacological treatment of HF recommended by guidelines has progressed over the past decades and has improved the prognosis in this group of patients. Recently, the importance of personalizing treatment has been emphasized because the population enrolled to the randomized trials does not always represent the patients we treat. In most HF clinical trials, patients predominantly were male with ischemic etiology. Thus, we decided for separate analysis of each group and we indicated that only the ICD presence was a common prognostic factor for both groups. The others (max VO2, LVEF, uric acid, and MDA) were risk factors for the endpoint in ischemic etiology whereas thiazide diuretic therapy, concentration of NT-proBNP, bilirubin, and MDA/PSH ratio in the nonischemic group. Our results proved influence of different redox biomarkers on one-year prognosis in HF cohorts of ischemic and nonischemic etiology. Nowadays, it is usually difficult to predict which patients will benefit most from the different therapy; maybe when we pay attention on deviations in redox biomarkers, it would help us in qualifying patients for the use of, e.g., allopurinol, as donors of SH groups.
The original data is available after contact with the corresponding author.
Conflicts of Interest
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work was funded by the Medical University of Silesia (grant no. KNW-1/096/K/8/0), Poland.
L. Lorente, M. M. Martín, and P. Abreu-González, “Prognostic value of malondialdehyde serum levels in severe sepsis: a multicenter study,” PLoS One, vol. 8, no. 1, article e53741, 2013.View at: Google Scholar
E. Romuk, C. Wojciechowska, and W. Jacheć, “Comparison of oxidative stress parameters in heart failure patients depending on ischaemic or nonischaemic aetiology,” Oxidative medicine and cellular longevity, vol. 2019, Article ID 7156038, 2019.View at: Google Scholar
The Criteria Committee of the New York Heart Association, Nomenclature and Criteria for Diagnosis of Diseases of the Heart and Blood Vessels, Little Brown, Boston, Mass, USA, 1964.
R. M. Lang, M. Bierig, R. B. Devereux et al., “Recommendations for chamber quantification: a report from the American Society of Echocardiography's Guidelines and Standards Committee and the Chamber Quantification Writing Group, developed in conjunction with the European Association of Echocardiography, a branch of the European Society of Cardiology,” Journal of the American Society of Echocardiography, vol. 18, no. 12, pp. 1440–1463, 2005.View at: Publisher Site | Google Scholar
C. Lourenço, F. Saraiva, H. Martins et al., “Ischemic versus non-ischemic cardiomyopathy: are there differences in prognosis? Experience of an advanced heart failure center,” Revista Portuguesa de Cardiologia, vol. 30, no. 2, pp. 181–197, 2011.View at: Google Scholar
H. Sakai, T. Tsutamoto, T. Tsutsui, T. Tanaka, C. Ishikawa, and M. Horie, “Serum level of uric acid, partly secreted from the failing heart, is a prognostic marker in patients with congestive heart failure,” Circulation Journal, vol. 70, no. 8, pp. 1006–1011, 2006.View at: Publisher Site | Google Scholar
S. Radovanovic, A. Savic-Radojevic, M. Pljesa-Ercegovac et al., “Markers of oxidative damage and antioxidant enzyme activities as predictors of morbidity and mortality in patients with chronic heart failure,” Journal of Cardiac Failure, vol. 18, no. 6, pp. 493–501, 2012.View at: Publisher Site | Google Scholar
C. Wojciechowska, E. Romuk, A. Tomasik et al., “Oxidative stress markers and C-reactive protein are related to severity of heart failure in patients with dilated cardiomyopathy,” Mediators of Inflammation, vol. 2014, Article ID 147040, 2014.View at: Google Scholar
M. F. Walter, R. F. Jacob, B. Jeffers et al., “Serum levels of thiobarbituric acid reactive substances predict cardiovascular events in patients with stable coronary artery disease: a longitudinal analysis of the PREVENT study,” Journal of the American College of Cardiology, vol. 44, no. 10, pp. 1996–2002, 2004.View at: Publisher Site | Google Scholar
S. Tani, I. Watanabe, T. Anazawa et al., “Effect of Pravastatin on Malondialdehyde-Modified Low-Density Lipoprotein Levels and Coronary Plaque Regression as Determined by Three-Dimensional Intravascular Ultrasound,” The American Journal of Cardiology, vol. 96, no. 8, pp. 1089–1094, 2005.View at: Publisher Site | Google Scholar
K. C. Gracia, D. Llanas-Cornejo, and H. Husi, “CVD and oxidative stress,” Journal of Clinical Medicine, vol. 6, no. 2, 2017.View at: Google Scholar
M. M. Cortese-Krott, A. Koning, G. G. C. Kuhnle et al., “The reactive species interactome: evolutionary emergence, biological significance, and opportunities for redox metabolomics and personalized medicine,” Antioxidants & Redox Signaling, vol. 27, no. 10, pp. 684–712, 2017.View at: Publisher Site | Google Scholar
I. T. Mak, A. M. Freedman, B. F. Dickens, and W. B. Weglicki, “Protective effects of sulfhydryl-containing angiotensin converting enzyme inhibitors against free radical injury in endothelial cells,” Biochemical Pharmacology, vol. 40, no. 9, pp. 2169–2175, 1990.View at: Publisher Site | Google Scholar