- About this Journal
- Abstracting and Indexing
- Aims and Scope
- Article Processing Charges
- Articles in Press
- Author Guidelines
- Bibliographic Information
- Citations to this Journal
- Contact Information
- Editorial Board
- Editorial Workflow
- Free eTOC Alerts
- Publication Ethics
- Reviewers Acknowledgment
- Submit a Manuscript
- Subscription Information
- Table of Contents
Journal of Transplantation
Volume 2012 (2012), Article ID 120702, 8 pages
Tricuspid Valve Regurgitation after Orthotopic Heart Transplantation: Prevalence and Etiology
1Heart Transplantation Unit, Department of Cardiac Surgery, Leviev Heart Center, Sheba Medical Center, and the Sackler Faculty of Medicine, Tel Aviv University, 52621 Ramat Gan, Israel
2Heart Institute, Leviev Heart Center, Sheba Medical Center, and the Sackler Faculty of Medicine, Tel Aviv University, 52621 Ramat Gan, Israel
Received 12 May 2012; Revised 13 July 2012; Accepted 27 July 2012
Academic Editor: Andreas Zuckermann
Copyright © 2012 Yaniv Berger et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background. Tricuspid valve regurgitation (TR) after orthotopic heart transplantation (OHT) is common. The aims of this study were to determine the prevalence of TR after OHT, to examine the correlation between its development and various variables, and to determine its outcomes. Methods. All 163 OHT patients who were followed up between 1988 and 2009 for a minimal period of 12 months were divided into those with no TR/mild TR and those with at least mild-moderate TR, as assessed by doppler echocardiography. These groups were compared regarding preoperative hemodynamic variables, surgical technique employed, number of endomyocardial biopsies, number of acute cellular rejections, incidence of graft vasculopathy, and clinical outcomes. Results. At the end of the followup (average 8.2 years) significant TR was evident in 14.1% of the patients. The development of late TR was found by univariate, but not multivariate, analysis to be significantly correlated with the biatrial surgical technique () and the presence of graft vasculopathy (). TR development was found to be correlated with the need for tricuspid valve surgery but not with an increased mortality. Conclusions. The development of TR after OHT may be related to the biatrial anastomosis technique and to graft vasculopathy.
Tricuspid regurgitation (TR) after orthotopic heart transplantation (OHT) is common, with reported prevalence that varies from 19% to 84% . The prevalence and severity of TR increase with the length of followup. In most cases TR is mild and asymptomatic, but some cases of moderate or severe TR are related to morbidity and mortality [1–5]. Doppler echocardiography is the most common technique used for the detection and evaluation of severity of TR [6, 7]. The treatment of severe symptomatic TR is mainly conservative with diuretics. In refractory cases there is an indication for tricuspid valve repair or replacement surgery. The etiology of TR after OHT is unclear, and several variables have been reported to be related, including the surgical anastomosis technique employed (biatrial versus bicaval) [8–13], iatrogenic damage due to endomyocardial biopsies (EMBs) [8, 14–18], number of acute cellular rejection episodes (ACRs) [8, 14], pretransplant pulmonary hypertension [8, 19, 20], discordance between the size of the donor’s heart and the recipient’s pericardial cavity , and cardiac allograft vasculopathy (CAV) . Preventive measures mentioned in the literature include prophylactic tricuspid annuloplasty during OHT [22–24], the use of a long bioptome sheath during EMB , and the use of noninvasive methods to monitor for graft rejections .
The aims of this study were to determine the short and long term prevalence of TR after OHT, to examine the correlation between its development and the above-mentioned variables, and to determine its clinical outcomes.
2. Material and Methods
The study is a retrospective cohort study of all 163 patients who underwent OHT between 1988 and 2009 and were followedup at the heart transplant clinic at the Sheba Medical Center for a minimal period of 12 months. The data of 10 aditional patients who underwent heart transplantations in China between 2005 and 2007 was excluded in accordance with current ethical guidelines of transplantation societies and journals. Patients were divided into two groups based on the severity of their postoperative TR at the end of the follow-up period at an average of 8.2 years: 140 patients with no TR or mild TR and 23 patients with significant (at least mild to moderate) TR. The two groups were compared regarding preoperative hemodynamic variables, surgical technique employed (biatrial versus bicaval anastomosis), number of EMBs taken, number of ACRs, incidence of CAV, right heart failure, and clinical outcome.
2.1. Evaluation of TR
Patients underwent routine echocardiographic follow-up examinations immediately after transplantation, at 3 months, 12 months, and annually thereafter. Severity of TR was assessed by comparing the ratio of TR jet area to the right atrial area on color Doppler and was scored from 0 to 3 : 0—no TR, 1—mild TR, 2—moderate TR, and 3—severe TR. Mild to moderate TR was scored 1.5 and moderate to severe TR was scored 2.5. Additional echocardiographic parameters reviewed included left ventricular ejection fraction, right ventricular size and function, and estimated pulmonary arterial pressure.
2.2. Invasive Evaluation
Every patient underwent routine right heart catheterization before transplantation, one week after transplantation and annually thereafter in which mean pulmonary arterial pressure, pulmonary vascular resistance, and right atrial pressure were recorded. The incidence of CAV was evaluated by annual coronary angiography.
EMBs were performed weekly during the first month after transplantation, biweekly during the second and third months, at the fourth, fifth, sixth, ninth, and twelfth months after transplantation, and annually following the first year. Further biopsies were performed when clinical suspicion of graft rejection was raised. Biopsies were evaluated for rejection using the revised ISHLT (International Society of Heart and Lung Transplantation) criteria .
2.3. Statistical Analysis
Patient data were analyzed with the Statistical Program of Social Sciences (SPSS version 19). Categorical data are expressed as percentages and continuous data are expressed as mean ± standard deviation. The prevalence of late irreversible TR and the survival difference between the two study groups were analysed according to the Kaplan-Meier method. Univariate analyses were used to compare differences between variables obtained in the two groups of patients (patients with no/mild TR versus patients with at least mild-moderate TR). We have conducted a nonparametric Kolmogorov-Smirnov test for normal distribution; based on the results, we used independent -test or Mann-Whitney test to compare continuous data (number of EMBs, number of ACRs, and preoperative hemodynamic variables) between the two groups; we presented median (and range) for variables which depart from normal distribution. Categorical data (CAV, surgical anastomosis technique and mortality rates) were compared by test. A multivariate analysis was performed by using the Cox regression model to identify risk factors for TR. A value of less than 0.05 was defined as significant.
3.1. Patients Demographics and Perioperative Data
There were no statistically significant differences between the two groups in the following preoperative demographic or clinical characteristics: gender, age, etiology of heart failure, or incidence of previous heart surgery (Table 1). Of the total 163 patients, 55.8% underwent the transplant at the Sheba Medical Center in Israel and the rest in other countries, with a statistically significant difference between the two groups (Table 1). The average total heart ischemic time during transplantation, as well as the average cardiopulmonary bypass time, was similar between the two groups (Table 1). The average follow-up period was significantly longer in the group of patients with significant TR compared to the group of patients with insignificant TR (Table 1).
3.2. TR Prevalence
Significant TR prevalence peaked at 34% immediately after transplantation (average days), decreased to a nadir of 6.4% after 3 years, and then increased gradually (Figure 1). At the end of the followup, 23 patients (14.1% of the study population) developed significant TR. The average length of time between transplantation and the development of significant irreversible TR was 6.8 ± 4.8 years.
A Kaplan-Meier analysis of freedom from late significant TR revealed that 10 years after transplant 85.2% were permanently free of significant TR (Figure 2).
3.3. Risk Factors for TR
Univariate analysis of risk factors for early (average 14.0 ± 11.8 days) development of significant TR (Table 2) yielded only higher values of pretransplant mean pulmonary arterial pressure. The relationship between early TR and pretransplant pulmonary vascular resistance was of borderline significance. The surgical technique—biatrial or bicaval anastomosis—did not have an impact on the development of early TR.
Univariate analysis of risk factors for the late development of significant TR at the end of the followup is depicted in Table 3. The average total number of EMBs taken during the follow-up period was significantly higher in the group that developed significant TR at the end of the follow-up period; however the average number of EMBs taken prior to the development of significant TR was similar in both groups.
The median of ACR episodes with any grade equal or higher than ISHLT grade 1R was similar in both groups, as well as the median of ACR episodes with ISHLT grade 2R or higher.
Hemodynamically, the significant TR group displayed higher pretransplant values of mean pulmonary arterial pressure and pulmonary vascular resistance compared with the no/mild TR group, and these differences were of borderline significance ( and , resp.).
The bicaval surgical anastomosis technique was employed in 55.8% of the patients and the biatrial technique was employed in 35%; in 9.2% of the patients (all of them underwent OHT in foreign countries) the surgical technique employed could not be determined due to missing surgery reports. The proportion of patients who underwent OHT by using the biatrial surgical technique was significantly higher in the significant TR group (65% compared with 34.4% in the no/mild TR group).
During the follow-up period 60 patients of the study population (39.7%) developed CAV of any kind (including insignificant disease of one or more coronary arteries). Significant CAV (defined as significant stenosis of at least one coronary vessel) was found in 48 patients (31.8%) and two or three vessels CAV was diagnosed in 36 patients (23.8%). The prevalence of CAV in the significant TR group was higher compared with the no/mild TR group (Table 3).
In multivariate analysis none of the explored variables were found to be an independent risk factor for the development of late significant TR. The explored variables included total number of EMBs taken, total number of ACRs with at least ISHLT grade 1R, the surgical technique employed, and incidence of CAV.
3.4. Clinical Outcomes of TR
The relationship between significant TR at the end of the follow-up period and clinical outcomes is depicted in Table 4. The total mortality rate over the follow-up period (average 8.2 years) was 31.3%. In univariate analysis, the mortality rate during the follow up period was higher in the significant TR group (47.8%) compared with the no/mild TR group (28.6%), and this difference was of borderline significance (). A Kaplan-Meier survival analysis did not demonstrate a significant difference between groups (Figure 3).
No significant differences were observed between the two groups in the median serum creatinine level at the end of the followup (Table 4).
During the follow-up period 8 patients underwent repeat heart surgery: 6 patients in the significant TR group and two patients in the no/mild TR group. Of the six patients in the significant TR group, one patient underwent tricuspid annuloplasty (8.7 months after OHT), two patients underwent tricuspid valve replacement (1.0 and 13.4 years after OHT), two patients underwent combined replacement of the tricuspid and mitral valves (5.3 and 8.8 years after OHT), and one patient underwent superior vena cava thrombectomy. In the no/mild TR group one patient underwent implantation of biventricular assist device as bridge to retransplantation and one patient underwent coronary artery bypass grafting.
The relationship between the development of significant TR at the end of the follow-up period and echocardiographic parameters (as measured in the last echocardiographic exam) is depicted in Figure 4. Patients in the significant TR group showed significantly higher values of estimated systolic pulmonary artery pressure, lower left ventricular ejection fraction, increased right ventricular dilatation, and worse levels of right ventricular dysfunction compared to the no/mild TR group.
TR after OHT is a common problem, varying in prevalence between 5.5% and 54% (Table 5). Although comparing different series may be problematic because studies vary in the length of followup, in the definition of significant TR and in the technique used for TR diagnosis, all reported series except one  demonstrated a higher prevalence of TR at the end of the followup in comparison to our study (14.1%). Marelli et al.  in a cohort of 670 patients found freedom from significant TR of 78% at 9 years. Our analysis demonstrated a slightly higher rate of 85.2% at 10 years (Figure 2), similar to the 85.8% at 10 years rate demonstrated by Chan et al. .
Regardless of the incidence of its occurrence, all reported series [2, 4, 5, 14] have demonstrated an increased mortality among patients who have developed postoperative significant TR, ranging from 15% to 62.5%. Similarly, while the overall mortality rate during the follow-up period (average 8.2 ± 4.6 years) in our study was 31.3%, similar to the reported ISHLT registry data , the mortality rate in the significant TR group was higher compared with the no/mild TR group (47.8% versus 28.6%), and this difference was of borderline significance (P = 0.065); the Kaplan-Meier survival analysis did not demonstrate a significant difference between the two groups in our study. Moreover, the development of significant TR is associated with the significant morbidity of the right heart failure and with the increased necessity to undergo repeat operations to repair or replace the regurgitant tricuspid valve, as evident in our series. Thus it is of paramount importance to identify the risk factors associated with this potentially lethal complication in order to try and avoid them.
The etiology of postoperative TR after OHT is no doubt multifactorial and several preoperative, intraoperative, and postoperative features have been implicated as potential causative factors. Several studies have reported a correlation between high pretransplant pulmonary vascular resistance or pulmonary arterial pressure and development of TR [8, 19, 20]. The suggested mechanism is the postsurgical right ventricular dysfunction and dilatation due to increased afterload. Our study has demonstrated a borderline significant correlation between high pretransplant values of pulmonary arterial pressure and pulmonary vascular resistance and the development of late TR (Table 3); we have also found a statistically significant correlation between high pretransplant values of pulmonary arterial pressure and early TR (Table 2), implying that elevated pulmonary arterial pressure and resistance may be risk factors for developing early and late TR after OHT.
Many studies have found a correlation between the biatrial surgical technique and the development of TR after OHT [8, 10–13]. The suggested mechanism in this case is geometric distortion of the atrioventricular annular ring and malcoaptation of the valve leaflets. Our study has also demonstrated this correlation with patients who underwent the biatrial anastomosis technique showing significantly higher incidence of late significant TR compared to patients who underwent the bicaval anastomosis (Table 3). The lack of significant difference in our series in the incidence of significant TR between the two anastomosis techniques early after the operation could be attributed to the residual high pulmonary artery pressures and resistance in the immediate postoperative period which resolves over time.
Many studies point to a possible relationship between EMBs and the development of TR. Nguyen et al.  reported a TR prevalence of 60% at the end of the followup among patients who underwent more than 31 EMBs compared to 0% among patients who underwent less than 18 EMBs and in a multivariate analysis found that only the total number of EMBs was an independent risk factor for TR severity. Lo et al.  have demonstrated correlation between EMBs number greater than 10 and iatrogenic damage to the tricuspid valve. Our study has demonstrated no correlation between the average number of EMBs taken prior to the development of significant TR, and in fact this number was actually higher in the no/mild TR group (Table 2). The increased total number of EMBs found in the significant TR group in our study simply reflects the longer follow-up period of the significant TR group. Among our total study population, only two patients were found by echocardiography to have developed iatrogenic damage to the tricuspid valve during the follow-up period, as evident by flail tricuspid leaflet in both cases and only one of these patients developed significant TR.
A few studies have found a correlation between the number of ACRs and TR. Aziz et al.  have shown a statistically significant relationship between ACRs number greater than or equal to ISHLT grade 2 and early and late TR. Hausen et al.  have demonstrated a correlation between TR and the number of ACRs during the first postoperative year. In our study late TR was not found to be correlated with the number of ACRs equal or greater than ISHLT grade 1R, and there was also no difference in the number of ACRs with grade 2R or higher among the two groups (Table 3).
The relationship between CAV and the development of TR has not been reported in previous studies and only one study  has found such a relationship. Our study has demonstrated a strong link in univariate analysis between CAV and late TR (Table 3) but no such link in multivariate analysis. As CAV is a well-known complication of the long-term course after transplantation, it is unclear whether this link with late TR is causative or purely incidental and time dependent. At the end of the followup the measured left ventricular ejection fraction in our series was lower in the significant TR group compared to the group with no TR. This unexpected finding, together with the increased levels of right ventricular dilatation and dysfunction (Figure 4), might hint to a common etiology of graft vasculopathy.
Our study has several limitations. The study is retrospective, and therefore some data regarding early postoperative parameters were missing, particularly for patients transplanted in foreign countries. The study population is heterogenous due to this retrospective design—some patients underwent OHT 20 years before other patients, so different patients were exposed to different treatment protocols. The grading of TR was assessed by measuring the size of regurgitant jet area on color Doppler—this technique, although used in other similar studies, is not quantitative. The average follow-up period of the significant TR group was significantly longer compared with the insignificant TR group, and as TR prevalence increases with time, this difference between groups can act as a confounder; this difference can also explain the similar survival curves of both groups (Figure 3) despite the higher mortality rate among patients with significant TR.
In conclusion, the results of our series have demonstrated that the development of TR after OHT is probably related to pretransplantation increased pulmonary artery pressure and pulmonary vascular resistance, biatrial anastomosis technique, and maybe to the development of graft vasculopathy. It is probably not related to the total number of EMBs and the number of ACRs. The development of TR is probably associated with increased mortality but definitely with the need for a repeat tricuspid surgery.
This work was performed by Yaniv Berger in partial fulfillment of the M.D. thesis requirement of the Sackler Faculty of Medicine, Tel Aviv University.
- R. C. C. Wong, Z. Abrahams, M. Hanna et al., “Tricuspid regurgitation after cardiac transplantation: an old problem revisited,” Journal of Heart and Lung Transplantation, vol. 27, no. 3, pp. 247–252, 2008.
- T. M. Aziz, R. A. Saad, M. I. Burgess, C. S. Campbell, and N. A. Yonan, “Clinical significance of tricuspid valve dysfunction after orthotopic heart transplantation,” Journal of Heart and Lung Transplantation, vol. 21, no. 10, pp. 1101–1108, 2002.
- M. C. Y. Chan, N. Giannetti, T. Kato et al., “Severe tricuspid regurgitation after heart transplantation,” Journal of Heart and Lung Transplantation, vol. 20, no. 7, pp. 709–717, 2001.
- D. Marelli, F. Esmailian, S. Y. Wong et al., “Tricuspid valve regurgitation after heart transplantation,” Journal of Thoracic and Cardiovascular Surgery, vol. 137, no. 6, pp. 1557–1559, 2009.
- G. Sahar, A. Stamler, E. Erez et al., “Etiological factors influencing the development of atrioventricular valve incompetence after heart transplantation,” Transplantation Proceedings, vol. 29, no. 6, pp. 2675–2676, 1997.
- W. A. Zoghbi, M. Enriquez-Sarano, E. Foster et al., “Recommendations for evaluation of the severity of native valvular regurgitation with two-dimensional and Doppler echocardiography,” Journal of the American Society of Echocardiography, vol. 16, no. 7, pp. 777–802, 2003.
- A. Mugge, W. G. Daniel, G. Herrmann, R. Simon, and P. R. Lichtlen, “Quantification of tricuspid regurgitation by Doppler color flow mapping after cardiac transplantation,” American Journal of Cardiology, vol. 66, no. 10, pp. 884–887, 1990.
- T. M. Aziz, M. I. Burgess, A. N. Rahman, C. S. Campbell, A. K. Deiraniya, and N. A. Yonan, “Risk factors for tricuspid valve regurgitation after orthotopic heart transplantation,” Annals of Thoracic Surgery, vol. 68, no. 4, pp. 1247–1251, 1999.
- R. De Simone, R. Lange, F. U. Sack, H. Mehmanesh, and S. Hagl, “Atrioventricular valve insufficiency and atrial geometry after orthotopic heart transplantation,” Annals of Thoracic Surgery, vol. 60, no. 6, pp. 1686–1693, 1995.
- A. Koch, A. Remppis, T. J. Dengler, P. A. Schnabel, S. Hagl, and F. U. Sack, “Influence of different implantation techniques on AV valve competence after orthotopic heart transplantation,” European Journal of Cardio-Thoracic Surgery, vol. 28, no. 5, pp. 717–723, 2005.
- J. P. Sun, J. Niu, M. K. Banbury et al., “Influence of different implantation techniques on long-term survival after orthotopic heart transplantation: an echocardiographic study,” Journal of Heart and Lung Transplantation, vol. 26, no. 12, pp. 1243–1248, 2007.
- K. Y. Park, C. H. Park, Y. B. Chun, M. S. Shin, and K. C. Lee, “Bicaval anastomosis reduces tricuspid regurgitation after heart transplantation,” Asian Cardiovascular and Thoracic Annals, vol. 13, no. 3, pp. 251–254, 2005.
- M. Schnoor, T. Schäfer, D. Lühmann, and H. H. Sievers, “Bicaval versus standard technique in orthotopic heart transplantation: a systematic review and meta-analysis,” Journal of Thoracic and Cardiovascular Surgery, vol. 134, no. 5, pp. 1322–1331, 2007.
- B. Hausen, J. M. Albes, R. Rohde, S. Demertzis, A. Mügge, and H. J. Schäfers, “Tricuspid valve regurgitation attributable to endomyocardial biopsies and rejection in heart transplantation,” Annals of Thoracic Surgery, vol. 59, no. 5, pp. 1134–1140, 1995.
- C. Y. Lo, H. H. Chang, C. P. Hsu, S. T. Lai, and C. C. Shih, “Endomyocardial biopsy-related tricuspid regurgitation after orthotopic heart transplantation: single-center experience,” Journal of the Chinese Medical Association, vol. 70, no. 5, pp. 185–192, 2007.
- V. Nguyen, M. Cantarovich, R. Cecere, and N. Giannetti, “Tricuspid regurgitation after cardiac transplantation: how many biopsies are too many?” Journal of Heart and Lung Transplantation, vol. 24, supplement 7, pp. S227–S231, 2005.
- L. Mielniczuk, H. Haddad, R. A. Davies, and J. P. Veinot, “Tricuspid valve chordal tissue in endomyocardial biopsy specimens of patients with significant tricuspid regurgitation,” Journal of Heart and Lung Transplantation, vol. 24, no. 10, pp. 1586–1590, 2005.
- M. J. A. Williams, M. Y. Lee, T. G. DiSalvo et al., “Biopsy-induced flail tricuspid leaflet and tricuspid regurgitation following orthotopic cardiac transplantation,” American Journal of Cardiology, vol. 77, no. 15, pp. 1339–1344, 1996.
- S. J. S. Bhatia, J. M. Kirshenbaum, R. J. Shemin et al., “Time course of resolution of pulmonary hypertension and right ventricular remodeling after orthotopic cardiac transplantation,” Circulation, vol. 76, no. 4, pp. 819–826, 1987.
- M. K. Lewen, R. J. Bryg, and L. W. Miller, “Tricuspid regurgitation by Doppler echocardiography after orthotopic cardiac transplantation,” American Journal of Cardiology, vol. 59, no. 15, pp. 1371–1374, 1987.
- A. Haverich, J. M. Albes, G. Fahrenkamp, H. J. Schäfers, T. Wahlers, and B. Heublein, “Intraoperative echocardiography to detect and prevent tricuspid valve regurgitation after heart transplantation,” European Journal of Cardio-Thoracic Surgery, vol. 5, no. 1, pp. 41–45, 1991.
- N. E. Brown, G. F. Muehlebach, P. Jones, M. E. Gorton, R. S. Stuart, and A. M. Borkon, “Tricuspid annuloplasty significantly reduces early tricuspid regurgitation after biatrial heart transplantation,” Journal of Heart and Lung Transplantation, vol. 23, no. 10, pp. 1160–1162, 2004.
- V. Jeevanandam, H. Russell, P. Mather, S. Furukawa, A. Anderson, and J. Raman, “Donor tricuspid annuloplasty during orthotopic heart transplantation: long-term results of a prospective controlled study,” Annals of Thoracic Surgery, vol. 82, no. 6, pp. 2089–2095, 2006.
- A. I. Fiorelli, N. A. G. Stolf, C. A. C. Abreu Filho et al., “Prophylactic donor tricuspid annuloplasty in orthotopic bicaval heart transplantation,” Transplantation Proceedings, vol. 39, no. 8, pp. 2527–2530, 2007.
- A. C. Yankah, M. Musci, Y. Weng et al., “Tricuspid valve dysfunction and surgery after orthotopic cardiac transplantation,” European Journal of Cardio-Thoracic Surgery, vol. 17, no. 4, pp. 343–348, 2000.
- S. Stewart, G. L. Winters, M. C. Fishbein et al., “Revision of the 1990 working formulation for the standardization of nomenclature in the diagnosis of heart rejection,” Journal of Heart and Lung Transplantation, vol. 24, no. 11, pp. 1710–1720, 2005.
- R. J. C. Chen, J. Wei, C. Y. Chang et al., “Tricuspid valve regurgitation and endomyocardial biopsy after orthotopic heart transplantation,” Transplantation Proceedings, vol. 40, no. 8, pp. 2603–2606, 2008.
- C. B. Huddleston, M. Rosenbloom, J. A. Goldstein, and M. K. Pasque, “Biopsy-induced tricuspid regurgitation after cardiac transplantation,” Annals of Thoracic Surgery, vol. 57, no. 4, pp. 832–837, 1994.
- J. Stehlik, L. B. Edwards, A. Y. Kucheryavaya et al., “The registry of the international society for heart and lung transplantation: twenty-eighth adult heart transplant report-2011,” Journal of Heart and Lung Transplantation, vol. 30, no. 10, pp. 1178–1194, 2011.